Quick Links

ShortTalk and EmacsListen

UPDATE See ShortTalk Executive Summary.

ShortTalk aspires to be a perfect and universal spoken language for editing and other intensive computer-mediated tasks. EmacsListen, the GNU Emacs code base in this project, implements ShortTalk and addresses key usability issues in dictation systems.

See ShortTalk Executive Summary, Video Demo, or Audio Demo.

EmacsListen is a software that turns GNU Emacs into a voice-enabled editor. It comes with a grammar for ShortTalk, but its generic notion of voice event and extensive support for text identification and normalization make it suitable for implementing other spoken interfaces.

EmacsListen provides:

a command language grammar notation for composite commands and commands that take natural language fragments as arguments;
a grammar for ShortTalk along with Emacs functionality for all ShortTalk concepts;
built-in support for common technical formats, like XML, LaTeX, and all programming modes;
a parser for speech recognizer output that identifies commands in a continuous stream of text tokens;
a notion of voice event and a powerful undo mechanism;
a sophisticated spacing engine for inserting spaces and for automatic capitalization;
advanced text identification heuristics for manipulating text pieces such as words, parenthesized expressions, and quoted strings;
numerous Emacs specific speech enhancements such as colored peek windows for buffer selection and automatic enabling of search mode in read-only buffers;
an interface to the speech engine of NaturallySpeaking, a product by ScanSoft, Inc; and
a logging mechanism for personalized language modeling and for research purposes (for calculation of active command vocabulary and entropy rate estimation)

See EmacsListen for more information about the software. The current release can be downloaded here.

The software is made available by the Computer Science Department at Carnegie Mellon University to further speech user interface research and deployment of speech-based solutions. The Speech at CMU Web Page provides more information about open source speech software.

The ShortTalk concept has not yet been implemented in any commonly used software such as word processors or browsers. For technically-oriented users, the Emacs implementation offers the ability to compose documents, program text, and emails.

ShortStep is another approach to hands-free input; see ShortStep for more information on this foot keyboard concept.

The ShortTalk concept and the EmacsListen implementation were developed by Nils Klarlund while at AT&T Labs - Research.

ShortTalk and EmacsListen are hosted at SourceForge