Limitations and Bugs

General Design Issues

• The performance of NaturallySpeaking on ShortTalk is suboptimal. It is actually amazing that the current solution works at all. Very often, however, misrecognition will occur that is plainly impossible had the speech recognizer been engineered with the ShortTalk grammar in mind. I have no doubt that very substantial improvements are possible. An excellent project in an academic setting!
• EmacsListen only ShortTalk-enables Emacs. This is a serious limitation of course. EmacsListen is only a proof of concept. Moreover, it is not obvious how to even formulate a strategy for enabling other applications at the level of perfection that EmacsListen offers and that is so crucial for speech recognition to be usable. Perfect consistency among applications can probably be obtained only if one common text engine (for calculating text intervals, normalization of spaces, lines, and capitalization) underlies all applications. Choosing EmacsListen as such an engine might be viable. It does not entail that users have to learn anything about Emacs, of course.
• Documentation could be much better.
• The code for spacing and capitalization should be completely rewritten. And, probably the new functionality should be based on probabilistically identified rules. An excellent student project!
• The concept of listen event and the modifications to the command loop are horribly complex. Yes, there are two reasons for this: (1) although ubiquitous, the concept of queuing events in queues is fundamentally flawed if events may be defined in terms of events (as is a case with keyboard macros and command sequences in Emacs). [A better way to handle is to queue them in trees, similar to the call trees of procedural abstractions.] (2) It did not seem possible to avoid using them; in particular, the concept of 'tair' (mouse-position) seems to call for command sequences. Additionally, the undo mechanism also relies on seq uences of events. Much of the complexity stems from the requirements of perfect undoability.
• The code for identifying balanced structures (parentheses, terms, quotations) is too 'hardwired'. The solutions that Emacs offers are inadequate because they are tied to the concept of mode. However, in practice, we now find mixture of modes everywhere: LaTeX snippets in e-mail, EcmaScript in XML, XML in comments in C, etc. The current code is not perfect: a general model of flexible, context-dependent parsing of text should be developed.
• Phonetic, morphological, and semantic choices in ShortTalk.

Specific Bugs

• Tab keys are not displayed correctly the listen buffer. They confuse VR-mode. Use "toob" instead.
• pc-select or delete-selection mode not tested.These modes must be loaded before listen-mode (since the listen-mode pre-command-hook must be executed before any changes to the buffer by other hooks). [These modes have not been tested with EmacsListen.] Instead, cua-mode is recommeded.
• Brown color extends everywhere. The mechanism EmacsListen uses for indicating mairk and loost is a little problematic. Sometimes, the indication for mark stretches over many characters. There is probably a specific bug to be fixed. However, with the new display facilities of Emacs 21, this code should probably be totally reengineered.
• The function fill-paragraph mode in XML does not always work as expected. This is not a problem with EmacsListen.
• The number of $-signs is sometimes not correctly counted for LaTeX-awareness. In the paragraph below:

  Words $w$ and $w'$ that interpret all variables in the same way satisfy the
  	  same set o   f formulas: if for all $i$, $1\leq i \leq K$,
  	

  The ShortTalk command save term tair, with the mouse over the beginning of $1\leq..., does not recognize the whole expression $1\leq i \leq K$.
• The command go aift loon twain does not go to a blank line as expected. This functionality used to work.
• snex hotel numeral 2 or snex H2 asks for attribute in minibuffer This is an issue with html-mode.
• Lack of generalization vox menu tair When ShortTalk encounters the language instinct, forms of commands may be spoken that are not yet implemented. This is one such example.
• Spacing for URLs. This doesn't always work: for example, saying ``A T T dot com'' results in ATT. Com.
• Spacing after p. There should be one space, not two.
• Listen peek windows get mixed up.Sometimes other applications within Emacs steal these windows.
• listen-conventional-undo is buggy. Problem is bad when listen-mode has not been activated. This has been corrected NK 9/7.