On 11/18/2014 8:50 AM, Paolo Bonzini wrote:
I'm adding two people who might know.
Do you have any idea what the "magic to pipe data back to the Linux
host" should look like? Does a normal serial port (COM1 for Windows,
/dev/ttyS0 for Linux) work?
The fine magic comes in three forms. Keystroke injection, context
feedback, and exporting UI elements such as microphone level,
recognition correction, partial recognition pop-ups into the linux
environment.
All of these have in common the magic trick of using the isolation of
the Windows environment to provide a single dictation target to
NaturallySpeaking. All of the information necessary for the above
capabilities would pass through this target. initially, this would be an
ssh session with command redirecting standard into whatever
accessibility inputs available.
The host side of this gateway would be responsible for all of the
proper input redirection. In theory, it would even be possible to direct
speech recognition towards two targets depending on the grammar. For
example in the programming by speech environment I'm working on, I would
dictate directly into the editor sometimes and into a secondary window
for focused speech UI action. At no time, would my hand touch the mouse.
:-) It will happen because of the context set by the speech UI as a
deliberate effect of certain commands.
--- longer ramble about speech and nuance issues. ---
Being a crip who's trying to write code with speech, it's not going to
be fast. once I get the Basic keystroke injection working, it will be
good enough to continuing developing my program by speech environment.
But to discuss that, would go down the rathole of current models of
speech user interfaces, why don't work, things you shouldn't do such as
speaking the keyboard, intentional automation, contextual grammars and a
host of other things of spent the past 15 years learning about and
figuring out how to make a change. By the way, that knowledge and
passion is why I I've started a consulting practice that focuses on
improving user experience/user interfaces starting from the intent of
the user and perspective of a disabled person with the result being an
improved UI for everybody.
The hardest part is going to be everything except a keystroke injection.
This is because they require special knowledge that nuance is loath to
give up. I don't get it. Nuance totes and gets federal benefits for
producing something that is "section 508 compliant" yet, the only way
you could be considered an accessibility tool is if you do nothing but
write in Microsoft Word. I worked for a dragon reseller for a while
with medical record systems and, nuance doesn't even make an attempt to
try and speech enable the medical record environment. They have people
using a couple of solutions that don't work well and effectively provide
no UI automation[1] tied into speech commands.
A bunch of us techno Crips have built environments that greatly enhance
the range of solutions NaturallySpeaking could be used for but, nuance
won't talk to us, won't give us any documentation to keep things running
on our own, won't sell us the documentation either and worst of all,
they have written terms into the AUP designed to bar extensions like our
environment unless you buy the most expensive version of
NaturallySpeaking available.
And did I mention that they have many bugs that are a significant
problem for every user, not to mention the scripts and the last time I
checked, it will cost about $10 to report a bug (support call cost) and
then there's no guarantee they'll ever fix. In version 13, I'm seeing
bugs that have been around since version 7 or 8.
I will do what I can to implement the magic and when I get stumped,
then, I'll figure out what I'm going to do technically and politically.
--- eric
[1] This is kind of a lie. They have the tools to what you navigate
blindly through an application (i.e. hit 15 tabs, two down arrows, and a
mouse click and it might end up in the right UI element to do something.
unfortunately, they do not have anything to make it predictable,
repeatable or survive revisions in the user interface. But this is one
of those rat holes I said it wouldn't go down.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
