----- Original Message ----- From: "Peter Korn" <peter.korn@SUN.COM> To: <JAVA-ACCESS@JAVA.SUN.COM> Sent: Thursday, May 16, 2002 12:16 AM Subject: Re: [ui-dev] Q: How to implement XAccessibleText interface in formula editor ? Hi Thomas, I've cross-posted to the GNOME Accessibility list, the Mozilla accessibility list, and the Java Accessibility list, as I think folks in those communities may have some good suggestions as well. If nothing else, there are communities that should be thinking about these issues. I think our ATV friends in the community are especially important to get feedback from on these questions (BAUM, University of Toronto, Freedom Scientific, & Ai Squared). I've also directly cc-ed John Gardener, a professor of Physics at Oregon State University in the U.S. John has spent many years looking at ways of representing mathematical equations to the blind. For the GNOME and Mozilla communities: the question Thomas raises is how should special characters (that are multi-character strings as they are stored within a document), and how should characters in formulas, be represented in the AccessibleText interface. Thomas' question is specific to the OpenOffice UNO Accessibility API (and the XAccessibleText interface), but the problem faces us in GNOME, Netscape, and Java land as well. I'm appending my person thoughts at the end of Thomas original e-mail below. Thomas Lange wrote: > > Hi, > > How should I implement the functions > virtual awt::Rectangle SAL_CALL getCharacterBounds( sal_Int32 nIndex ) > virtual sal_Int32 SAL_CALL getIndexAtPoint( const awt::Point& aPoint ) > from the XAccessibleText interface in the formula editor ? > > Especially considering that some symbols in the graphical display are > represented as a single character (for example the greek Sigma > character) where the character will be represented by a 'word' (here > '%SIGMA') in the text. > > If for example the formula text is "%ALPHA + %BETA equals %KAPPA" which > would result in a graphical representation like "A+B=K" which has only 5 > characters. > > Should I return the bounding rectangle of "A" for the indices 0-5 > (%ALPHA) when calling getCharacterBounds ? > And when calling getIndexAtPoint for a point of "A" should I return 0 as > index ? > > Or should I expand the symbolic names to the unicode character they are > representing and thus having the > rtl::OUString SAL_CALL getText() > function return the string "A+B=K" also where the symbolic names will > have been replaced with the corresponding unicode character. > This approach would make the implentations of getCharacterBounds and > getIndexAtPoint quite obvious. > However many of the unicode characters used in the formula editor come > from within the private area of the unicode character set. Will this be > a problem ? > > The next problem will be how should I implement these functions for > formatted text ? > For example when a text or character has super- and subscripts in its > graphical representation ? > > n > AAA x > m > > Which should be the order of assigning the indices to the text ? > Should index 0 be the 'n', 1 'AAA' and 2 'm' ? > > How about some unusual attributed text like symbols with a (single) > vector arrow above ? > --> > V > > Regards, > Thomas I think for symbols like "%ALPHA", they should be seen as a single (Unicode or UTF8) character wherever possible. Failing that, I think the characters of that "word" should share the same bounding rectangle. But a single character would, I think, be better. For more complex formulas, like a summation (or integral) from m to n, I think you'll have to pick a cannonical order (and it should be one we all agree to so it's standard throughout all accessible representations of math formulas). We could also introduce a bunch of new AccessibleRelations to try to cover formulas, making each of these characters (or collections of characters) their own object with relations to other objects. Unfortunately, none of these approaches is really all that satisfying. A longer-term approach we should consider is finding a way to represent the entire formula as it's own object, and create an AccessibleDescription for the formula that explains what it is (so something like "Integral from m to n of the function f(x)"). Also, we should look toward standards like MathML to represent the formulas, and expose the MathML directly for a screen access product to parse as it chooses. If there is a MIME type for MathML, we could use the AccessibleStreamable interface (in GNOME, not yet in the Java Accessibility API) to indicate there is a MathML stream which can be parsed for this object. Regards, Peter Korn Sun Accessibility team ======================================================================== === To unsubscribe, send email to listserv@java.sun.com and include in the body of the message "signoff JAVA-ACCESS". For general help, send email to listserv@java.sun.com and include in the body of the message "help".