Go Back   Cockos Incorporated Forums > Projects > Deprecated REAPER issue tracker > Feature Request

Implement accessibility through MS UI Automation support Issue Tools
issueid=3888 01-20-2012 10:45 AM
Human being with feelings
Implement accessibility through MS UI Automation support
Reaper users with visual handycaps rely on screen reader software. This software needs to obtain the context of all GUI elements. On Windows platforms this can be easily done by imploying the MS UI Automation classes and setting corresponding propert

Discussion thread: http://forum.cockos.com/showthread.php?t=137090

Reaper already supports to bind most - if not all - actions to key strokes. The difficulty for screen readers is to obtain information about the GUI elements and their relation and meaning.
UIA is extensively documented on MSDN and this Wikipedia article may serve as a short introduction
http://en.wikipedia.org/wiki/Microsoft_UI_Automation
Since Reaper already supports tailering of the GUI chances are that UIA support is not a large effort and it would certainly help many inthusiastic Reaper users.
BTW: UIA can also be applied for automatic testing.
Issue Details
Issue Type Feature Request
Project Deprecated REAPER issue tracker
Category GUI and graphics
Status Suggested
Priority 3
Suggested Version 4.15
Implemented Version (none)
Users who would use this feature 162
Users who would not use this feature 1
Assigned Users (none)
Tags (none)

04-15-2013 07:59 PM
Administrator
 
We have spent a great deal of effort on supporting accessibility APIs on both Windows and OSX, unfortunately it is a very difficult challenge due to extremely poor documentation, on both sides. If anybody has any good resources that may be of help on the code side for supporting this, we would be happy to take a look and spend more time improving support for these APIs.
Reply
12-26-2014 07:03 AM
Human being with feelings
 
I'm one of the two lead developers of the NVDA screen reader for Windows. I appreciate the effort to support accessibility so far. While I certainly agree some of the documentation could be easier to find or follow, I think the available documentation does cover a lot of major points that aren't implemented here. Regardless, here's a quick summary of some major problems (some well covered by documentation, some not) that would make things a lot better if they were fixed.
  • On Reaper's root accessibility objects (with ROLE_SYSTEM_GROUPING), IAccessible::accParent currently returns NULL. It should return the oleacc window object for this HWND; i.e. AccessibleObjectFromWindow with the HWND and OBJID_WINDOW.
  • IAccessible::accNavigate is not implemented. Although MSDN says it is deprecated, it is still used by NVDA and probably other clients. (There are good reasons I believe this shouldn't be deprecated at all.) It'd be great if support for NAVDIR_FIRSTCHILD, NAVDIR_NEXT and NAVDIR_PREVIOUS could be implemented.
  • IAccessible::accDefaultAction and IAccessible::accDoDefaultAction are not implemented. For controls which can be activated, these should activate the control. accDefaultAction could return a name like "activate", "click" or "press".
  • The faders don't expose a value. It'd probably make sense for these to expose the textual value via IAccessible::accValue.
  • The grouping accessibles should expose a useful name where possible; e.g. the track number and name for tracks. Also, some of the groupings seem to be toolbars. They should ideally expose ROLE_SYSTEM_TOOLBAR.
  • At present, without specific code (and assuming Reaper even fires accessibility events), a screen reader or other assistive technology (AT) product doesn't know what changes to report to the user. One simple option is to pretend (as far as accessibiliy is concerned) that any control that is manually adjusted/activated by the user (e.g. via keyboard or mouse) receives the focus. This includes the track accessible (ROLE_SYSTEM_GROUPING) when a track becomes the current track. This is controversial because it doesn't really have focus, but it does tell the user exactly what they need to know. To do this, you'll need to fire EVENT_OBJECT_FOCUS using NotifyWinEvent and expose STATE_SYSTEM_FOCUSED via IAccessible::accState on the control with "fake" focus. You'll also need to implement IAccessible::accFocus. If the control already has the fake focus (e.g. multiple contiguous changes to the same control), it shouldn't get focus again, but an event needs to be fired indicating that the control changed; e.g. EVENT_OBJECT_VALUECHANGE if the value changed or EVENT_OBJECT_STATECHANGE if the state changed.
  • Regarding toggles such as mute and solo, it might be better to always expose the names as "Mute", "Solo", etc., give them ROLE_SYSTEM_CHECKBOX and expose STATE_SYSTEM_CHECKED when they're enabled. Currently, the name changes depending on the state; e.g. "Unmute track" when the trakc is muted. This could get confusing particularly if the above focus idea is implemented.
  • Unless I'm missing something, I don't think there is any accessibility information exposed for items.
  • IAccessible::accLocation and IAccessible::accHitTest need to deal in physical screen coordinates. Because Reaper doesn't seem to be DPI aware, it's currently using the coordinates it gets internally, but these are logical, not physical. These need to be converted with LogicalToPhysicalPointForPerMonitorDPI/LogicalToPhysicalPoint and vice versa. (Annoyingly, you'll have to use the *ForPerMonitorDPI functions on Windows >= 8 and the older versions without this suffix on Windows < 8.)

This is far from complete, but it should be a good start.

In addition, it's important that the various context menus are accessible from the keyboard. If I make a track current with the keyboard, the Applications key doesn't seem to do anything unless I click in the track first. Also, I believe there are multiple context menus. The solution implemented by ReaAccess was to use modifiers and the Applications key for other context menus; e.g. control+Applications.
Reply
12-26-2014 05:42 PM
Human being with feelings
 
Quote:
Originally Posted by Scottsdale
Just chiming in to point out that I don't think the modifiers would be necessary in addition to the Applications key once elements of each track in the TCP can obtain focus from keyboard navigation.
Note that I was originally suggesting that Reaper just fakes the focus for accessibility. However, it'd certainly be better if focus were more "real" in some cases (such as items) so that keys like the Applications key are truly context sensitive as @Scottsdale suggests.
Reply
Reply

Issue Tools
Subscribe to this issue

All times are GMT -7. The time now is 04:24 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.