jLuger.de - Lessons learned from making a swing application accessible

What is this article about
Lets start with accessible. Here it means that people with a disability can use your program. The disability may be that they are blind, have a visual impairment (varying degree), are color blind or at least red-green color blind, have problems focusing a target via mouse, or can't hear.

Swing is a GUI framework for the Java programing language. The special thing about it is, that it paints all the widgets itself instead of using the ones of the operating system. This means that the operating system will not know anything about the buttons and textfields of a Java application that uses swing. Well, when the operating system knows nothing about them then the accessibility tools can neither. To solve this issue there is an accessibility bridge for Java. I was told that this bridge doesn't offer all the features that you get when you have a native application.

This article will highlight some of the problems I've solved for making an existing swing application accessible. It will not be a guide on how to make applications accessible in general nor will it cover topics that weren't a problem for me (e.g. The application uses no sounds. So people that can't hear weren't an issue.)

The tool settings
Depending of the disability people may use assistive technology. For people with a visual impairment there is screen magnifier software. The concrete type that the application had to support was LunarPlus. It works in that way that the whole screen will just show a part of the desktop. The size of the desktop that is shown decreases when the magnification increases. Typical magnification rates are 4 to 5 but some people will go up to 12. With this rates you will need to scroll a lot even on small 19" monitors with 1024*768 resolution. As it is pretty easy to lose orientation with this setting LunarPlus has the ability to follow the focus. When you press tab to get to the next component it will set the screen to there.
For people with slight visual impairment LunarPlus offers the possibility to change colors in order to increase contrast. This goes from some descend color inverting up to some hard black and white mode.

Blind people have the option to use a screen reading software or a braille display (or both of course). A braille display is a very expensive hardware and pretty useless if you have never learned braille alphabet. For this reasons developers that could see, like me, didn't get a braille display. Fortunately JAWS, the screen reading software to be used, had a braille display emulator.

Starting the application so that JAWS could read its content wasn't easy. Nor starting it from eclipse neither from command line had helped. It turned out that you have to use Webstart (javaws + launch.jnlp). I don't know if it was a problem on my computer or if it is a bug in the accessibility bridge of Java.

Lessons learned
Grouping JRadioButtons
JAWS will tell you which position in the group a radio button has and how many the radio buttons the group has (e.g. one of three). This will work fine as long as you add all radio buttons to a ButtonGroup object. Now imagine that you have a framework that forbids the usage of ButtonGroup. That is what happened to me. The solution was to search where the accessibility bridge get its information from. It turned out that it is the method AccessibleRelationSet getAccessibleRelationSet() in the class AccessibleAbstractButton. If you want to override the method you have not only to extend the AccessibleJRadioButton (the descendant of AccessibleAbstractButton for JRadioButton) but also JRadioButton as it is an inner class that is protected and as you need to override the creation of an instance (public AccessibleContext getAccessibleContext()) in JRadioButton .

Increasing contrast for widget selection
All kinds of buttons (Buttons, Checkboxes, RadioButtons) and the JCombobox have a small border when they are selected. Thats a good thing as it shows you if the component is selected when you work with LunarPlus. I had to work with FocusListener and CompoundBorders  to get something like this for textfields. It just turned ugly when I wanted to set a different color to increase contrast. To set a new color you use UIManager:

You may think one color for all borders or at least one color for the buttons but that is wrong. Each type of button has its own color. And thats the easy part. JCombobox has no color you could set but uses the focus color of the theme (at least for Metal based L&F). Read this as extend your theme and override the method that returns the focus color. If you're just asking why you have to set the color with keys when there is a focus color wait a moment. There is even more crazy stuff. While Buttons take the new color directly, Checkboxes and RadioButtons need a kind of restart:

The same code will work for RadioButtons. sampleCheckBox is an instance of a JCheckBox but the changes will be for all existing and future CheckBoxes.

How to get out of this hell? The SUN/Oracle JDK deliver sample applications with several themes. Use one of these to get a theme with high contrast and use or extend it. Don't start to do little extensions to an existing theme. The time you need will allow you to create a new one. Then let the user choose which one he needs.

BTW, here is the code to update the Look and Feel after the user selected another theme:

Contrast in Black White mode
In the previous section I've recommended to create a new theme with high contrast. Even when you do that there is a small pitfall with LunarPlus. LunarPlus offers a black and white mode where you really have only those two colors. As one is needed for background you have only one for the foreground. When your application needs tow foreground colors to distinguish two states, e.g. disabled/enabled, you will have a logical problem. One of the foreground color will either get the background color or merge with the other foreground color into one.

Check your application for all those possible conflict points and decide whether the information is irrelevant or if it needs to be displayed.

Accessing inactive elements

JAWS will read out all the content of a dialog/window when it is opened. When you want to reread the contents of an element you have to put focus on it. Per default this is not possible. This means that disabled elements aren't existing for a blind person.

How did this affect me? Well, the application needed some read only views on the data someone has entered. This was done via disabling all input components. For textfield I could have changed the setEnabled(false) to setEditable(false) but for the Comboboxes and the various button types this was not possible. Creating a special read-only view would have been a very cumbersome way.

While thinking about another way I started wondering whether it is possible to put the focus on disabled components and if JAWS will then be able to read out the content. For the first question a little search on the web got me this link: http://www.coderanch.com/t/342205/GUI/java/Tab-order-swing-components. The post of Michael Dunn (20.01.2006 12:07) showed a possible solution. A quick check with JAWS revealed that it would work. Sounds like the problem is solved. This is true for the KeyEventDispatcher but a closer look on the implementation of FocusTraversalPolicy showed that it knows about all the components used. Not pretty good for a general purpose implementation.

In order get a good general purpose implementation I've started with DefaultFocusTraversalPolicy. The accept method contains a lot of code but the most is about not going to disabled components. After taking out all that stuff I've stayed with a query that asked if the component is visible, displayable, and focusable. With this policy you are not just able to focus on disabled components but also on JLabel (good thing) and panels, scrollpanes, .... (bad thing). I've added some if statements that queried via instanceof for unwanted focus targets. This list got quickly very large. You better use a list to keep track of components that shouldn't get a focus.

Getting the content of JOptionPane accessible
In the last paragraphs I've showed how to access disabled elements and Labels. However the solution requires that you set the FocusTraversalPolicy for each dialog where it should be used. That shouldn't be a problem for your own dialogs but when it comes to JOptionPane you have no chance to set it. The only way to get around this is to build custom message dialogs. For a start I recommend to look at the showOptionDialog method of JOptionPane.

When you do use JOptionPane for your custom message dialogs you will face a problem with the message icon. It has no alternative text and as it is a label you can't easily exclude it from the focus traversal. To make things worse, the icon, the text and even the buttons aren't part of the component but of the UI. Depending on the theme you use you may try to extend it but this wasn't possible in my case. I had to use the brute force method that searched all components of the pane for a label with an icon and no text. There I could set an accessible name.

It turned out that this was just half of the way. JAWS recognizes that there is an icon and wanted to get more information from the AccessibleIconDescription. Somehow there were no information but only a null value that caused a NullPointerException. I had to set manually a description text. Then it worked and I've got some text for the icon.

Defining Shortcuts
Shortcuts are a good thing. They ease the usage of the application not only for people with disabilities but for all your users. Well, if you do them right. Doing it right means that they are ease to get and that they are consistent. Easy to get means that you can see them or even better guess them. Most user will guess that ALT+S is for save. Seeing means that the action key is highlighted. E.g. the u in Turn when ALT+U triggers the key. Consistent means that the keys are the same on all screens. While this should be a no-brainer for common actions like Save and Close but could be difficult for less common actions when your set of action keys is limited to one character. Then you will be quickly forced to give the same key different meanings on different screens. I haven't found a solution for this but when you start choosing a framework to create your GUI you should take care that they support shortcuts with multiple keys.

Another thing with shortcuts is the processing by JAWS. Sometimes it will cause shortcuts to be fired twice. So take care that no security question is confirmed with the same shortcut as with which the action was initially triggered. Should be a great issue. Well, except the great ALT+F4. When you use this to close a modal dialog it may close your application too. Depending on how difficult this situation will be for your users you may think about reintroducing one of those old and nasty "Are you sure to quit?" dialogs.

When you change the accessibility name of a visible component JAWS will read out the new text. Sounds like a great way to get status messages to the user? Rethink it again. There may be users that only use the braille display. Those will not get notified.

What really brought trouble to the project was the fact that the accessible name was even read out when there was another dialog in front of the window with that component. That may be not that bad but the covering dialog had a textfield that showed error messages via a tooltip and setting these error message as text in the hidden status bar. That caused wondering why the tooltip gets read out but won't be accessible by the braille display. It seems that the tooltip wasn't accessible all the way. It was the status bar whose accessible name just got the same message.

Accessing information outside the focus cycle
The focus cycle includes all components that you can reach from a starting position via (Ctrl-)Tab. All those component that didn't receive the focus after you've reached the starting position again are out of the focus cycle. Normally you don't want to have any components outside as these aren't "visible" for the blind users.

What to do when you can't get a component in the focus cycle? You may want to define a shortcut (when you've got some characters left) that starts an action that lets the component request the focus. Yes even labels can request the focus. Just put that code in an Action class and register an instance of the class at the input map of the parent dialog.

Screen resolution
When designing the GUI of your application keep in mind that the time of 800x600 screen resolution isn't gone. People will try to use the lowest possible screen resolution to get a kind of magnification effect. When the graphic card and the monitor allow to go down to 800x600 they will try it. To cope with this, always use layout manager and don't add to many elements on a single screen. Use tab panes or a card layout instead.

HTML and Labels
You may have heard that Swing supports HTML. A common use case is to use HTML to display the text in multiple lines within one label. The hidden disadvantage is that JAWS doesn't extract the text from the HTML automatically. Instead it will read the HTML tags too. The simple solution is to explicitly set accessible name of the label.

When the providing of two texts for one label seems to be cumbersome and the HTML stays simple one can use the build in HTML-Parser of the JDK. Just override the method handleText(char[],int) in the class HTMLEditorKit.ParserCallback. There put the char[] content together with a blank to a string buffer and give the arguments of handleText to the overridden method. Then use the parse method of ParserDelegator to start parsing. The pure text of the label will be then string buffer and could be assigned to the accessible name of the component.

Read-Only document for HTML in JEditorPane
JEditorPane is able to display simple HTML documents and with this component JAWS is able to extract the text. In order to repeat the read out of a piece of the document the user has to place the cursor to the start of the line/table cell. This is a problem as there will be no cursor when the document is not editable. So the user will get only the document once as a complete text. Thats not pretty nice when you only want to get a piece of information or when you need some kind of navigation.

The solution seems to turn on the cursor for read only documents. The code for turning on and off the cursor is hidden pretty good. So it was easier to create a read only document and tell the editor that it is writable.

In order to make a document read only override the methods insertString and remove. As it is pretty difficult to fill a HTMLDocument on creation you may add a flag that will toggle between write mode (here you just call the methods of the super class) and read only mode (do nothing).

Focus for JLabel
While working on another project I've found out that you can call setFocusable(true) on JLabel. The text of the label is then accessible to the screen reader and the braille display. While it makes the text accessible it has the problem that you don't see whether it is focused or not. So I've replaced them most times with a textfield that was formatted to look like a label. The advantage of the textfield is that you can still select the text. The disadvantage for you will be that it isn't a standard widget of Swing but was part of an internal Framework based on JGoodies (there is a good chance that is now part of JGoodies but you have to search yourself).

JAWS seems to have a problem with JComboBox. When the first entry of a combobox is selected it won't read the text, the position (1 in that case), and the total number of entries. When the users selects another value it will read these three information. Even when switching back to the first entry they will be read. When selecting another entry than the first one using the method setSelectedItem(Object) of JComboBox it will read the text, index, and total count of entries. So far so bad. Things get worse when you use data binding and don't use the setSelectedItem(Object) method of JComboBox but instead call setSelectedItem(Object) method of ComboBoxModel. JAWS won't recognize the new selected value. I guess it thinks that the first value is still selected. Anyway it won't read the text and position of the selected entry or the total number of entries. So when you use data binding ensure that setSelectedItem(Object) method of JComboBox is called.