/documentation/talks/google-io-2011-access-101/android-access.html

http://eyes-free.googlecode.com/ · HTML · 591 lines · 577 code · 0 blank · 14 comment · 0 complexity · dfe6a496c4f3a9a70f9390cece907226 MD5 · raw file

  1. <!DOCTYPE html>
  2. <!--
  3. Google I/O 2011 HTML slides template
  4. Created by Luke Mahé (lukem@google.com)
  5. and Marcin Wichary (mwichary@google.com).
  6. URL: http://go/io-html-slides
  7. -->
  8. <html>
  9. <head>
  10. <title> Leveraging Android Accessibility APIs To Create An Accessible Experience</title>
  11. <meta charset='utf-8' />
  12. <script src='http://io-2011-slides.googlecode.com/svn/trunk/slides.js'></script>
  13. </head>
  14. <style>
  15. /* Your individual styles here, or just use inline styles if that’s
  16. what you want. */
  17. .slides > article::before {
  18. content: '';
  19. display: block;
  20. width: 500px;
  21. height: 100px;
  22. position: absolute;
  23. left: 0;
  24. bottom: 0;
  25. background: url(google_hubbell.png) 30px 20px no-repeat;
  26. }
  27. </style>
  28. <body style='display: none'>
  29. <section class="slides">
  30. <! -- Introduction: 4 slides -->
  31. <article class='biglogo'></article>
  32. <article>
  33. <h1> Leveraging Android Accessibility APIs To Create An Accessible Experience</h1>
  34. <p>May 10, 2011 </p>
  35. <p>Google I/O 2011</p>
  36. </article>
  37. <article>
  38. <h3 id="sec-1">Speakers </h3>
  39. <table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
  40. <colgroup><col class="left" /><col class="left" />
  41. </colgroup>
  42. <tbody>
  43. <tr><td class="left">Project</td><td class="left"><a href="http://eyes-free.googlecode.com">http://eyes-free.googlecode.com</a></td></tr>
  44. <tr><td class="left">T. V. Raman</td><td class="left">raman@google.com</td></tr>
  45. <tr><td class="left">Charles L Chen</td><td class="left">clchen@google.com</td></tr>
  46. <tr><td class="left">Tim Credo</td><td class="left">credo@google.com</td></tr>
  47. <tr><td class="left">Blog</td><td class="left"><a href="http://eyes-free.blogspot.com">http://eyes-free.blogspot.com</a></td></tr>
  48. </tbody>
  49. </table>
  50. <br>
  51. <p>Please provide us feedback:</p>
  52. <table border="2" cellspacing="0" cellpadding="6" rules="groups"
  53. frame="hsides">
  54. <tr>
  55. <td>Feedback</td>
  56. <td><a href="http://goo.gl/AjOLf">http://goo.gl/AjOLf</a></td>
  57. </tr>
  58. <tr>
  59. <td>Twitter</td>
  60. <td>#io2011 #android #eyesfree </td>
  61. </tr>
  62. </table>
  63. </article>
  64. <article>
  65. <h3 id="sec-2">Overview </h3>
  66. <ul>
  67. <li>
  68. Motivation
  69. </li>
  70. <li>
  71. Accessibility support in the Android framework
  72. </li>
  73. <li>
  74. Accessibility Services
  75. </li>
  76. <li>
  77. Making <i>inaccessible</i> applications <i>accessible</i>
  78. </li>
  79. <li>
  80. Well-written apps that are accessible with zero effort
  81. </li>
  82. <li>
  83. Testing for accessibility using freely available tools
  84. </li>
  85. <li>
  86. Fluent eyes-free interaction
  87. </li>
  88. <li>
  89. Developer resources
  90. </li>
  91. </ul>
  92. </article>
  93. <!-- Motivation: 3 slides -->
  94. <article>
  95. <h2>Motivation</h2>
  96. </article>
  97. <article>
  98. <h3>Motivation</h3>
  99. <ul>
  100. <li>Applications <em>work out of the box</em> for blind/low-vision users</li>
  101. <li>Make Android applications usable in more user contexts</li>
  102. <li>Encourage redundancy in the UI for graceful degradation</li>
  103. </ul>
  104. </article>
  105. <article>
  106. <h3>Changing How We Work and Play </h3>
  107. <ul>
  108. <li>Mobile devices can sense user's environment</li>
  109. <li> Can provide feedback via a multiplicity of modalities</li>
  110. <li> Has a profound impact on independent living</li>
  111. </ul>
  112. </article>
  113. <!-- Android Framework Access 5 slides-->
  114. <article>
  115. <h2>Accessibility in the Android Framework</h2>
  116. </article>
  117. <article>
  118. <h3>Android Platform Accessibility</h3>
  119. <ul>
  120. <li>
  121. First introduced in Android 1.6 &mdash; September 2009
  122. </li>
  123. <li><a href="http://developer.android.com/reference/android/speech/tts/TextToSpeech.html">Text-To-Speech API</a>
  124. <ul>
  125. <li>
  126. Enables applications to produce spoken feedback
  127. </li>
  128. <li>
  129. Starting in 2.2, users can choose their voice
  130. </li>
  131. </ul>
  132. </li>
  133. <li>
  134. Platform API enables <a href="http://developer.android.com/reference/android/accessibilityservice/AccessibilityService.html">Accessibility Services</a> (TalkBack)
  135. </li>
  136. </ul>
  137. </article>
  138. <article>
  139. <h3>Anatomy of an Accessibility Service</h3>
  140. <ul>
  141. <li>
  142. Register with platform to receive <code>AccessibilityEvents</code>
  143. </li>
  144. <li>
  145. Process event payload and produce feedback
  146. </li>
  147. <li>
  148. Events are only dispatched when accessibility is enabled
  149. </li>
  150. </ul>
  151. </article>
  152. <article class="smaller">
  153. <h3><a href="https://market.android.com/details?id=com.google.android.marvin.talkback">TalkBack: Accessibility Service</a></h3>
  154. <ul>
  155. <li>
  156. Processes <code>AccessibilityEvents</code> depending on their types (<span style="text-decoration:underline;">selected</span>, <span style="text-decoration:underline;">clicked</span>,
  157. etc.)
  158. <ul>
  159. <li>
  160. No special commands; speech happens in response to user interaction
  161. </li>
  162. <li>
  163. Enabling navigation with the trackball/d-pad is critical for accessibility
  164. </li>
  165. </ul>
  166. </li>
  167. <li>
  168. Able to filter on package name for special cases
  169. <ul>
  170. <li>
  171. Avoid doing voice recognition on the TTS inside Voice Search
  172. </li>
  173. </ul>
  174. </li>
  175. <li>
  176. Recently added a talking keyboard with virtual d-pad feature
  177. </li>
  178. <li>
  179. APIs being improved to enable spoken access to all on-screen content
  180. </li>
  181. </ul>
  182. </article>
  183. <article>
  184. <h3>Features of an Accessible Android <code>View</code></h3>
  185. <ul>
  186. <li>
  187. Responds consistently to D-Pad/Trackball interaction
  188. </li>
  189. <li>
  190. Sets focus correctly as selection changes
  191. </li>
  192. <li>
  193. Raises <code>AccessibilityEvents</code> during user interaction
  194. </li>
  195. </ul>
  196. </article>
  197. <!-- Creating Accessible UIs -->
  198. <article>
  199. <h2>Creating Accessible UIs</h2>
  200. </article>
  201. <article>
  202. <h3>Standard <code>Views</code></h3>
  203. <ul>
  204. <li>
  205. Most of the work has been done already, and styling it should not have any
  206. adverse impact.
  207. </li>
  208. <li>
  209. Make sure images, especially <code>ImageButtons</code>, are labeled with
  210. <code>contentDescription</code>
  211. </li>
  212. </ul>
  213. </article>
  214. <article>
  215. <h3>Modified <code>Views</code> Extended from Standard <code>Views</code></h3>
  216. <ul>
  217. <li>
  218. Should mostly work
  219. </li>
  220. <li>
  221. Check to see that your changes are not interfering with trackball/d-pad
  222. navigation
  223. </li>
  224. <li>
  225. If you also want to customize how your <code>View</code> speaks, override
  226. <code>dispatchPopulateAccessibilityEvent</code>
  227. </li>
  228. </ul>
  229. </article>
  230. <article>
  231. <h3>Completely Custom <code>Views</code><br>(Extended from the Base <code>View</code> Class)</h3>
  232. <ul>
  233. <li>
  234. Make sure you support trackball/d-pad navigation
  235. </li>
  236. <li>
  237. Implement <code>dispatchPopulateAccessibilityEvent</code>
  238. </li>
  239. </ul>
  240. </article>
  241. <!-- Examples 5 Slides-->
  242. <article>
  243. <h2>Example of Making Custom <code>Views</code> (More) Accessible</h2>
  244. </article>
  245. <article>
  246. <h3>Background</h3>
  247. <ul>
  248. <li>
  249. <a href="http://code.google.com/p/google-authenticator/">Google Authenticator</a> is an open source app that generates one-time passcodes for two-step verification.
  250. </li>
  251. <li>
  252. <code>UserRowView</code> is a custom <code>View</code> that displays the username, the generated code, and a "Get code" button
  253. </li>
  254. </ul>
  255. <p><img class="centered" style='height: 300px' src='google_auth.png' alt='Screenshot of Google Authenticator.'>
  256. </p>
  257. </article>
  258. <article>
  259. <h3>Default Behavior: Usable but Clunky</h3>
  260. <ul>
  261. <li>
  262. Composite <code>View</code>; Android's platform accessibility tries to do what it can.
  263. </li>
  264. <li>
  265. User hears: "username at domain. One hundred twenty three thousand four hundred fifty six. Get code."
  266. </li>
  267. <li>
  268. Most users only have one account; hearing the username at domain at the start is a waste of time.
  269. </li>
  270. <li>
  271. Parsing out the digits to type is difficult.
  272. </li>
  273. <li>
  274. Can we do better?
  275. </li>
  276. </ul>
  277. </article>
  278. <article>
  279. <h3>Yes! And Here's How:</h3>
  280. <ul>
  281. <li>
  282. Override <code>dispatchPopulateAccessibilityEvent</code>
  283. </li>
  284. <li>
  285. Optimize the text to be spoken
  286. </li>
  287. <ul>
  288. <li>
  289. Add spaces to the digits of the pin code so that it will not be spoken as a very large number
  290. </li>
  291. <li>
  292. Move the username information to the end
  293. </li>
  294. </ul>
  295. <li>
  296. Code for doing this fits on a single slide (&lt;25 lines)
  297. </li>
  298. <li>
  299. See the <a href="http://google-authenticator.googlecode.com/hg/mobile/android/src/com/google/android/apps/authenticator/UserRowView.java"><code>UserRowView class </code></a>
  300. </li>
  301. </ul>
  302. </article>
  303. <article class="smaller">
  304. <h3>Improving Accessibility for <code>UserRowView</code></h3>
  305. <pre class="build"><span>
  306. public boolean dispatchPopulateAccessibilityEvent(AccessibilityEvent accessEvent) {
  307. Context ctx = this.getContext(); </span><span>
  308. String message = "";
  309. CharSequence pinText =
  310. ((TextView) findViewById(R.id.pin_value)).getText();
  311. if (ctx.getString(R.string.empty_pin).equals(pinText)){
  312. message = ctx.getString(R.string.counter_pin);
  313. } else {
  314. for (int i = 0; i &lt; pinText.length(); i++) {
  315. message = message + pinText.charAt(i) + " ";
  316. }
  317. } </span><span>
  318. CharSequence userText =
  319. ((TextView) findViewById(R.id.current_user)).getText();
  320. message = message + " " + userText; </span><span>
  321. accessEvent.setClassName(getClass().getName());
  322. accessEvent.setPackageName(ctx.getPackageName());
  323. accessEvent.getText().add(message);
  324. return true;
  325. }</span>
  326. </pre>
  327. </article>
  328. <!-- Accessible Applications 3 Slides -->
  329. <article>
  330. <h2>Examples of Well Written Apps that are Accessible with Zero Effort</h2>
  331. </article>
  332. <article>
  333. <h3>Accessible Applications on Android</h3>
  334. <ul>
  335. <li>
  336. Many Android applications <i>work out of the box</i> with our
  337. Accessibility APIs
  338. </li>
  339. <li>
  340. The Bad News: Many applications do not work as expected
  341. </li>
  342. <li>
  343. The Good News: This is really easy to fix
  344. </li>
  345. </ul>
  346. </article>
  347. <article>
  348. <h3><a href="https://market.android.com/details?id=tunein.player">TuneIn Radio: An Accessible Android Application</a></h3>
  349. <ul>
  350. <li>
  351. Raman's personal favorite radio tuner application
  352. </li>
  353. <li>
  354. We did not (initially) talk to the developer about accessibility
  355. </li>
  356. <li>
  357. The application was usable <i>out of the box</i>
  358. </li>
  359. <li>
  360. Application is built on top of platform <code>Views</code> that are already accessible by default
  361. </li>
  362. <li>
  363. Interface is D-Pad/trackpad navigable
  364. </li>
  365. <li>
  366. Most controls were correctly labeled
  367. </li>
  368. </ul>
  369. </article>
  370. <!-- Testing 3 slides -->
  371. <article>
  372. <h2>Testing for Accessibility</h2>
  373. </article>
  374. <article>
  375. <h3>Testing by Doing</h3>
  376. <ul>
  377. <li>
  378. Accessibility is built-in; makes testing easy!
  379. </li>
  380. <li>
  381. Turn on Settings Accessibility
  382. </li>
  383. <li>
  384. Enable Accessibility
  385. </li>
  386. <li>
  387. Enable TalkBack
  388. </li>
  389. <li>
  390. Use your application via the D-Pad
  391. </li>
  392. </ul>
  393. </article>
  394. <article>
  395. <h3>Checks to Perform</h3>
  396. <ul>
  397. <li>
  398. All tasks can be completed via the D-Pad or trackball
  399. </li>
  400. <li>
  401. All critical on-screen text is focusable
  402. </li>
  403. <li>
  404. All user actions produce meaningful spoken output
  405. </li>
  406. <li>
  407. There is meaningful spoken output as you navigate
  408. </li>
  409. </ul>
  410. <p>
  411. Extra Credit: All tasks can be completed <b>efficiently</b>
  412. </p>
  413. </article>
  414. <!-- Eyes-Free Interaction 6 slides -->
  415. <article>
  416. <h2>Going the Extra Mile:<br>Fluent Eyes-Free Interaction</h2>
  417. </article>
  418. <article>
  419. <h3>Eyes-Free Interaction</h3>
  420. <ul>
  421. <li>
  422. TalkBack speech-enables standard visual user interfaces
  423. </li>
  424. <li>
  425. Brings a consistent level of accessibility on the platform
  426. </li>
  427. <li>
  428. Now, let's go the extra mile toward eyes-free interaction!
  429. </li>
  430. </ul>
  431. </article>
  432. <article>
  433. <h3>Eyes-Free Tasks</h3>
  434. <ul>
  435. <li>
  436. Obtaining feedback in eyes-busy environments
  437. </li>
  438. <li>
  439. Minimizing interactions needed to perform a task
  440. </li>
  441. <li>
  442. Leveraging voice interaction
  443. </li>
  444. </ul>
  445. </article>
  446. <article>
  447. <h3>Obtaining Feedback in Eyes-Busy Environments</h3>
  448. <ul>
  449. <li>
  450. Leverage built-in TTS APIs to generate speech
  451. </li>
  452. <li>
  453. Leverage haptic feedback for additional eyes-free polish
  454. </li>
  455. <li>
  456. Augment output with subtle auditory cues
  457. </li>
  458. </ul>
  459. </article>
  460. <article>
  461. <h3>Minimizing Interactions Needed to Perform a Task</h3>
  462. <ul>
  463. <li>
  464. Android devices have many <i>eyes and ears</i> that can sense the environment
  465. </li>
  466. <li>
  467. Android devices can catch user's attention in many ways
  468. </li>
  469. <li>
  470. User interaction is about communicating user intent to the
  471. device, and obtaining feedback
  472. </li>
  473. <li>
  474. Leverage these features to minimize explicit user
  475. interaction
  476. </li>
  477. </ul>
  478. </article>
  479. <article>
  480. <h3>Leveraging Voice Interaction </h3>
  481. <ul>
  482. <li>
  483. <a href="http://developer.android.com/reference/android/speech/RecognizerIntent.html">Speech recognition</a> is just one API call away
  484. </li>
  485. <li>
  486. Use network-based speech recognition to easily
  487. voice-enable your application
  488. </li>
  489. </ul>
  490. </article>
  491. <!-- examples: eyes-free 4 slides-->
  492. <article>
  493. <h2>Examples of Eyes-Free Applications</h2>
  494. </article>
  495. <article>
  496. <h3><a href="https://market.android.com/details?id=com.marvin.rocklock">RockLock</a></h3>
  497. <ul>
  498. <li>
  499. Eyes-Free access to your music collection on Android
  500. </li>
  501. <li>
  502. Tap to play-pause music
  503. </li>
  504. <li>
  505. Stroking in different directions lets you jump tracks, switch between artists/albums, and rewind/ff
  506. </li>
  507. </ul>
  508. </article>
  509. <article>
  510. <h3><a href="https://market.android.com/details?id=com.googlecode.eyesfree.walkytalky">WalkyTalky</a></h3>
  511. <ul>
  512. <li>
  513. Takes advantage of the Android Intents mechanism to have Google Maps Navigation provide the user with walking directions
  514. </li>
  515. <li>
  516. Uses the GPS and the Google Maps API to periodically reverse geocode the user's current location and put it in the notification bar if it has changed
  517. </li>
  518. <li>
  519. Since TalkBack speaks notifications, the user hears the addresses as they walk by
  520. </li>
  521. <li>
  522. Useful for everyone
  523. </li>
  524. </ul>
  525. </article>
  526. <article>
  527. <h3><a href="https://market.android.com/details?id=com.google.android.marvin.intersectionexplorer">Intersection Explorer</a></h3>
  528. <ul>
  529. <li>
  530. Auditory StreetView-like experience that lets you virtually explore a place
  531. </li>
  532. <li>
  533. Map starts centered at your current location
  534. </li>
  535. <li>
  536. Stroke in a direction to hear where you would end up if you went that way
  537. </li>
  538. <li>
  539. Lift up finger to move in that direction
  540. </li>
  541. <li>
  542. Can virtually explore any location covered by Google Maps
  543. </li>
  544. </ul>
  545. </article>
  546. <article>
  547. <h2>Conclusion</h2>
  548. </article>
  549. <article>
  550. <h3>Conclusion</h3>
  551. <ul>
  552. <li>
  553. Android has built-in accessibility support
  554. </li>
  555. <li>
  556. Writing accessible Android applications is <b>easy</b>
  557. </li>
  558. <li>
  559. Accessibility Significantly increase your user base
  560. </li>
  561. </ul>
  562. </article>
  563. <article>
  564. <h3>Developer Resources</h3>
  565. <dl>
  566. <dt>Eyes-Free Project</dt>
  567. <a href="http://eyes-free.googlecode.com">http://eyes-free.googlecode.com</a>
  568. <br><br>
  569. <dt>Android Guide</dt>
  570. <a href="http://d.android.com/guide/practices/design/accessibility.html">http://d.android.com/guide/practices/design/accessibility.html</a>
  571. </dl>
  572. </article>
  573. <article>
  574. <h3>Thank you!</h3>
  575. <p><img class="centered" style='height: 350px' src='hubbell_flying.jpg' alt='Hubbell flying a plane.'>
  576. </p>
  577. <table border="2" cellspacing="0" cellpadding="6" rules="groups"
  578. frame="hsides">
  579. <tr>
  580. <td>Feedback</td>
  581. <td><a href="http://goo.gl/AjOLf">http://goo.gl/AjOLf</a></td>
  582. </tr>
  583. <tr>
  584. <td>Twitter</td>
  585. <td>#io2011 #android #eyesfree </td>
  586. </tr>
  587. </table>
  588. </article>
  589. </section>
  590. </body>
  591. </html>