[R-pkg-devel] Screen reader help request

Sat Mar 18 20:25:45 CET 2023

Duncan et al., I report success.

The promised text was attached to a graphic, and only appeared once. Not sure what the hidden text is about, or why it is needed. (?) so I went and edited the raw html (to make sure the alt text is only located in there) and my edit is plain to hear.

So there are not two audible versions and the one that matters is in the right place. I can tell the second graphic has a different style of alt text and that later graphics are separated by other code chunks that do not generate graphics (8, 9, and 10) N.B. I was skimming to look for graphics, not reading the associated text or code.

>From a user perspective, the graphics created using canvas are indistinguishable from those inserted using img. This is an example where success in access is unnoticeable to the end user and seldom gets acknowledge of effort as a consequence. I'll say thank you now on behalf of the blissfully unaware blind readers.

I'd like to know that the effort required by future authors is equal for all types of images. The failing of fig.alt to deliver on multiple images in a chunk is something to worry about in a different conversation. I'm watching this and other issues with respect to the development of quarto documents, but in order to test those, I  start with he Rmd performance as a reference point. I hadn't tested multiple plots yet, probably because I don't use them very often.

I can't address the Shiny issues. I can tell people though that there are access issues with some Shiny widgets. These were being investigated by another blind R user so I can check how that has progressed. I sense that his work is not yet complete.

Jonathan

-----Original Message-----
From: Duncan Murdoch <murdoch.duncan using gmail.com> 
Sent: Sunday, 19 March 2023 7:39 am
To: Jonathan Godfrey <A.J.Godfrey using massey.ac.nz>; R Package Development <R-package-devel using r-project.org>
Subject: Re: [R-pkg-devel] Screen reader help request

I've made another attempt at this now.  I'm a bit more hopeful about this one, but still not sure.

In the new code, I write the text in a <p> element which is hidden, and use aria-labelledby to say that both the <div> and the <canvas> are labelled by that text.

I am quite hopeful that the text will be detected by a screen reader now.  You may hear it twice; if so, I have a couple of ideas how to fix that.

I've also thought about what you said about the default text, and have changed it slightly.  Now the first plot in the page

   https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdmurdoch.github.io%2Frgl%2Fdev%2Farticles%2Frgl.html&data=05%7C01%7CA.J.Godfrey%40massey.ac.nz%7C8f0eb72383f74148cef008db27e01261%7C388728e1bbd0437898dcf8682e644300%7C1%7C0%7C638147615570986760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3B9Oapk8PCYTb62K1UBbvHg7DoxdMX3e9ZwRyXFg6CA%3D&reserved=0

will have alt text "Rotatable plot with label unnamed-chunk-1". 
Obviously customized text would be better, but I think this should be better than nothing.

For authors using this, the R Markdown optional fig.alt setting almost works.  It's fine if you only have one plot, but if you have multiple plots with different fig.alt settings rgl will only use the first one. 
I think a change in knitr is needed so that rgl can work out which one you really want.

The Shiny support is also a little deficient.  It would make sense in an interactive Shiny page to be able to change the alt text in response to user actions, but that's not possible yet.  If there are any Shiny experts reading this, I could use some help with that.

Duncan Murdoch

On 17/03/2023 7:59 p.m., Jonathan Godfrey wrote:
> Hello again Duncan,
> 
> I could see the two successes were <img> ... </img> creations and that the others weren't.
> 
> I worry though that in searching for a solution, you need to end up with something that is as simple for authors as what little effort is needed for fig.alt; having looked at the HTML, I'd rather know what is happening at the Rmd file stage and how anything created there gets passed on.
> 
> IMO:  Your reflections on the content of the tags is bang on target. In a perfect world, this is where authors' efforts should be, not on the technical "how do I get that done" aspects.
> 
> I suspect that fixing the toolkit is necessary. A demonstration that a problem can be solved is often a first step in creating access. Furthermore, this is not a problem unique to your single document. Other authors will want to use the same tools which you are demonstrating.
> 
> Alt tags are a perfect demonstration of access that does not impinge on everyone who doesn't need that feature. Use of fig.cap (initially) and latterly fig.alt to provide the access was done at the toolkit level, and it cannot get any easier for authors. That maximises the chances that authors do then turn to the content of the tags.
> 
> I'm happy to assist on projects that lead to positive improvements under the hood that ultimately create access without authors having to become experts in providing access.
> 
> We might take the testing off list, but keep the discussion of access on list.
> 
> Jonathan
> 
> 
> -----Original Message-----
> From: Duncan Murdoch <murdoch.duncan using gmail.com>
> Sent: Saturday, 18 March 2023 12:20 pm
> To: Jonathan Godfrey <A.J.Godfrey using massey.ac.nz>; R Package Development 
> <R-package-devel using r-project.org>
> Subject: Re: [R-pkg-devel] Screen reader help request
> 
> Thanks very much for your response.
> 
> The two images where the tags showed up are regular figures, using lattice and base graphics.  There are also a lot of rgl figures, which have HTML code like this in the source .html file:
> 
>     <div id="rgl61108" style="width:480px;height:296.662546353523px;"
> class="rglWebGL html-widget "></div>
> 
> (Looks like I should think about rounding the height value!)
> 
> All the interesting stuff (including the alt tags) is added within the div by Javascript that runs after the page is loaded.  The fact that it didn't show up in your reader indicates to me that either it needs to be there before the Javascript runs, or that I put the text in the wrong place.  I'll try a few variations on the code and hopefully find something that works.
> 
> Your comments about the content of the tags was also very helpful.  I am imagining that there are at least two groups of readers who might make use of them:
> 
>     - readers who would skip over the graphics completely, but who would be helped by knowing what they missed.
> 
>     - readers who can get information from the graphics but only with an effort, so they would want to know where to apply that effort.
> 
> For the document I posted, there might not be very many people in either group, but I'm hoping to make the package useful to others, who would be writing documents with different audiences.
> 
> Duncan Murdoch
> 
> On 17/03/2023 5:18 p.m., Jonathan Godfrey wrote:
>> Hello Duncan,
>>
>> I guess a few people might expect me to contribute to your request. First I'll assure you that two graphics do have an alt tag which my screen reader is telling me about, so in a sense that is success. I did not check with multiple screen readers or different browsers because I would only find fault if I used inferior software; I expect success with JAWS or NVDA to represent the other and likewise for browsers. These two successes are the lattice and base graphics relating to the volcano data. Others are not showing to my screen reader (JAWS) and browser (Chrome) on my OS (Windows). I note that these are inserted using <img> though.
>>
>> What should appear in an alt tag is an area of much debate though, and the attitude towards their creation differs widely. I believe that the context is the primary driver for what should be communicated. In your context, where you are obviously writing about the package (in general) and comparison of lattice v base graphics (where I do see the alt tags), it is the main text that should be communicating with your audience. You know what you are writing and why. If you do not write about the differences then you leave it up to chance that your (mostly visually-dependent) audience sees the differences that matter to you and your efforts as an author/developer. Complementing that information intended for the whole audience with a sensible alt tag on the graphics for the blind part of your audience, would be fundamentally different to what you might need to offer for the same graphics in another context, say an analysis feeding into a business report for example. In your context, the blind person needs to know that the graphic does appear, and that the graphic has a unique (enough) label to make it obvious which graphic appeared where. We do not need a full description of every last detail of each graphic, although in other contexts, that is exactly what we need.
>>
>> Generating alt tag text automatically (no human interaction) is going to deliver on the technical aspects, without the analytic outcomes, although our recent efforts with the BrailleR package are trying to offer more and more factual information that could help with the analysis of the data. Our main aim is to assure the blind producer of a graphic that it does actually exist and what (roughly) it shows; it would not be great text for an alt tag in most circumstances. I've done some limited testing with AI solutions but haven't yet been overwhelmed by positive experiences.
>>
>> Anyone using an R markdown workflow can add alt text using the fig.alt option in the header of their chunks. Without that, the fig.cap option is used, and if neither is used, the screen reader user is victim to their personal settings. (We choose whether to be told about unlabelled graphics which used to be helpful for ignoring advertising.) I don't put unique alt tags in all of my work because I just need to convey that the graphic is there, but I do make sure there is a default bit of text that saves blind users with the wrong settings from their decisions.
>>
>>
>> An author wishing to support blind consumers of their efforts, could spend lots of time working out what to write, and never get it perfectly correct because the desires of those consumers are not homogeneous. I advise people to use an alt tag, but to make sure it does not duplicate what is already said in the main text of their document. Authors already do this when thinking about the right caption for their figures, but it is easy to see how varied that can be. At least the target audience of captions is well known to the author given the author is part of that population; in contrast, most authors are not blind and may over or under-estimate the needs of the blind readers of their work. Before doing everything an individual requests, please make sure that what is being sought is in keeping with what you are hoping to communicate.
>>
>> I despair at the retrospective creation of alt tags, especially if applied to living documents. Creation of alt tags is simple for anyone using regular markdown based insertion of their graphics because the ![]() construct embraces them in the []. Authors using a LaTeX workflow must load an additional package and add an optional argument to their \includegraphics{} commands. MS Word users can label their graphics anytime but of late there is an increased use of an automated alt tag creation for them (and messages in Outlook too for that matter). I am loathed to fully describe the LaTeX pipeline unless the author is also committed to producing their output in HTML as I have utter contempt for the pdf (or post script) that is the default for most authors using any TeX workflows.
>>
>> Post hoc editing of html is perhaps the worst possible action I observe. This process has separated the creator of the content from the creator of the access for blind people, and is heavily reliant on the skill set of the accessibility "expert". Such editing is common in university settings where the Disability Support Service is responsible for the accessibility. If I was an undergraduate today, I'd be camping outside the doors of my teaching staff to get the alt tags inserted. I spent plenty of time back then, sitting in corridors waiting my turn, but so many of the access challenges of the dead-tree era are thankfully behind us.
>>
>>
>> I'm curious though that a blind person is going to benefit much from your package given so much of what I read relies on using a mouse, or having the ability to imagine the 3d data projected into an inherently 2d page/screen/image. In turn, that inaccessibility might heavily influence my effort to provide comprehensive alt tags.
>>
>> I'd be more than happy to continue this conversation, and to do so on list because I think there are so many opportunities on offer today for people to embrace; many of which will not create piles of extra work.
>>
>> Jonathan
>>
>>
>>
>> -----Original Message-----
>> From: R-package-devel <r-package-devel-bounces using r-project.org> On 
>> Behalf Of Duncan Murdoch
>> Sent: Saturday, 18 March 2023 8:26 am
>> To: r-package-devel using r-project.org
>> Subject: [R-pkg-devel] Screen reader help request
>>
>> I received a request a few days ago to add alt text to the plots in the HTML vignettes for the rgl package.
>>
>> That was easy to do, and I did it, but realized most of the graphics that rgl produces are not in <img> format, they are WebGL canvases, and those don't support the same method of adding alternate text.  Online I've seen a variety of suggestions about how to do it for <canvas> elements, and have implemented one of them.
>>
>> Could I ask someone who is familiar with one or more screen readers to test whether the method that I used works?  A sample web page produced with the new code is here:
>>
>>      
>> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdmu
>> r%2F&data=05%7C01%7CA.J.Godfrey%40massey.ac.nz%7C8f0eb72383f74148cef0
>> 08db27e01261%7C388728e1bbd0437898dcf8682e644300%7C1%7C0%7C63814761557
>> 0986760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL
>> CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6O%2BFOsnLvWvYHVE
>> ElsNJ0cCbVcs1N33xHsv8YAjHf%2FE%3D&reserved=0
>> doch.github.io%2Frgl%2Fdev%2Farticles%2Frgl.html&data=05%7C01%7CA.J.G
>> o
>> dfrey%40massey.ac.nz%7Cc0745ee90c414eebafa508db273e28e6%7C388728e1bbd
>> 0 
>> 437898dcf8682e644300%7C1%7C0%7C638146920163482319%7CUnknown%7CTWFpbGZ
>> s 
>> b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
>> D 
>> %7C3000%7C%7C%7C&sdata=tgPjXHerM93yrUrb963Tsa1MwsY7AKbFzaAFP8ZRYc8%3D
>> &
>> reserved=0
>>
>> The very first graphic is a plot of the iris data, and it should have alt text reading "rglwidget in chunk unnamed-chunk-1".
>>
>> That was default text generated without user input, users could put whatever they wanted there instead.
>>
>> Some related questions:
>>
>>     - Could you suggest better default text to use?
>>
>>     - Could you suggest good custom text for this particular graphic?
>>
>>     - The text is inserted when some Javascript has run to initialize the page, so if the Javascript fails, there'll be no alt text.  Is that a reasonable limitation?
>>
>> Duncan Murdoch
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsta
>> t%2F&data=05%7C01%7CA.J.Godfrey%40massey.ac.nz%7C8f0eb72383f74148cef0
>> 08db27e01261%7C388728e1bbd0437898dcf8682e644300%7C1%7C0%7C63814761557
>> 0986760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL
>> CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yxBjLllMyBklfekVB
>> SFESvDi7wc2AInXELdc9OjSMm4%3D&reserved=0
>> .ethz.ch%2Fmailman%2Flistinfo%2Fr-package-devel&data=05%7C01%7CA.J.Go
>> d
>> frey%40massey.ac.nz%7Cc0745ee90c414eebafa508db273e28e6%7C388728e1bbd0
>> 4 
>> 37898dcf8682e644300%7C1%7C0%7C638146920163482319%7CUnknown%7CTWFpbGZs
>> b 
>> 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
>> % 
>> 7C3000%7C%7C%7C&sdata=DTNqoAST5GrdK70Y%2B6X0gSOgn0dQQoXEN6cIR608YyE%3
>> D
>> &reserved=0
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsta
>> t%2F&data=05%7C01%7CA.J.Godfrey%40massey.ac.nz%7C8f0eb72383f74148cef0
>> 08db27e01261%7C388728e1bbd0437898dcf8682e644300%7C1%7C0%7C63814761557
>> 0986760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL
>> CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yxBjLllMyBklfekVB
>> SFESvDi7wc2AInXELdc9OjSMm4%3D&reserved=0
>> .ethz.ch%2Fmailman%2Flistinfo%2Fr-package-devel&data=05%7C01%7CA.J.Go
>> d
>> frey%40massey.ac.nz%7Cc0745ee90c414eebafa508db273e28e6%7C388728e1bbd0
>> 4 
>> 37898dcf8682e644300%7C1%7C0%7C638146920163482319%7CUnknown%7CTWFpbGZs
>> b 
>> 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
>> % 
>> 7C3000%7C%7C%7C&sdata=DTNqoAST5GrdK70Y%2B6X0gSOgn0dQQoXEN6cIR608YyE%3
>> D
>> &reserved=0
> 