Use aria-labelledby for figures instead of alt/aria-label

[xworld21]

Tentative improvements towards #1473, #1440 (cc @nxg).

The current use of the figure caption as alt text for images has two problems. One, it is semantically wrong: the caption is visible so it shouldn't be repeated (a screen reader will read it twice!). The other is that it picks the first text node of the caption, so the text may well be truncated.

I modified the schema so that ltx:caption gets an ID. Then the image gets an @aria-labelledby containing that id, instead of trying to copy the caption into the alt text. This seems to be the correct way of doing things, as far as I can tell [1]. @alt is always set to @description, and so for now it is always empty since there is no mechanism to set it until #1452 is resolved. The treatment of <object> can be further improved with aria-describedby, but one thing at a time!

Sending this as draft as I haven't tested this in all cases, like with labels, and I have to check with a screen reader. Also, it probably needs to be generalised to all environments with captions (e.g. tables), but I'd prefer to hear some comments before moving forward.

[1] https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-labelledby_attribute

[dginev]

I've recently been experimenting with aria-labelledby and aria-describedby for MathML formulas and they seem quite workable, though what screen readers end up doing with them can be a bit inconsistent - and may be worth examining across AT tools.

If I'm reading one tutorial on accessible images correctly, the basic reality for latexml is that we don't have enough information to set the alt attribute, and should leave it empty. What we ought to use may be aria-describedby pointing to the caption, rather than the more mandatory aria-labelledby.

An ideal alt image for a scientific plot may be e.g. " a loglog line plot with X axis going from 0 to 10,000 and Y axis going from -1 to 1. First data series covers ...", essentially narrating the details of the plot components. While some neural approaches have gotten quite good at this for imagenet images, we'd need some novel research to get that for diagrams. Or let authors do this as you've suggested in #1452 . So for now an empty alt is most honest.

Would the aria-describedby attribute fit better in your PR?

[xworld21]

What we ought to use may be aria-describedby pointing to the caption

Should @aria-labelledby point at the tag, then?

Note that there is also @aria-details for the really long description, so label => tag (~ @title), description => caption (~ @alt), details => long description seems like a good default.

(Necessary reference: https://www.w3.org/TR/wai-aria/#aria-labelledby. )

[dginev]

Do we even need the label? I think that serves a similar purpose to the alt in that it should have a narration of the exact constituents of the figure. The tag is visible on the page already right? So it ought to be read by the AT tool as-is.

I think the only attribute we can try to set is the aria-describedby pointed at the caption, by default. And even that is somewhat suboptimal since the caption is visible on the page, so it doesn't really enhance the AT in any material way.

Unless the author adds annotations, or we do some new research magic to auto-generate what the figure <img> contains, we can't set any of the AT attributes in a helpful way, or so I suspect.

Edit: Note on practical realities. It turns out one is best off double-checking any claims about the way attributes are meant to be used by their specs with the concrete AT tools that exist today in 2021, so that we merge PRs that deliver quality of life improvements to real people today. I had to learn this the hard way for the <math> element, where the screen readers are in a bit of an impasse with what to do, and there's some experimentation to make a more useful annotation. We're still working on standardizing the details there in the Math Working Group, so nothing stable yet. But for images - it ought to be stable already, so we might as well check some AT tools? Especially when we get to annotations from authors and how they get turned to speech/Braille.

[xworld21]

I am checking my old PRs, trying to make sense of them... and making sense of the ARIA attributes is quite hard!

The additional challenge I didn't notice is that @aria-(labelled|described)by must both point to pure text content, while the caption can contain arbitrary content. So the only safe attribute is @aria-details? But that doesn't seem right either!

Suggestion: can we just add an ID to the caption, so that users have the options to change the behaviour via XSLT?

[dginev]

I am still not sure the figure <img> tags need any annotations beyond a simple alt attribute in the default LaTeX use case.

I also still do not know - but would like to eventually find out - what AT implementations currently do for the various suggested uses of ARIA, as well as for a simple img and figcaption combination without ARIA. But that requires me to get a Windows installation with NVDA or JAWS to check the mainstream vendors, which I am not yet technically prepared to do...

So, instead, I checked what some legitimate modern e-journal platforms do today, using the browser inspector. Some examples with links to randomly sampled articles:

• eLife uses an empty alt="" on all its figure images, and lets the <figcaption> captions do the a11y work on their own.
• PLOS One seems to have dedicated figure pages with aria-hidden="true" making them fully silent, which isn't too helpeful
• Nature has some actionable markup:

  <img alt="extended data figure 1" aria-describedby="rc-Fig1" src="...">

  with the aria-describedby referencing a <b> element inside the caption:

  <figcaption><b class="c-reading-companion__figure-title u-h4" id="rc-Fig1">
    Fig. 1: Antarctic coastal change since 1997.</b></figcaption>

• Science hides its figures in a "supplementary PDF", so they just dodge the question entirely. (a handful of other journals I tried also skirt away from the question, won't namedrop those...)
• biorXiv's HTML auto-assigns an alt attribute that follows the TOC tag, as in <img alt="Fig. 1." ....
• AMS uses an alt default similar to the one Bruce chose in his latest PR,
  namely <img alt="Graphic without alt text" ...

So I think this somewhat confirms to use alt and to keep the use of ARIA limited to an aria-describedby pointing at the caption. From the tour I took, only Nature seems to have started using ARIA for figures (then again it is Nature). But they demonstrate you can use both alt and ARIA, which was my original point - we should just embrace both, and use them whenever they accomplish a better experience.

I have now also seen the dedicated infobox @xworld21 mentioned, which warns to use aria-details if the target contains anything beyond plain text. So aria-details would be the safer default for scientific content, as e.g. just having MathML elements in the caption goes beyond the text-only restriction. Unsure if it is worth doing a check of the contents in the XSLT to determine which attribute is more appropriate automatically...

---

Suggestion: can we just add an ID to the caption, so that users have the options to change the behaviour via XSLT?

Curious, I hadn't realized the captions don't have IDs, not even when explicitly requested via --preload=[ids]latexml.sty. That leads to some funny results in ar5iv, where you may have a span holding an italic/colored piece of text in a caption, which will have its own id attribute, but the caption holding it will not.

I'm somewhat neutral on the general question, but see no reason to prohibit ids on <figcaption>, as global attributes are allowed - well - globally in HTML.

[brucemiller]

I'm not sure why ids on things inside non-id'd captions is "funny"; you can still reach the caption using any other selector or xpath mechanism, which works for everything except aria (which is somehow a recurring theme, isn't it?).

Somehow putting ids on captions is uncomfortable; most things with ids have some kind of referencing mechanism associated. Most latex-y things cooperate with \label so there's the labelled.attributes list; ah, but putting a \label inside a caption works quite differently --- it refers to the figure (or table). So, currently the LaTeXML schema doesn't even allow id's there. We might contemplate allowing ids, but adding them to all captions will cause lots of churn in the test suite (which is kinda annoying). And unless we go to the other extreme of requiring the ids, then the XSLT for aria can only link up the caption if there is an id.

Is it reasonable to leave the id creation up to the XSLT?

[dginev]

I'm not sure why ids on things inside non-id'd captions is "funny";

I was already in a "funny mood" due to the way Nature was doing something similar that is correct on an operational level, but has entertaining semantics. In:

<figcaption><b id="rc-Fig1">
  Fig. 1: Antarctic coastal change since 1997.</b></figcaption>

where the aria-describedby="rc-Fig1" points at the <b> element, a "purist reading" would suggest that it is not the caption describing the figure, but the bold text in it. Which I indeed find funny. That would be comparable to an imaginary solution for latexml where we do:

<figcaption><span id="span.id.N">...some caption text...</span></figcaption>

It is not technically wrong, but it is - at the least - conceptually a bit funny.
Apologies if my amusement was confusing.

ARIA has indeed made the choice of working exclusively with the global id attributes for its long-range dependencies, but I don't think they are alone in that choice. Our Presentation+Content MathML cross-referencing is employing the exact same technique from the MathML side, and as you mention - so do the href values for \ref references and the citation machinery. No XPath/selector options there either, as far as HTML interactivity is concerned.

---

All that said, I think we can focus on the improved alt from #1924 and defer the ARIA side until a later time, where we have people actively testing with screen readers. I suspect just the presence of <figcaption>, the way we have things organized now, will be of good baseline quality for the figure readouts. The aria-describedby/aria-details annotations may offer some additional interactivity options, but seem somewhat optional.

[xworld21]

Is it reasonable to leave the id creation up to the XSLT?

Somehow I did not think I could generate the id from XSLT. That's a very appropriate solution on the user side, no need for new code in latexml.

where we have people actively testing with screen readers

I used to do occasional testing with VoiceOver, ChromeVox, TalkBack. I also have a laptop with NVDA now, but I haven't figured it out yet. I could try again on the journals you mentioned.

Given the latest alt patch, I think this PR can be closed, and the ARIA question moved to an issue?

[dginev]

So, instead, I checked what some legitimate modern e-journal platforms do today, using the browser inspector. Some examples with links to randomly sampled articles:

Hi all. I wanted to add a follow-up comment here to my small survey of what publishers do with alt-text on figures in 2022, as I found a modern example, advertised by ACM on twitter. The annotation is at this article:
https://dl.acm.org/doi/fullHtml/10.1145/3517428.3544796

Here is a screenshot from the browser inspector over Figure 1:

ACM uses aria-describedby to deposit a long textual overview of a figure, which an author deposited in their TeX source.
They also have a guide on describing figures. And they describe their custom \Description macro here:
https://sigchi.org/conferences/author-resources/accessibility-guide/#for-latex-users

used as:

\begin{figure}
\includegraphics{figure1.svg}
\caption{Your figure caption here.}
\label{fig:label}
\Description{A full description of the image here}
\end{figure}

Maybe we should try to find a way to connect them to the LaTeX team's official efforts towards creating new techniques for annotating accessibility, given that they have some prior work in LaTeX authoring?

[brucemiller]

I'm probably missing a lot of subtlety in the discussion, but given all the developments, I'm wondering if we don't have a decent solution sketched out here?

• For an ltx:graphic, if the alt keyword has been given, we use that as the alttext for the img. Otherwise we either leave it off, or set it empty to quiet validators. In any case, we should not copy the caption.
• For a figure, we add an id to the caption, and place (somewhere?) aria-describedby=id.

Does that get us close to reasonable, if not perfect, accessibility?

[dginev]

My position here continues to be:

All that said, I think we can focus on the improved alt from #1924 and defer the ARIA side until a later time, where we have people actively testing with screen readers.

I think we don't have enough macros at present to do this right, and we may hardwire some pieces together that will seem unfortunate in the long run. For an author who truly wants to add accessible descriptions to their figures, the final HTML result will likely have the caption as an entirely separate construct that isn't used for accessibility. I think ACM are onto something with their \Description macro, as the language needed for high-quality readouts is not the same as the caption language.

[xworld21]

Two years on, I now agree with @dginev: using the correct ARIA attribute is not possible without the author's help – and even then it is not trivial (one has to pick between labelledby, describedby, and details).

I think that an empty alt='' is a good default: if there is a caption, it's going to be read by screenreaders anyway (and so I disagree with my younger self apparently). At this point, it's more important to stop copying part of the caption in @alt, it duplicates the text when it works, it's confusing when it doesn't.

[brucemiller]

OK, so we have a clear consensus that the best, correct, accessibility requires author (& markup) support. We now can get that with the alt keyword for graphics images.

Over time, authors will start supplying that information. Likely other constructs will also acquire a similar bit of markup enhancement. Likely in the form of a simple chunk of descriptive text, rather than some complicated data structured specifically for aria (at least for non-math). So, we can start thinking about how best to use what we're given.

Question 1: Although the LaTeX folks were clearly motivated by HTML's alt attribute for img, the alt keyword allows for more than simple strings; markup, even math. Should we stick with flattening it to a string and copy it to the alt attribute? Or should we make a place for an ltx:accessible-description element? If the latter, we'll have to make a connection between the img and description using some aria mechanism. (For eventual non-img descriptions, the attribute approach assumes an appropriate HTML attribute, while the latter aria approach can be adapted to any object).

Question 2: What do we do for the currently common case where there is no alt keyword, or other specific markup for author supplied accessibility? We've got consensus that we shouldn't copy the caption to the alt attribute. But, should we make alt empty? Should we use same aria mechanism from Question1 to point to the caption, with the assumption that it is better than nothing?

[xworld21]

Question 1: Although the LaTeX folks were clearly motivated by HTML's alt attribute for img, the alt keyword allows for more than simple strings; markup, even math. Should we stick with flattening it to a string and copy it to the alt attribute? Or should we make a place for an ltx:accessible-description element? If the latter, we'll have to make a connection between the img and description using some aria mechanism. (For eventual non-img descriptions, the attribute approach assumes an appropriate HTML attribute, while the latter aria approach can be adapted to any object).

That sounds good to me. We have to be careful though. My understanding is that aria-label and aria-describedby must still point to text-only descriptions; if you put anything more complicated, it gets flattened out to text. So a screen reader won't let you navigate a list, for example. Apparently we are supposed to use aria-details for structured content.

For when it is pure text, it can be put back in @alt. Presumably XSLT can distinguish between the two. For structured text, the challenge becomes where to put that content. Also I don't understand from the spec if the elements referred to in @aria-details can be hidden (as in display: none). Although good practice is to make descriptions available to everybody, not just screen reader users!

Question 2: What do we do for the currently common case where there is no alt keyword, or other specific markup for author supplied accessibility? We've got consensus that we shouldn't copy the caption to the alt attribute. But, should we make alt empty? Should we use same aria mechanism from Question1 to point to the caption, with the assumption that it is better than nothing?

The problem is that @alt is required for valid HTML5. But that should be responsibility of the authors after all. What about:

• if there is no caption: do whatever the author asked for (use @alt only if the author specified alt=, whether empty or not)
• if there is a caption: have @alt empty, or, have @alt replicate the label, like e.g. Figure 1, but not the entire caption?

In this way, captioned images are considered valid in all situations; uncaptioned images with no @alt will be flagged by validators and accessibility checkers.

Some references:
https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Attributes/aria-details
https://w3c.github.io/aria/#aria-details