AXE Passes, But It's Still Unusable: The Accessibility Bugs Automated Tools Can't Catch
There's a particular kind of false confidence that comes from a green accessibility report. The CI check passes, the Lighthouse score says 100, the AXE panel is all ticks. You ship it. And somewhere, a person using a screen reader hits your component and simply can't.
I've shipped that component. More than once. Here's what I learned the hard way: automated accessibility testing checks whether your HTML is well-formed, not whether your interface is usable. Those are very different questions, and the gap between them is where real users get stranded.
AXE is genuinely excellent - wire it into CI today if you haven't. But it works by inspecting a static snapshot of the DOM and checking it against a ruleset. It can see that an attribute is present. It cannot see whether the attribute is true, whether the sequence makes sense, or whether anything actually happens when you press a key. It catches roughly a third of WCAG issues. The other two-thirds need a human.
Let me show you the gap, with five components that all pass AXE clean.
1. The modal that swallows your focus
A dialog opens, you confirm, it closes. AXE is delighted: role="dialog", aria-modal, labelled heading, focus trapped while open. All correct.
confirm() {
this.save();
this.open.set(false);
// ✅ AXE passes. ❌ focus just fell to <body>.
}
Here's what a keyboard user experiences: the dialog vanishes and their focus lands on <body>, at the top of the page. They've lost their place entirely and have to Tab back through the whole UI. AXE can't catch this, because where focus goes after a state change isn't in the DOM snapshot - it's in time.
The fix is to remember the trigger and restore to it:
private trigger: HTMLElement | null = null;
openDialog() {
this.trigger = this.doc.activeElement as HTMLElement;
this.open.set(true);
}
close() {
this.open.set(false);
this.trigger?.focus(); // put them back where they were
}
2. The error you can only see if you can see
A form field fails validation. A red border appears. AXE checks the label is present and the border colour has enough contrast - both fine. Green. But the only signal that something went wrong is colour. A colour-blind user sees nothing. A screen-reader user hears nothing, because the message was never wired to the field or announced:
<!-- passes AXE, communicates nothing -->
<input formControlName="email" class="error">
<p class="error-text">Enter a valid email</p>
Nothing connects that <p> to the input, and nothing tells assistive tech it appeared.
The fix is to make the error programmatic:
<input
formControlName="email"
[attr.aria-invalid]="showError()"
[attr.aria-describedby]="showError() ? 'email-error' : null"
>
@if (showError()) {
<p id="email-error" role="alert">Enter a valid email.</p>
}
Now aria-invalid states the field is wrong, aria-describedby ties the explanation to it, and role="alert" makes it announce the moment it renders. None of those three things is something AXE will ever require - it only knows the label exists.
3. The toggle that looks operable and isn't
This one is sneaky, because the ARIA is perfect:
@Component({
host: {
'role': 'switch',
'[attr.aria-checked]': 'checked()',
'tabindex': '0',
// …no keyboard handler.
},
})
AXE sees a valid switch with a valid aria-checked state and a tabindex. It has no way to know that pressing Space does absolutely nothing, because "is this control actually operable by keyboard" requires running it, not reading it. A mouse user toggles it happily; a keyboard user can focus it and then... nothing.
host: {
'role': 'switch',
'[attr.aria-checked]': 'checked()',
'tabindex': '0',
'(keydown.space)': 'toggle(); $event.preventDefault()',
'(keydown.enter)': 'toggle()',
}
(Or - better - wrap a native <button> and skip the role entirely. The native element can't have this bug.)
4. The live region that never speaks
You added an aria-live region for "12 results" after filtering. Textbook. Except you did it like this:
@if (announcement()) {
<p aria-live="polite">{{ announcement() }}</p>
}
AXE sees a valid live region and approves. But screen readers only announce changes to a live region that was already in the DOM when the change happened. By @if-ing the whole element in at the same moment as the text, you've defeated the mechanism - the region and its content arrive together, and nothing is announced.
The region has to exist empty, then receive text:
<p class="sr-only" aria-live="polite">{{ announcement() }}</p>
Or skip the footgun and use the CDK's LiveAnnouncer, which maintains a persistent region for you:
private announcer = inject(LiveAnnouncer);
this.announcer.announce(`${count} results`, 'polite');
This is my favourite example because the code looks more correct than the version that works. Tooling can't tell the difference; a screen reader tells you immediately.
5. The chart that doesn't exist
You render a beautiful SVG line chart. Contrast is AA, there's a <title>, AXE is satisfied. To a screen-reader user, it is a blank rectangle. SVG internals aren't exposed as data, and "this picture conveys information that exists nowhere else on the page" is not a rule any automated tool can express.
The only real fix is to provide the same data in a form assistive tech can read - a visually-hidden table behind the visual:
<figure>
<svg aria-hidden="true">…</svg>
<table class="sr-only">
<caption>Monthly revenue, Jan–Dec</caption>
<tr><th>Month</th><th>Revenue</th></tr>
<!-- one row per point -->
</table>
</figure>
This is the single biggest blind spot in automated testing, and it's why data visualisation is the hardest thing in front-end accessibility.
The only test that actually works
Notice the pattern across all five: AXE was right every time. The HTML was well-formed. The bug was always in something a snapshot can't see - sequence, interaction, timing, meaning.
So keep AXE in CI. It's a brilliant regression net for the third of issues it covers, and it's free. Just don't mistake it for the test.
The real test takes two minutes:
- Put the mouse down and Tab through it. Can you reach and operate everything? Does focus go somewhere sane after every action?
- Turn on a screen reader - VoiceOver (⌘+F5) or NVDA - and listen. Does what you hear match what's on screen?
If those two passes are clean, you've covered the two-thirds that matter most. If you only run the automated check, you've verified your HTML is tidy and learned almost nothing about whether a real person can use what you built.
This gap is exactly why I built NgBracket the way I did - every component is driven by keyboard and screen reader before it ships, not just run through AXE. Accessible-by-default means the two-thirds the tools can't see are already handled, so a green CI check is telling you the truth for once.
Comments
No comments yet. Start the discussion.