What Automated Accessibility Testing Can Catch, and What Developers Still Need to Test Manually

Accessibility automation is incredibly useful. Run a scanner against a page and within seconds you may find missing labels, contrast problems, ARIA mistakes, structural issues, and other WCAG-related signals. For developers, that feedback loop is valuable. The problem starts when we confuse: "No automated failures found" with: "This interface is accessible." Those are very different statements. W3C's guidance is explicit that accessibility evaluation tools can quickly identify potential issues, but they cannot automatically check every aspect of accessibility. Human judgment is still required. The practical takeaway for developers is simple: Automate what is deterministic. Manually verify what depends on behavior, context, and actual user interaction.

Missing Alternative Text

Automation is very good at detecting missing alt attributes. Consider:

<img src= "/products/red-chair.jpg" >

An automated accessibility checker can reliably detect that the image does not have an alt attribute. The technical issue is deterministic. A possible fix might be:

<img src= "/products/red-chair.jpg" alt= "Red upholstered lounge chair with wooden legs" >

But even when an alt attribute exists, automation cannot always determine whether the text communicates the right meaning. For example:

<img src= "/products/red-chair.jpg" alt= "image123" >

Here, the attribute exists, so a simple technical presence check may pass. However, "image123" is probably not useful alternative text. Determining whether the text conveys the intended meaning requires understanding the purpose of the image-whether it is decorative or informative-and choosing the appropriate implementation, such as:

<img src= "/decorative-divider.svg" alt= "" >

This highlights two separate questions: Can automation detect that an attribute is missing? Often, yes. Can automation always determine whether the text communicates the right meaning? No-that second question requires context.

Form Labels

Automation catches the association between inputs and labels. Consider:

<input type= "email" name= "email" placeholder= "Email address" >

The placeholder is visually useful, but the input has no persistent programmatic label. A better implementation pairs a <label> with the input:

<label for= "email" > Email address </label>
<input id= "email" type= "email" name= "email" autocomplete= "email" >

Beyond basic labeling, form accessibility extends to validation feedback. When validation fails:

<p class= "error" > Invalid value. </p>

A developer must then ask deeper questions: Does the user know which field failed? Is the error associated with that specific field? Does a screen reader announce the error? Does focus move somewhere unexpected? Can the user correct the problem without losing previously entered information?

Even when an accessible name exists, the implementation may be insufficient. Consider an icon button:

<button aria-label= "Open" > <svg aria-hidden= "true" > ... </svg> </button>

A scanner may confirm the button has a role and an accessible name. Technically, there is something for assistive technology to announce. But "Open" provides very little context if the page contains multiple similar controls. A more meaningful label would be:

<button aria-label= "Open billing settings" > <svg aria-hidden= "true" > ... </svg> </button>

This illustrates that accessibility testing goes beyond binary attribute-validation exercises and requires evaluating whether labels actually convey meaningful purpose.

Custom Controls and Keyboard Behavior

Custom implementations often hide their limitations. A common pattern uses a div with ARIA roles and event handlers:

<div role= "button" tabindex= "0" aria-label= "Add to cart" onclick= "addToCart()" > Add to cart </div>

At first glance, this looks accessibility-aware-it has a role, tabindex, and an accessible name. However, depending on the implementation, nothing may happen when the user presses Enter or Space. The simplest fix is often native HTML:

<button type= "button" onclick= "addToCart()" > Add to cart </button>

Native elements bring substantial built-in browser behavior. This is one reason accessibility bugs frequently surface when teams recreate native controls with generic elements.

Actual keyboard testing reveals gaps that scanners miss. Try using the element without a mouse: Tab to it, then press Enter or Space. Without proper keyboard handling, nothing happens. WCAG 2.2 Level AA requires a mode where keyboard focus is visible. A reasonable CSS pattern ensures focus visibility:

button :focus-visible,
a :focus-visible,
input :focus-visible {
  outline: 3px solid currentColor;
  outline-offset: 3px;
}

But even this code is not the end of the test. Navigate through the interface manually:

1. Open the page and set aside the mouse.
2. Press Tab repeatedly, then Shift+Tab, then Enter and Escape.
3. Ask: Can I always see where I am?
4. Does focus follow a logical sequence?
5. Does a sticky header cover the focused control?
6. Does opening a component unexpectedly move focus?
7. Does focus disappear inside a custom widget?

WCAG 2.2 also introduced the Level AA Focus Not Obscured requirement: keyboard-focused components must not be entirely hidden by author-created content. These problems are discovered much faster by actual navigation than by examining markup.

Modals and Interaction Flow

Modals represent areas where automated confidence can become dangerous. Consider:

<button id= "open-dialog" > Delete account </button>
<div role= "dialog" aria-modal= "true" aria-labelledby= "dialog-title" >
  <h2 id= "dialog-title" > Delete account? </h2>
  <button> Cancel </button>
  <button> Delete </button>
</div>

While the modal may look reasonable, several critical questions remain:

• When the modal opens, where does keyboard focus go?
• Can the user Tab into content behind the dialog?
• Does pressing Escape close it?
• After closing, does focus return to the control that opened it?
• Can a screen reader understand that the interaction context changed?
• Are background controls effectively unavailable while the modal is open?

These are not theoretical edge cases-they determine whether someone can actually use the component.

Dynamic Content and Status Messages

Visual changes can be obvious to sighted developers yet invisible to assistive technology. Suppose a user saves a profile and JavaScript updates a status element:

document . getElementById ( " save-status" ). textContent = " Profile saved successfully " ;

Visually, the message appears. A sighted developer sees it and considers the flow complete. But someone using a screen reader may receive no indication that anything changed. A better implementation uses an aria-live region:

<div id= "save-status" role= "status" aria-live= "polite" ></div>

Then update its text programmatically:

document . getElementById ( " save-status" ). textContent = " Profile saved successfully " ;

WCAG 2.2 Success Criterion 4.1.3 addresses status messages that need to be programmatically determinable so assistive technologies can present them without moving focus.

Contrast Calculations

Contrast is a strong automation candidate since software can calculate ratios given text and background colors. However, real interfaces complicate this. Text may appear over gradients, photographs, video backgrounds, hover states, disabled states, overlays, or dynamically selected themes. A static check may identify many straightforward failures, but a developer still needs to verify the states users actually encounter.

From Components to User Journeys

Most development teams organize work by components, but users experience workflows. Consider an ecommerce checkout with individual reasonably-performing components:

• Product page → Cart → Shipping form → Payment → Validation error → Confirmation

Even if every page passes automated checks, the complete journey can fail-for example, a cart drawer that doesn't manage focus correctly, validation errors not announced, a third-party payment widget with keyboard problems, focus jumping to the top of the page after submission, a confirmation that is visual but not announced, or a timeout modal interrupting keyboard users. Problems like these don't make sense when evaluated only as isolated scanner alerts. You need to execute the full flow. Accessibility testing becomes more valuable as interfaces grow more interactive.

A Practical Accessibility Testing Workflow

For most teams, I would not choose between automation and manual testing. Use both. A practical workflow is:

1. Automated scan - Run accessibility checks during development rather than waiting until release. Automation is cheap feedback; use it.
2. Reproduce the candidate - Do not blindly fix every scanner message. Understand what the tool actually observed. False positives, duplicate symptoms, and component-level repetition can turn one underlying issue into dozens of tasks.
3. Inspect semantics - Use browser developer tools and the accessibility tree where useful. Ask what role, name, state, and relationship assistive technologies are actually receiving.
4. Use the keyboard - This takes minutes and catches a surprising number of serious interaction bugs. Test sequences like Tab, Shift+Tab, Enter, Space, and Escape. Do not just verify that focus reaches a control; verify that the entire interaction can be completed.
5. Verify context - Ask whether labels, errors, instructions, link purposes, and state changes make sense to an actual user.
6. Fix the source, not every symptom - If 40 pages fail because the same navigation component is broken, you have one shared component problem with broad impact-not 40 independent remediation tasks.
7. Retest the exact finding - "Code deployed" does not mean "accessibility issue fixed." Return to the original steps and verify the behavior.

Where Automated Scanning Fits

We recently built a free website accessibility scanner called Auditzo. It is designed as a first-pass technical check for one public webpage, covering supported automated WCAG 2.2 A/AA-oriented checks. It is not an ADA compliance certificate, not proof of complete WCAG conformance, and not a substitute for human accessibility testing. It is the first stage: Discover.

For issues that matter enough to investigate further, the workflow becomes:

• Discover → Verify → Remediate → Retest

Understanding whether you are looking at an automated candidate or a verified accessibility finding is important. If you want a broader breakdown of where automated testing ends and human accessibility review begins, we also provide a practical guide on Accessibility Scanner vs Manual Audit.

A Mental Model for Automation vs Manual Testing

When deciding whether a test belongs in automation, ask: Can the correct answer be determined reliably from machine-observable state alone?

• If yes, automate it. Missing attributes, deterministic relationships, parsable semantics, certain contrast calculations, and similar rules are excellent candidates.
• If no, test it with a human. Does the answer depend on meaning, context, sequence, interaction, or whether somebody can actually complete a task?

This model scales beyond accessibility. Good engineering automation handles repeatable facts. Human review handles ambiguity and context. Accessibility simply makes that boundary particularly visible.

Final Thought

Automated accessibility testing should be part of a modern frontend workflow. Run it early. Run it often. Put appropriate checks into your regression process. But do not let a clean automated result give you false confidence. A scanner can inspect a lot about a page. It cannot experience your application the way a person does. The workflow I keep returning to is:

Automation discovers. Developers reproduce. Humans verify. Teams remediate. Then we test again.

That is far more useful than chasing a perfect accessibility score. Auditzo provides automated website accessibility scanning and separately scoped human-reviewed accessibility evidence services. Automated scan results do not establish complete WCAG conformance, ADA compliance, legal compliance, or certification.