Manual vs AI Accessibility Testing: Why Plugins Alone Won’t Guarantee an Inclusive Website
Automated accessibility tools have made it easier to quickly find obvious problems. Missing alt text, low contrast, unlabelled form fields, a decent scanner covers these in seconds across hundreds of pages. That is genuinely useful.
But these very same tools only catch around 30-40% of real accessibility issues. The rest require a human to find them. Here is what that means in practice.
What the automated scanners do well
These types of scanners can provide a good start for finding issues. However, automated scanners can only look at the structure of the website. They can identify contrast issues and missing labels (most of the time), however, they are very consistent at identifying this type of issue at scale.
Due to the fact that automated scanners can run against hundreds of pages on large websites, they can also provide a high-level overview of a website's current state of accessibility. This provides a baseline to measure future improvements and identifies quick wins for development teams.
But remember, automated scanners are a floor, not a ceiling.
What the automated scanners do poorly
Automated scanners can only look at the code structure, and therefore do not interact with the website like a human would. While an automated scanner may be able to determine if an element has an aria-label, it will not be able to evaluate whether the aria-label makes sense contextually, or whether the overall experience created by the element is logical. Automated scanners often fail to identify:
Keyboard Focus Traps
Where a user tabbing through a webpage becomes trapped within a Modal or Carousel, and cannot exit.
Illogical Heading Orders
Where headings are present, but are ordered in a manner that creates confusion when reading the webpage in sequence by a Screen Reader.
Confusing Form Flows
Where fields are labelled, but the overall flow of the form fails to work properly once submitted.
Hidden Buttons or Links
Buttons or links that are invisible at certain screen sizes or interaction states.
Dynamic Content Updates
These do not announce themselves to assistive technologies.
It is possible to have a website that passes all of the automated checks, yet is completely unusable to a Screen Reader User. This is a known reality, and one of the many reasons why relying solely on automated scanning is not enough for any serious Accessibility Program.
What Manual Testing Finds
Manual testing involves a trained tester who navigates the website using various assistive technologies including Keyboard-Only Navigation, Screen Readers (such as NVDA, JAWS, VoiceOver), Magnification Software, and Mobile Accessibility Tools.
The goal is not just to identify technical failures. It is to experience the site the way a real user does, to find the barriers that make tasks difficult or impossible, not just the attributes that are missing or malformed.
Manual testing can help identify:
- Confusing keyboard paths: Catching focus order problems that trap users in a loop or send them to the wrong part of the page.
- Contextually useless alt text: Spotting images that have descriptions, but don't actually explain why the image is there. A picture of a "blue button" doesn't help if the button's actual purpose is "Download Q3 Report."
- Misleading announcements: Catching labels that exist in the code but make absolutely no sense when read aloud by a screen reader.
- Useless error messages: Identifying those generic "Invalid input" alerts that tell a user something is wrong but offer zero help on how to resolve it.
- ‘Technically correct’ forms: Pinpointing forms that pass an automated scan but are a nightmare to use in the real world, like those that time out too quickly or clear all your data if you make one mistake.
- Shifting layouts: Noticing when a page re-sorts or changes its structure mid-interaction, which can be incredibly disorienting for anyone using assistive tech.
- Hidden interactive elements: Finding things like ‘hover-only’ menus that a keyboard user or someone with a touch screen can't actually open.
- Vague link text: Fixing links that just say ‘click here’ or ‘read more’. These are useless for someone scanning a page with a screen reader who needs to know exactly where that link is taking them.
One of the most important additions to any manual testing program is to involve people with lived experience of disability. These individuals will identify barriers that trained testers (working from WCAG criteria) may not anticipate. They know what meaningless alt text sounds like when read aloud, they know when a dropdown closes too quickly for someone who requires additional time to interact with it, and they will notice issues that no automated checklist will ever capture.
Accessibility Overlays: A Word of Caution
There are numerous plugins, apps, and overlays available that claim to make your site accessible with a single install, scan, or code injection. Some of these products appear to offer on-the-fly fixes for keyboard navigation, contrast adjustments, and auto-generated alt text. Treat these with caution.
Most overlays do not fix underlying code issues; they just attempt to mask them at the surface. They have repeatedly failed under legal scrutiny. Reports show that hundreds of businesses are sued every year despite having these widgets installed. Because they rely on JavaScript, they can be disabled or bypassed entirely, leaving your site just as vulnerable as it was before.
An accessibility overlay is not an accessibility audit. It is not WCAG compliance. It is definitely not a substitute for fixing your code.
A Practical Approach
The most effective methodology to ensure website accessibility is to combine both methods.
First, start with an automated scan. Automated scanning tools such as WAVE, Axe, Google Insights, and our very own scanner here at Accessima quickly allow developers to identify structural errors.
Second, getting an independent accessibility audit will give you an overall picture of your website’s level of accessibility. The results will include details about every accessibility error, their level of severity, and provide you with specific guidance as to how you can address each issue.
If you'd like us to review your site and explain everything in plain English, book a free consultation! We'd be more than happy to guide you through our basic evaluation process.
