Bad labels made AI agents work harder, then fail the form

01

WHAT ACTUALLY CHANGED

AudioEye published its study and accompanying company release on September 24, 2026.

The company ran 1,560 independent AI-agent sessions built from six commercial models.

The evaluation covered thirteen tasks across six websites, with every agent starting fresh and each test repeated ten times.

Five sites came from retail, travel and direct-to-consumer commerce.

On those five sites, AudioEye says the only experimental change was whether its accessibility fixes loaded. The underlying websites were not otherwise modified.

The sixth site was the W3C Before and After Demonstration, which provides paired accessible and inaccessible versions for education and testing.

On the site with the most accessibility issues, task completion was 31 percent without the fixes and 96 percent with them.

AudioEye describes the change as an up to 68 percent relative drop when accessibility fixes were absent.

The 31 and 96 percent figures describe one site, not the average across all six websites.

Across all 1,560 sessions, the median run used 128,000 tokens without accessibility fixes and 90,000 with them.

That is the basis for AudioEye's reported 43 percent median token increase on inaccessible versions.

On the most inaccessible site, every tested model used at least twice as many tokens without the fixes, according to the company.

The largest reported model-level difference on that site was six times as many tokens.

One task required numbers shown only inside an image that lacked a text alternative in the code.

All six models failed that task in all ten repetitions, producing sixty failed attempts.

AudioEye says results were rechecked with WebJudge, an open-source evaluation system from Ohio State University's NLP group.

The WebJudge recheck agreed with AudioEye's task scoring 95 percent of the time.

The public materials do not name all six models, identify the five commercial sites or provide the complete prompt, trajectory and score record for every run.

02

WHY THIS MATTERS

Visual design and machine-readable structure are different things. A control can look perfectly obvious while remaining unnamed to anything that reads the page programmatically.

Screen readers use semantic information to announce what an element is, what it is called and what state it is in.

Many browser-using agents also consume accessibility-tree representations because they compress a page into structured controls and relationships.

A missing accessible name turns a clear-looking button into an ambiguous action such as button with no useful purpose attached.

An image containing prices, dates or quantities can be visible to a sighted person while remaining absent from the programmatic record.

More model reasoning cannot reliably recover information that the interface never exposes.

The sixty failed attempts matter because they show repeated search and larger token budgets do not necessarily repair a missing input.

Higher token use also converts a quality problem into a cost and latency problem.

An agent that retries, scrolls, inspects and guesses may consume more compute while becoming less predictable.

The same semantic repair can improve keyboard navigation, screen-reader use, automated testing and agent operation without requiring four separate versions of the page.

That makes accessibility work part of core product architecture rather than a compliance layer pasted on at launch.

The business case must not replace the civil-rights case. A page should work for disabled people even if no agent ever visits it.

Agent compatibility can supply additional budget and executive attention, but disabled users should remain involved in defining whether the result is actually usable.

Passing an automated accessibility check does not prove that a real person can complete a workflow efficiently or understand the content.

Likewise, an accessible tree does not guarantee that an agent will reason correctly, respect authorization or recover from a changed page.

AudioEye's result is commercially interested evidence, so buyers should reproduce the task suite on their own pages before projecting the percentages into a budget.

The most transferable finding is architectural: when meaning lives only in pixels, every nonvisual user is asked to guess.

03

WHERE IT COULD HELP

• Give every interactive control a clear programmatic name that matches the visible label.
• Use native buttons, links, inputs, headings, tables and landmarks before recreating their behavior with generic containers.
• Add concise text alternatives for meaningful images and keep decorative images out of the accessibility tree.
• Never place a required price, date, quantity or instruction only inside an image.
• Associate every form field with its label, instructions, required state and validation message.
• Expose expanded, selected, checked, disabled and busy states programmatically as they change.
• Keep keyboard focus visible and move it deliberately after dialogs, validation failures and dynamic updates.
• Preserve a logical heading order and landmark structure so users and agents can scan before acting.
• Test forms, bookings and purchases using only a keyboard and a mainstream screen reader.
• Inspect the browser accessibility tree, not only the rendered DOM and screenshots.
• Create agent tests that record the selected control, accessible name, action, result, token use and failure reason.
• Run the same task against the production page before and after a semantic fix while holding prompts and model versions constant.
• Separate failures caused by missing page information from failures caused by model planning or authorization.
• Include disabled people in usability studies and pay them for expertise rather than treating automated scans as their substitute.
• Block releases when a critical task becomes keyboard-inaccessible or loses required labels.
• Publish task-level results and failure traces when making performance claims about accessibility and agents.
• Measure completion, retries, time, tokens and unsafe guesses instead of celebrating a single successful demo.
• Treat vendor overlays as one intervention to evaluate, not as permission to stop fixing source components and content workflows.