In collaboration with AI
The Short Answer
The lowest-cost model produced a usable first-pass SEO audit in all three tries. The two priciest models found every planted problem, but none of their reports met all our requirements without an edit.
Paying more did not buy a more usable report. GPT-5.6 Luna, the lowest-cost model, produced a report we could hand over without fixes in all three tries, at roughly one-fifteenth the estimated cost of GPT-5.6 Sol.
The premium models were thorough, not careful. Claude Opus 5 and Claude Fable 5 found all ten planted problems every time, but every one of their six reports either invented that a checklist was free or ran past the word limit.
Check the claims, not just the SEO. Seventeen of 18 reports caught all four critical problems. What separated usable reports from unusable ones was mostly a made-up detail.
Can a low-cost AI model write a technical SEO audit you could actually use? We gave six models the same fictional eight-page website, asked each for the audit three times and checked all 18 reports against the problems we had planted.
What We Asked the Models to Audit
The fictional company had an eight-page B2B training site. Every model got the same evidence: a crawl, robots.txt, an XML sitemap and 90 days of Search Console data. We also told the models what information was unavailable so they could not fill gaps with guesses. We planted ten problems, four of them critical.
The four critical problems
The courses page was blocked from crawling. The courses and industries pages had canonical tags, which name the main version of a page, pointing to the homepage instead. The pricing page was marked noindex even though the owner wanted it in search.
The six supporting problems
The sitemap listed a dead page and left out two live ones. A useful checklist page had no internal links. Several pages had duplicated metadata or a missing heading. The Search Console data showed which page should rank for which search.
We checked the recommendations against Google’s guidance on robots.txt, canonical URLs, noindex and sitemaps.
How Each Model Did
We ran three OpenAI models through Codex CLI and three Claude models through Claude Code, all at medium effort with web browsing off. Then we judged each report as if it were about to go to a client.
What “usable without fixes” means. A report counted only if it did all four of these:
- Caught all four critical problems.
- Caught at least eight of the ten planted problems.
- Stayed within the 1,800-word limit and the format we asked for.
- Stated nothing the evidence did not support.
The full review also required a weighted score of at least 75 and compliance with the time, correction and spending limits. Missing any one meant someone would have to correct the report before sending it. It does not mean the SEO advice was bad.
These figures use the test’s recorded API list prices and usage. They are historical price equivalents, not charges or current quotes. The per-1,000 figures are simple extrapolations, not a 1,000-audit trial. The runs added $0 in incremental cash through existing subscriptions; setup, review, correction and subscription allocation are excluded. Luna’s estimate was about one-fifteenth of Sol’s and one-thirty-seventh of Fable’s.
How to read “0 of 3.” It does not mean Opus and Fable were bad at SEO. They caught more problems than any OpenAI run. They failed on discipline: a detail they made up or a length limit they ignored. That still matters, because a client-facing report that promises a free download creates a business problem that no SEO fix solves.
What a Usable Report Looked Like
All 18 reports caught the robots.txt block. The usable ones tied each recommendation to a supplied fact and told the reader how to confirm the fix. This is the first priority item from one Luna report, copied without edits:
Affected URL: /courses/
Observed fact: “robots.txt access: disallowed”; robots.txt says Disallow: /courses/; canonical target is /; XML sitemap status is “included.”
Risk/inference: Search engines may be unable to crawl the page while being told it is a preferred sitemap URL, and its canonical conflicts with the owner’s desire for the courses page to appear.
Fix: If the owner still wants this page eligible, remove the /courses/ disallow and set its canonical to its own absolute URL. Confirm the page should remain publicly indexable.
Verify: Re-run a crawl and robots test; confirm allowed access, index,follow, self-canonical, and successful fetch.
Effort: small
Notice what it does: names the page, quotes the evidence, gives one fix and says how to check it. Every report that passed followed that pattern.
Why Reports Failed
The brief described a downloadable checklist. Six reports called it free, usually in a suggested page title such as “Free GMP Audit Checklist for Food & Supplement Facilities.” We never supplied a price or access terms.
One word did not change the technical diagnosis, but publish that title and the company has promised something it may not offer. All six came from Claude models: three from Fable, two from Opus and one from Sonnet.
One critical miss
One Sonnet report did not flag that the courses page canonical pointed to the homepage. Left alone, that could keep a valuable page folded into the wrong URL. The other 17 reports caught it.
Two reports ignored the length limit
Two Opus reports ran to about 2,100 words against a 1,800-word limit. More detail is not more useful when the reader asked for a short action plan.
What This Means for You
Start with a low-cost model for the first pass. On this structured audit with good evidence, it did the job. Keep a person responsible for reviewing the decisions that carry real risk.
If you use a premium model for its thoroughness, plan an edit pass. Check every price, promise and product detail against your own facts, and hold it to the length you asked for.
Give the model evidence, not a broad prompt. A crawl, robots.txt, your sitemap, Search Console data and what each important page should do. Tell it which facts are missing so it does not fill the gaps.
Ask for proof beside every fix, like the excerpt above, so a reviewer can trace each recommendation to the supplied facts.
Do not judge a model by one run. Sonnet’s best report was the top scorer in the test, and its worst missed a critical problem.
Do not implement it blindly. A real audit also needs the live site, analytics, backlinks, performance, schema, competitors and the limits of your CMS. AI gives you a fast first draft, not a finished plan.
Turn the first pass into a real implementation plan
NisonCo combines crawl data, Search Console evidence and human SEO review to find what matters, prioritize the work and verify the changes.
About This Comparison
Each of the six models got the same fictional website evidence and instructions three times, for 18 reports. We checked every report against the ten planted problems, the supplied facts and the requested format, and two reviewers scored the judgment calls without knowing which model wrote which report.
The OpenAI settings were configured model names; the CLI did not expose their backend model IDs. This was one synthetic site, not a live client audit. Three runs per model show real differences on this assignment, but not enough to rank the models for every SEO task. The three usable OpenAI models scored within about three points of each other, too close to call a winner. We did not test implementation or ranking changes.
Frequently Asked Questions
Can ChatGPT or Claude perform a technical SEO audit?
They can produce a useful first pass when they get structured crawl and Search Console evidence. A person still needs to check the recommendations, and any business claims, before changing the site.
Are expensive AI models worse at SEO audits?
Not at finding problems. In this test the two most expensive models found all ten planted problems every time. They were worse at sticking to the facts and the brief, so none of their reports could go out without an edit.
What does “usable without fixes” mean?
The report caught all four critical problems, caught at least eight of ten overall, stayed within the length and format we asked for and made no claim the evidence did not support.
Did the cheapest model win?
No model won outright. Luna was the lowest-cost model that produced a usable report every time on this assignment.