# SEO tools teardown: coding notes, codebook v2.0 (collected September 26, 2026) The published data is coded_v2/: reader codes under gold2/READER_PROTOCOL_v2.md. mentions.csv mention_type is the broad level (recommended includes picked); the 0/1 "picked" column is the answer's own verdict. ## Coding method and agreement - Every one of the 100 answers was coded by two independent AI readers (Claude agents, not people), each working blind from the written protocol and without seeing any machine coding. Where they disagreed, a third AI reader decided. - Reader vs reader in this teardown: 97.6% on the six codes, 99.3% at the broad (recommended) level (gold2/CODED_V2_SUMMARY.md). - Codes decided by the third reader: 10 (listed below). - A rules-based first pass (v1.6 rules) was checked against the final reader codes: 96.1% agreement at the broad level. The reader codes are what is published. The older v1.5 coding in coded/ is retired and not used. - Totals: 413 mentions; 366 recommended at the broad level, of which 309 picked; 9 listed, 19 passing, 19 anchor. 94 answers pick at least one dictionary brand. ## Collection All 100 ChatGPT tasks and 100 Google tasks were posted and fetched through the DataForSEO standard queue between about 8:15 and 8:30 PM US Mountain time on September 26, 2026 (the raw files carry UTC timestamps, early September 27). No task failed and nothing was re-run. Three answers did not answer the question. q039 ("surfer seo vs clearscope") and q078 ("ahrefs alternative for agencies") are one-sentence preambles that promise a comparison and stop, with no sources; the readers coded the brands in them passing (q039) or anchor only (q078). q080 ("profound alternative for smaller brands") was read as a vocabulary question and suggests other words for "smaller brands"; no tool is named. All three are kept as returned and count among the six answers with no pick. ## Judgment calls that matter in this category - Picked vs recommended in "best X" lists. Many answers give every tool a "best for" label and then end with a short verdict ("My recommendation", "If I had to pick one", "My picks by situation", "I'd start with"). Brands in the verdict are picked; brands that only carry a fit label in the list or table are recommended. This is the main source of the recommended-minus-picked gap: Moz (19 recommended, 8 picked) is most often the "simpler option for smaller teams" or "best for beginners" row that the verdict does not choose (q001, q002, q008, q010, q020, q083). - Conditional on existing use. "Worth considering if you already use Semrush" (q067), a table fit "agencies already using Semrush" (q089) and "Semrush Local can make sense if you already use Semrush" (q091) were coded recommended, not picked: the answer assigns a case but does not choose it for the buyer. - Suites as the companion pick. In specialist questions the verdict often pairs a specialist with a suite for research: q024 "I'd pair your app with Ahrefs or Semrush" (the app, Tiny SEO, is outside the dictionary), q021 "one tool for everything: Semrush", q018 "keyword and competitor research: Semrush or Ahrefs". Both readers coded these picked. - Anchor stays anchor. In q074 ("free alternative to screaming frog") the verdict's first line is "Use Screaming Frog free"; Screaming Frog is the brand named in the question and is coded anchor under the protocol. The other picks there (SiteOne Crawler, BeamUsUp, LibreCrawl, Crawly) are outside the dictionary, so the answer counts as having no pick. - Descriptive answers without a verdict. q009 ("most popular SEO software") gives each of eight tools a "particularly strong for" row and then asks the buyer's use case: all eight recommended, none picked. q064 ("se ranking alternatives") describes eight alternatives with no fit: all listed. - Source labels and integrations are passing. q038: Keyword.com appears only as a cited source label, coded passing. Google Search Console named as a data source or integration (q041, q050, q057, q065) is passing. - Head-to-heads. 29 of 30 picked every brand named ("Choose A if..., choose B if..."). q041 ("screaming frog vs sitebulb") picks both, with Sitebulb for agencies producing recurring client audits. ## Codes decided by the third reader (gold2/out/J03.jsonl) - q012 Frase: recommended (best-fit table row for lean teams; not in the verdict). Readers split recommended / picked. - q018 Whitespark: recommended (fit label, citation research; not in "My recommendation"). Split recommended / picked. - q022 Semrush: recommended (best-for table row; not in the narrowed set). Split recommended / picked. - q024 Yoast SEO: recommended (best-for table row; the answer's own pick is Tiny SEO). Split recommended / picked. - q062 Moz: recommended (best for beginners; the replacement list is not a verdict). Split picked / recommended. - q062 Ubersuggest: picked (neilpatel.com named as the best budget option). One reader coded it from the web address; the other did not code it. - q069 Google Search Console: recommended (table row "best for small sites / DIY", written as GSC). One reader only. - q073 Bing Webmaster Tools and Moz: recommended (feature-map entries for backlink analysis; absent from the answer's strongest-setup verdict). Split picked / recommended. - q075 Moz: recommended (beginner-friendly fit; not in the quick recommendation). Split recommended / listed. ## Brand mentions added by the readers Ten reader codes (counting both readers) cover brand mentions no dictionary alias matched, in four answers: q062 Screaming Frog (by its domain) and Ubersuggest (neilpatel.com), both picked; q069 Google Search Console (written "GSC"), recommended; q075 Google Search Console (search.google.com), recommended; q076 Google Search Console (search.google.com) and Google Keyword Planner (ads.google.com), recommended as an optional free stack. ## Dictionary No changes after collection; brands.csv is the dictionary as fixed before collection (see INPUT_NOTES.md). - Case-sensitive aliases (capitalized only): Surfer, Conductor, Majestic, Profound. Lowercase "profound" in q080 is correctly not matched. - google.com is shared by Google Search Console, Keyword Planner, Analytics and Trends; its 9 citation events cannot be attributed to one product. Ubersuggest is matched on neilpatel.com. - "GSC" is not an alias. Where it names a tool offered to the buyer (q069), the readers coded it; the other uses (q006, q037, q041, q060) are integration or data mentions. - Not in the dictionary and picked in a few answers: Shopify SEO apps (TinyIMG / Tiny SEO, Smart SEO, SearchPie; the two Shopify questions), free crawlers (SiteOne Crawler, BeamUsUp, LibreCrawl, Crawly in q074), Siftly (q028, q097), LowFruits (q070, q092), SERProbot (q015, q016). None reaches 5 answers, so none was added. coded_v2/out_of_dictionary.csv lists the out-of-dictionary entities found in 5 answers. - q084 repeats its closing sections after the sign-off (as returned). Codes are per brand per answer, so the repetition does not change the coding.