PrimalMogul AI

Generative Engine Optimization: A Practitioner’s Study of AI Search Visibility

INSTITUTE EDITION: PAIR-CRT-001
SCHOOL: School of Creation · LEVEL: Practitioner · EDITION TYPE: Core
READING TIME: 22 minutes · EVIDENCE LABEL: Evidence-Based
LAST REVIEWED: September 2026
COMPANION POST: Getting Recommended by AI: How Businesses Earn Visibility in ChatGPT, Claude, and AI Search

1. Learning Objectives

By the end of this study, you will be able to:

1. Explain the two ways AI assistants acquire information: model training and live retrieval.

2. Classify AI crawlers by function and configure robots.txt to allow search access while controlling training access.

3. Apply the five layers of the PrimalMogul Recommendation Stack to a business website.

4. Distinguish between non-commodity and commodity content.

5. Design a monthly prompt panel to measure AI share of voice.

6. Evaluate common GEO claims against published platform guidance.

2. Prerequisites and Context

No formal prerequisites. Access to your website’s robots.txt file and Google Search Console is recommended for the exercises.

Customers increasingly ask AI assistants for recommendations instead of scanning search results. Search platforms have responded with new measurement tools.

Google launched generative AI performance reports in Search Console in June 2026, and Bing introduced an AI Performance report earlier in 2026, focused on citation frequency. The discipline of earning visibility in these systems is called generative engine optimization.

3. Key Terms

4. Core Instruction

4.1 How AI Answers Are Assembled

AI systems draw on two sources:

Google describes its generative AI features as using retrieval-augmented generation and query fan-out to highlight content from its search index. A single customer question may trigger multiple searches. A page that answers several related questions clearly is more likely to be retrieved across those searches.

Rule: Live retrieval is where most business recommendations originate. Prioritize retrievability over training presence.

4.2 Layer One: Access

Crawler Classification

Each crawler requires its own robots.txt directive. Blocking ClaudeBot does not block Claude-SearchBot or Claude-User.

Anthropic states that all three of its crawlers honor robots.txt. OpenAI and Perplexity note that robots.txt rules may not apply to user-initiated fetchers.

Policy: Allow AI search, block AI training

# Allow AI search and user-requested access
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: PerplexityBot
Allow: /

# Block AI model training
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

# Traditional search
User-agent: Googlebot
Allow: /

User-agent: Bingbot
Allow: /

Warning: Blocking training crawlers is a legitimate business decision, but it may reduce how well future models know your business. Allowing them increases long-term familiarity. Choose deliberately.

Google’s AI opt-out. Search Console now includes a control that removes a site’s content from Google’s AI features. Sites that opt out receive no traffic or impressions from those features.

4.3 Layer Two: Clarity

AI systems must confidently identify a business before recommending it.

1. Consistent core details: identical name, address, phone, hours, and service area across every listing.

2. A definitive About page: plain statements of what the business does, whom it serves, where, and since when.

3. Google Business Profile and Bing Places: complete, verified, and current.

4. Organization structured data: helpful for entity clarity, though Google states no special schema is required for its AI features.

4.4 Layer Three: Credibility

AI systems weigh third-party evidence.

Rule: Earn mentions; never manufacture them. Google states that seeking inauthentic mentions to influence its AI features is unlikely to help, because those features rely on the same spam safeguards as core search.

4.5 Layer Four: Content

Non-commodity content is the central content principle. Google’s 2026 guidance emphasizes providing a unique point of view and helpful, people-first content that goes beyond commodity information.

Answer-first structure:

1. State the direct answer in the first one or two sentences.
2. Support it with specific evidence: numbers, sources, examples.
3. Address related follow-up questions on the same page.
4. Display a visible last-updated date.

Research supports evidence density. In a 2024 academic study, GEO: Generative Engine Optimization (Aggarwal et al., presented at KDD 2024), adding citations, quotations, and statistics to content improved source visibility in generative engine responses by up to roughly 40% in the researchers’ test environment. Results in live systems vary.

4.6 Layer Five: Measurement

The Prompt Panel Method:

1. Write 10 questions a real customer would ask, mixing broad and specific.

2. Ask each question in four assistants monthly, in a fresh session.

3. Record: mentioned (yes/no), position in the answer, accuracy of details, and competitors named.

4. Calculate share of voice: mentions ÷ total checks.

4.7 Evaluating GEO Claims

5. Worked Example

Inland Comfort Services is a fictional business used for illustration. Figures are illustrative.

Situation. An HVAC company in Southern California wants to appear when customers ask AI assistants for local recommendations.

Step 1: Access audit.
The robots.txt file, installed by a previous web developer, blocks all bots except Googlebot. OAI-SearchBot, Claude-SearchBot, and PerplexityBot cannot read the site. Fix: apply the Section 4.2 policy.

Step 2: Clarity audit.
The business appears under three name variations across 14 listings, with two outdated phone numbers. Fix: standardize all listings and publish a definitive About page.

Step 3: Baseline prompt panel.

Step 4: Competitor analysis.
The leading competitor publishes a financing page with real payment examples, has 600 detailed reviews, and was featured in a regional news story about heat-wave preparedness.

Step 5: 90-day plan.
Publish a furnace and AC replacement cost guide using the company’s own installation data; publish a financing page with real payment examples; request detailed reviews from 60 recent customers; pitch a seasonal maintenance story to local media.

Step 6: Day-90 prompt panel.

Lesson: the largest single gain came from Step 1. Content cannot be cited if crawlers cannot read it.

6. Case Study

PrimalMogul’s Search Performance, June–September (Founder Protocol, written in third person)

The founder of PrimalMogul reviewed three months of Google Search Console data for primalmogul.com. Original mogul profiles and rankings dominated performance.

The ranking of the richest Black women entrepreneurs earned 755 clicks from 22,331 impressions, a click-through rate of 3.4%. Original profiles of lesser-known executives such as Samiel “Blacc Sam” Asghedom earned tens of thousands of impressions.

By contrast, generic AI how-to posts on widely covered topics recorded few or no clicks. One post on AI agents for small business recorded 44 impressions and no clicks in the period.

Principles demonstrated:

• Content with original analysis on underserved subjects earned visibility; commodity topics did not.

• Search performance data reveals which content systems treat as unique.

• Measurement must precede strategy.

7. Common Errors

8. Implementation Protocol

Phase 1: Foundation (Weeks 1–2)

1. Audit robots.txt by crawler function and apply a deliberate policy. (1 hour)
2. Verify Google Search Console and Bing Webmaster Tools. (1 hour)
3. Run the baseline prompt panel. (2 hours)

Phase 2: Clarity and Credibility (Weeks 3–6)

4. Standardize business details across all listings. (4 hours)
5. Rewrite the About page as a definitive entity statement. (2 hours)
6. Launch a review request process for recent customers. (2 hours setup)

Phase 3: Content (Weeks 7–12)

7. Identify the five questions customers ask most before buying.
8. Publish one non-commodity page per question, answer-first, with original evidence.
9. Add visible last-updated dates and review each page quarterly.

Phase 4: Measurement (Monthly)

10. Rerun the prompt panel and record share of voice.
11. Review the Search Console generative AI report.
12. Adjust content priorities toward questions where competitors are mentioned and you are not.

9. Practice Exercise

Deliverable: A completed AI visibility baseline report.

Estimated time: 90 minutes

1. Open yourdomain.com/robots.txt and list every AI crawler rule.

2. Classify each rule as training, search, or user-initiated.

3. Write 10 customer questions.

4. Test each in ChatGPT, Claude, Perplexity, and Google AI Mode.

5. Record mentions, accuracy, and competitors named.

6. Calculate share of voice and identify the top competitor.

Self-check:

1. Every search crawler’s status is confirmed.

2. Questions reflect real customer language.

3. Share of voice is calculated as mentions ÷ total checks.

4. At least one specific content opportunity is identified.

10. Knowledge Check

Q1. Which crawler indexes content for Claude’s search results?

A. ClaudeBot
B. Claude-SearchBot
C. GPTBot
D. Google-Extended

Answer: B. ClaudeBot handles training; Claude-SearchBot handles search indexing.

Q2. A site blocks ClaudeBot only. What happens to Claude-SearchBot?

A. It is also blocked
B. It is unaffected; each crawler needs its own directive
C. It is blocked for 30 days
D. It switches to training

Answer: B.

Q3. According to Google’s 2026 guidance, optimizing for its generative AI features is primarily:

A. A separate discipline requiring llms.txt
B. Still SEO
C. Achieved through special schema
D. Achieved through content chunking

Answer: B.

Q4. A business is mentioned in 6 of 40 prompt panel checks. What is its share of voice?

A. 6%
B. 12%
C. 15%
D. 40%

Answer: C. 6 ÷ 40 = 15%.

Q5. What does the Search Console generative AI report currently not provide?

A. Impressions
B. Page data
C. Click data
D. Country data

Answer: C.

Q6. Which page offers the highest citation value?

A. “Top 10 Tips for Hiring a Plumber”
B. A rewritten industry article
C. Original local pricing data from the company’s own jobs
D. A keyword-rich services list

Answer: C. Original, non-commodity information gives AI something unique to cite.

Q7. An agency guarantees first-position ChatGPT recommendations. The correct response:

A. Sign immediately
B. Treat the guarantee as a warning sign; no party controls AI outputs
C. Request a larger package
D. Block all AI crawlers

Answer: B.

11. Summary

AI assistants answer from training data and live retrieval, and most business recommendations come from retrieval.

The PrimalMogul Recommendation Stack organizes visibility into five layers: access for crawlers, clarity of business identity, third-party credibility, non-commodity content, and ongoing measurement.

Google’s 2026 guidance confirms that optimizing for its AI features remains SEO and dismisses llms.txt, chunking, and special schema as requirements. Monthly prompt panels and Search Console’s generative AI reports provide the evidence needed to direct effort.

12. References and Further Study:

• Google Search Central. Optimizing Your Website for Generative AI Features on Google Search. May 2026, updated July 2026.
• Google Search Central Blog. Introducing Search Generative AI Performance Reports in Search Console. June 3, 2026.
• Anthropic. Crawler documentation: ClaudeBot, Claude-User, Claude-SearchBot.
• OpenAI. Overview of OpenAI crawlers.
• Microsoft Bing. Introducing AI Performance in Bing Webmaster Tools. February 2026.
• Aggarwal, P., et al. GEO: Generative Engine Optimization. Proceedings of KDD 2024.
• PrimalMogul. Internal Google Search Console data, June–September 2026.

Further Study in the Vault:

Mogul Content Traffic Protocol principles

· Choosing a Business Structure

· Starting a Business in the AI Era

This study is educational. AI platforms change their systems frequently; verify current crawler documentation with each provider before changing robots.txt.