Lighthouse Journey #4 — Writing for AI Search Engines

2025/11/27136 回视听by Huynh

“When the reader is not a human — but understands like one.”

This final entry in our Lighthouse Journey looks at a different kind of “reader.”
Not Google.
Not a browser.
Not even Lighthouse.

But AI search engines — the systems that don’t just index the web,
but interpret it.

Gemini, Bing Copilot, Claude.ai, Perplexity, ChatGPT Search —
these tools don’t simply crawl links.
They read meaning.
They look for structure.
They try to understand intention.

And building WakaranEng for them
turned out to be a surprisingly human process.

🧩 The Problem: AI Could Find Us… but Not Fully Understand Us

When we first tested WakaranEng on multiple AI platforms,
the results were a mix of encouraging and funny.

Here are the examples you can insert:

Gemini response:

Bing Copilot response:

Claude web response:

All three systems could:

See the site
Summarize the content
Identify the theme
Recognize WakaranEng as “something related to English learning”

…but they didn’t always interpret Wakaran Eng as a project.

Most AI engines initially thought:

“Wakaran (わからん) is a shortened Japanese slang word…”

Which is technically correct.
But not what we meant.

Only after prompting with:

“Wakaran Eng project”

…did they consistently switch to describing our website.

This told us two things:

AI could crawl our site.
AI could read our content.
But AI still lacked the identity signal it needed.

That was the real challenge.

📄 Step 1 — Giving AI a Proper Invitation (robots.txt)

We modified our robots.txt to explicitly allow AI crawlers:

// Abbreviated example
{
  userAgent: [
    'GPTBot',
    'ClaudeBot',
    'Google-Extended',
    'PerplexityBot',
    'Meta-ExternalAgent',
    'Applebot-Extended',
    'Amazonbot',
    // ...
  ],
  allow: '/',
  disallow: ['/private/', '/admin/', '/api/'],
}

This made two things happen:

AI systems treated the site as open, indexable, learnable
They started retrieving more structured, consistent descriptions

We didn’t give them “SEO tricks.”
We simply gave them permission and clarity.

🧠 Step 2 — AI Search Needs Semantic Anchors

Through testing, we learned:

AI requires strong contextual anchors to “lock onto” the identity of a project.

Examples of strong anchors:

Clear site description
Consistent project naming
Structured metadata
Correct locale formatting
Rich alt text
Accessible headings
Canonical routing
OpenGraph & JSON-LD
Sitemap clarity

These serve as the “grounding points” for LLMs.

And because WakaranEng had:

accessibility above 90
valid language-region metadata
multilingual stable routing
strong content structure
clean alt text system

AI search performance improved naturally.

This is the new era of SEO — not based on keywords,
but on semantic integrity.

🧩 Step 3 — Why This Matters

AI-based search engines don’t crawl like Google circa 2010.
They do something closer to reading comprehension.

They care about:

Meaning
Internal consistency
Structure
Accessibility
Context
Relationships

Not just raw text.

And the fact that AI could describe WakaranEng
as a real project, website, and learning platform —
without us submitting anything —
confirmed that the foundation was solid.

This is the quiet success of this entire optimization journey.

🟢 Final Lighthouse Result (After All Fixes)

Here is the final Lighthouse result of WakaranEng
after implementing everything from all 4 posts:

From performance to semantics to readability,
the site now stands on a foundation that is:

Fast
Accessible
Multilingual
AI-friendly
SEO-consistent

All achieved through dozens of small, honest improvements.

🎉 Closing the Journey

We started this series looking at scores.
But we’re ending it looking at clarity.

Because that’s what modern web development really is:

Not tricks
Not hacks
Not chasing numbers
But communicating clearly to humans and machines

Performance taught us timing.
Accessibility taught us empathy.
Locale taught us precision.
AI search taught us meaning.

And together, these fixes shaped a website
that behaves exactly the way WakaranEng should:

Clear. Kind. Understandable.

“The web understands you only when you understand yourself first.”

And now — it understands WakaranEng.

WAKARAN ENGLISH