Skip to content

SEO + AI discoverability: real descriptions, trailing-slash canonicals, spec llms.txt, JSON-LD - #171

Merged
anttiviljami merged 5 commits into
mainfrom
seo-ai-discoverability
Oct 20, 2026
Merged

anttiviljami merged 5 commits into
mainfrom
seo-ai-discoverability

Conversation

@anttiviljami

Copy link
Copy Markdown
Member

Summary

This PR makes docs.epilot.io easier to find for search engines and AI agents, and fixes a live bug in the .md pages and llms-full.txt. Each change is its own commit.

  1. Descriptions (81036c3, e2591dd)
    • Adds a short description that leads with what the page covers to 60 hand-written pages. Each one is drawn from the page's own intro.
    • The SDK and CLI generators now write a description for each generated page; the committed SDK and CLI pages are updated to match.
    • Weak descriptions drop from 176 to 40. Before, many were link labels such as [API Docs], or  .
  2. Trailing slashes (36f6451)
    • Sets trailingSlash: true in the Docusaurus config.
    • Amplify already sends /docs/x to /docs/x/ with a 301, so the canonical URLs, og:url and all 877 sitemap URLs pointed at redirects. No public URL changes.
    • Fixes 87 relative links in the SDK and CLI overview pages that would otherwise break, and updates the generator scripts to match.
  3. llms.txt and Markdown output (f01dcdb)
    • Bug fix: the .md pages and llms-full.txt had every {…} and import line removed, including inside code blocks. Live example: /docs/sdk/overview.md is missing import { epilot } from '@epilot/sdk'.
    • llms.txt now follows the llmstxt.org format:
      • a > summary line;
      • one H2 per sidebar section;
      • descriptions on 332 of 363 links, up from 79 of 340 live;
      • "Documentation sets" and "Optional" sections.
    • llms-full.txt now has a Source: URL for each page.
    • Each section also gets its own docs/<section>/llms-full.txt.
  4. Metadata and structured data (e8f233f)
    • The home page gets a title and description, plus WebSite and Organization JSON-LD.
    • Every doc page gets TechArticle JSON-LD and a text/markdown alternate link to its Markdown version.
    • API reference pages:
      • each gets its own description;
      • each links to its OpenAPI spec with a rel=alternate link;
      • the static HTML now contains the API name, description and spec URL (before, it was empty until Redoc rendered).
    • Also:
      • the default og:image is the logo;
      • /api is titled "REST API Reference";
      • changelog entries use their excerpt as the description;
      • "Energy XRM" is now written "energy XRM".

Test plan

  • npm run build passes.
  • Lint and tsc report the same problems as main, none in changed files.
  • Checked in the build output:
    • All 293 doc pages have a trailing-slash canonical, a .md alternate and TechArticle JSON-LD that parses.
    • All sitemap URLs end in /.
    • Every local link in llms.txt points to a file that exists in the build.
    • Code blocks in the generated .md files match the source.
  • Check an API reference page (its loading state) and a doc page in a browser on the Amplify preview. Both were only checked in the build output.

Follow-ups (not in this PR)

  • Amplify console: unknown URLs return 404 but show the home page, with a canonical pointing to /. The rewrite should point at /404.html.
  • Social image: a proper 1200×630 og:image should be designed; the logo is a stopgap.
  • lastmod and last-updated dates: these need showLastUpdateTime and a full git clone in the Amplify build.
  • llms-full.txt size: it grows from 3.1 MB to 4.8 MB because code blocks are no longer stripped. The per-section files run from 5 KB to 914 KB.
  • Changelog URLs: they repeat the path (/api/changelog/api/changelog/…). Left as is, because changing URLs is out of scope here.

Generated by Claude Code

anttiviljami and others added 5 commits October 20, 2026 05:50
… link label

60 pages had no frontmatter description, so Docusaurus used the first
paragraph as meta description, which for most of them is a link row such
as "[API Docs]" or "&nbsp;". Each description is taken from the page's
own intro. Generated reference pages (SDK clients, CLI commands, library
typedoc) are left untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Et6RzideGVXaP9rWvwx1QZ
…real descriptions

The SDK client and CLI command pages got "- Base URL://..." as their meta
description. The update-sdk/update-cli scripts now write a description
derived from the page title, and the committed pages carry the same line
so the next regeneration is a no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Et6RzideGVXaP9rWvwx1QZ
Amplify 301-redirects /docs/foo to /docs/foo/, so every canonical, og:url
and sitemap <loc> pointed at a redirect. With trailingSlash: true the
canonical, sitemap and internal links use the final URL.

The SDK and CLI overview tables used route-relative links (clients/x),
which only resolve correctly without the trailing slash. They are now
file links (clients/x.md) that Docusaurus resolves to the page, and the
update scripts generate them that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Et6RzideGVXaP9rWvwx1QZ
- llms.txt: blockquote summary under the H1, one H2 section per sidebar
  section with its sidebar label (instead of H3 path groups), descriptions
  for pages without frontmatter description taken from the rendered meta
  description, a "Documentation sets" section and an "Optional" section
  with the API changelog and the apis.json discovery document.
- llms-full.txt: every page gets a canonical "Source:" URL; only pages
  that were built are included. Each docs section also gets its own
  docs/<section>/llms-full.txt, since the full file is several MB.
- Markdown cleanup now skips fenced code blocks and inline code. It used
  to strip "import" lines and every {...} span, so TypeScript samples lost
  their imports and JSON samples lost their objects in the .md pages and
  llms-full.txt. Simple HTML (spacer paragraphs, headings, img) becomes
  markdown.
- The frontmatter description now follows the page H1 instead of
  preceding it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Et6RzideGVXaP9rWvwx1QZ
…alternates

- Home: real title and meta description (it had none) plus WebSite and
  Organization JSON-LD.
- Docs pages: <link rel="alternate" type="text/markdown"> to the .md
  version the llms-txt plugin publishes, and TechArticle JSON-LD built
  from the page title and description (DocItem wrapper).
- API reference pages: per-API description instead of "Open API
  Reference Docs for the API", an alternate link to the OpenAPI spec, and
  the API name, description and spec link rendered server-side while the
  client-side reference loads (the static HTML had no content at all).
- /api: "REST API Reference" title instead of "Getting Started".
- Changelog entries: use the entry excerpt as description, not the title.
- Default og:image/twitter:image (the square logo, with a summary card).
- "energy XRM" casing in the tagline and copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Et6RzideGVXaP9rWvwx1QZ
@anttiviljami
anttiviljami merged commit 533474b into main Oct 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant