The honest answer: for most Singapore SMEs right now, llms.txt is a low-priority nicety — not a growth lever. It takes under an hour to implement, carries no downside, and may eventually matter. But Ahrefs found that 97% of domains with a valid llms.txt file received zero crawler requests for it. If your marketing hour is finite — and it always is — read this before you spend it on a text file.
What is llms.txt? llms.txt is a plain-text file, proposed by fast.ai in 2024, that you place at yourdomain.com/llms.txt. It summarises your site’s content, structure, and purpose in a format designed to be easy for large language models to parse. Think of it as a condensed briefing document for AI systems — not a technical crawler instruction like robots.txt, but a curated index you write yourself, in plain language, pointing to your most important pages.
What llms.txt Actually Does (and Does Not Do)
The proposal is elegant in theory. You write a Markdown-formatted file listing your key pages, their descriptions, and any context an LLM might need. An AI crawler visits the file, reads it, and — in theory — builds a better model of your site than it could from crawling thousands of pages cold.
The problem is the gap between theory and adoption. No major AI lab has formally committed to reading llms.txt as a ranking or citation signal. There is no published spec from OpenAI, Anthropic, or Google confirming they parse it. The standard is community-proposed, not industry-ratified.
What the bots do read is your raw HTML — every time, no exception. GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot all lack the ability to execute JavaScript. They see only what a browser renders before any script fires. That means a beautifully structured llms.txt file sitting on a React site with server-side rendering gaps may be the least of your problems.
The structural work that actually improves AI citation probability — clean HTML, schema markup, answer-first prose, entity-consistent naming — remains more valuable than any single text file.
The Ahrefs Finding Singapore Owners Should Know
In a crawl study published in 2025, Ahrefs found that 97% of domains with a valid llms.txt file received zero crawler requests for that file. Not “low traffic.” Zero.
That is a useful data point, not a reason to panic. It tells you two things. First, AI crawlers are not yet routing significant traffic through llms.txt as a discovery mechanism. Second, the sites implementing it are, for now, doing so on faith rather than evidence.
The counterargument — and it is worth hearing — is that early adoption of technical standards often looks like this. robots.txt was ignored by many crawlers in its early years. Structured data saw the same slow burn before Google made it a featured-snippet signal. The 97% figure describes the present, not the ceiling.
Kaizenaire’s view: implement it if you have a developer or a technical marketing resource who can do it in an afternoon. Do not implement it instead of the fundamentals. And do not let an agency charge you a meaningful retainer line item for it.
The Bing Prerequisite Most Guides Skip
Here is a fact that almost every llms.txt explainer omits: ChatGPT Search is built on Bing’s index. If your site is not indexed in Bing, ChatGPT Search cannot surface it — full stop. No llms.txt file changes that.
Run a quick check: open Bing and search site:yourdomain.com. If you see no results, or a fraction of the pages you expect, your first priority is Bing indexation — submitting your sitemap to Bing Webmaster Tools, verifying ownership, and ensuring your pages are crawlable. That takes an hour and has a direct, demonstrable effect on your probability of appearing in ChatGPT Search results.
For Singapore SMEs, this matters more than it might for, say, a US business that has always optimised for Google. Many local sites were built Google-first. Bing was an afterthought. ChatGPT Search has changed that calculation, and catching up on Bing indexation is a concrete, achievable action — unlike waiting for AI labs to formally adopt llms.txt.
The same logic applies to Perplexity, which builds its index partly from its own crawls and partly from Bing. Fix the foundation before adding the furniture.
Myth vs. Fact: Common llms.txt Claims Tested
| Claim | Reality | Verdict |
|---|---|---|
| “llms.txt gets you cited by ChatGPT” | No AI lab has confirmed it reads the file as a citation signal. ChatGPT Search depends on Bing indexation first. | Unproven |
| “It’s the new robots.txt” | robots.txt is a W3C/RFC-backed standard with universal crawler support. llms.txt is a community proposal with no formal backing yet. | Misleading |
| “97% failure rate means it’s useless” | 97% got zero requests — today. Standards adoption takes time. The file costs little; the option value is real. | Overstated |
| “Any developer can do it in an hour” | Mostly true for a basic implementation. A well-structured llms.txt with accurate page summaries takes longer to do properly. | Broadly correct |
| “AI crawlers read JavaScript-rendered content anyway” | They do not. GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot read raw HTML only — no JS execution. | False |
What a Minimal llms.txt Looks Like
If you decide to implement it, here is what a basic file contains. It is plain Markdown. No special server configuration required — just a text file placed at the root of your domain.
- A one-paragraph summary of what your business does, written in plain language. Treat it like the first paragraph of a good About page.
- A list of your most important pages — each with a URL and a one-sentence description. Prioritise pages that answer questions, not pages that just sell.
- Optional: an
llms-full.txtlink — a more detailed version for crawlers that want depth. Some implementations separate the brief index from the full content dump. - Contact or attribution details — your business name, location (Singapore, if relevant), and the date the file was last updated. Recency signals matter to AI systems.
Keep it under 2,000 words for the summary file. The goal is a coherent briefing, not a data dump. If a new employee could read it and understand what your business does and where to find key information, it is working correctly.
The Spike: What Nobody Wants to Tell You
AI citation drives a fraction of total website traffic today. For most Singapore SMEs, the percentage of inbound leads arriving via “someone asked ChatGPT and it mentioned your business” is not yet material. If you need pipeline this quarter, llms.txt — and frankly most GEO work — is not your lever. That is not a reason to ignore it permanently. It is a reason to sequence your effort correctly: indexation, content quality, and structured data first; speculative file formats after.
Who Should Implement llms.txt (and Who Should Skip It)
Implement it if: you have a developer on retainer or in-house, your site is already well-indexed on Google and Bing, you publish regular content that answers questions, and you are thinking twelve months ahead on AI search.
Skip it for now if: your site has JavaScript rendering issues that block crawlers, you are not yet indexed properly in Bing, your core pages lack clear H2 structure and schema markup, or your marketing budget is genuinely constrained. Fix the crawlable foundation first.
There is no shame in the second category. Most Singapore SME sites sit there. A site that loads cleanly, renders in raw HTML, answers questions directly in its copy, and carries proper schema markup will consistently outperform a poorly structured site with a pristine llms.txt file. The bots are not impressed by gesture.
If you want to know exactly where your site stands on AI visibility — crawlability, Bing indexation, schema gaps, content structure — a structured audit will show you the priority list. Kaizenaire’s free AI-Visibility Check does exactly that, and it is one place to start. Learn more about the AEO, GEO and SEO services behind it, or go straight to the audit below.
Frequently Asked Questions
Does llms.txt improve my Google ranking?
No. Google has not indicated it uses llms.txt as a ranking signal. Google’s AI Overviews draw from the standard web index, not from a separate text file you provide. If your goal is Google visibility — organic or AI Overviews — focus on structured data, E-E-A-T signals, and clear on-page content. llms.txt is aimed at LLM-native systems, not the Google crawler.
How do I know if AI crawlers are visiting my site at all?
Check your server access logs for known bot user-agent strings: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot. Most shared hosting dashboards do not surface this easily — you may need raw log access or a tool like Cloudflare’s bot analytics. If none of these bots have visited in the past 90 days, that is a signal worth investigating before optimising for their behaviour.
Is llms.txt the same as robots.txt?
No. robots.txt is a technical instruction file with decades of crawler support — it tells bots which pages to crawl or avoid, and they are expected to obey it. llms.txt is a content-summary file with no enforcement mechanism. You cannot use it to block AI crawlers, and there is no obligation for any crawler to read it. Think of it as a welcome note, not a bouncer.
Should I block AI crawlers with robots.txt instead?
That depends on your goal. If you do not want your content used to train AI models, you can disallow GPTBot and others in robots.txt — OpenAI has stated it will respect these directives for training data. However, blocking crawlers also reduces your probability of appearing in AI-generated answers. For most Singapore SMEs trying to grow visibility, blocking is counterproductive. If content protection is a concern, take legal advice rather than relying solely on robots.txt.
My site is built on React or Next.js. Does that affect this?
Yes, significantly. All major AI crawlers — GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot — read raw HTML only. They cannot execute JavaScript. If your React or Next.js site relies on client-side rendering for key content, those bots may see a near-empty page. Server-side rendering (SSR) or static generation is essential for AI crawlability. This is a more urgent fix than llms.txt for most JS-heavy Singapore sites.
What does a proper llms.txt cost to implement?
A basic file — a summary paragraph plus a list of key pages — is a one-person, one-afternoon job for a developer or technically competent marketer. It costs whatever an hour or two of that person’s time costs. No special tools or subscriptions required. A more thorough implementation with well-written page summaries and regular updates might take four to six hours annually to maintain. It is not a meaningful budget line; it is a maintenance task.
Will kaizenaire.ai implement llms.txt as part of its service?
Yes, where it is appropriate. As part of the AEO/GEO/SEO service, Kaizenaire reviews the full technical and content layer — including crawlability, Bing indexation, schema markup, and yes, llms.txt where relevant. The starting point is always the free AI-Visibility Check, which identifies the highest-priority gaps for your specific site before any work begins.
Run your free AI-Visibility Check. It takes about three minutes to submit. You’ll get a structured read on where your site stands on AI crawlability, Bing indexation, schema coverage, and content structure — the things that actually move the needle on AI citation probability. Start the AI-Visibility Check here.