Search engines and social previews often cannot run the interactive JavaScript in a typical Caffeine app. Caffeine keeps a ready-made snapshot of each public page for those visitors, so your live app can appear in search results without any extra setup or extra credits. Your live app also publishes a few small files that help AI agents understand it.
What Caffeine does for search engines
Every live Caffeine app includes this automatically. When a search engine or similar crawler requests a page, Caffeine can serve a static snapshot of that page instead of the empty loading shell a crawler might otherwise see.
People using a normal browser still get the live, interactive app. The snapshot is only for crawlers.
This applies to the apps you publish — including on a custom domain. You do not turn it on, and it does not use build or Go Live credits.
How snapshots stay up to date
Caffeine saves a snapshot the first time a crawler requests that page. The snapshot stays in use for 15 days.
After 15 days, the next crawler request still gets the existing snapshot immediately, and Caffeine refreshes it in the background. You do not need to take any action.
Only pages that crawlers actually visit are snapshotted. A page that never receives a crawler request will not have a snapshot yet.
What this does not cover
- Password-protected live URLs. If the live site is locked with password protection, crawlers hit the password gate and cannot index the app. Leave live protection off if you want the site in search results.
- Pages that require sign-in. Content behind login is not something search engines should index.
- Draft URLs. Drafts are for you and collaborators, not for public search.
- The first crawler visit to a new URL. A snapshot may not be ready for that first request. Later crawler visits use the saved snapshot.
What Caffeine publishes for AI agents
Every live Caffeine app also serves a few machine-readable files, so an AI agent that is given only your app's URL can work out what it is and how to use it:
-
/llms.txt— a short plain-text description of your app for AI tools, following the llms.txt format. -
/.well-known/ic-architecture— a list of the parts that make up your app and what each one does. -
/.well-known/ii-derivation-originand/.well-known/ii-alternative-origins— tell Internet Identity which address your app's sign-in belongs to, so signing in from any of your app's addresses, including by an AI agent acting for a user, reaches the same account.
Caffeine writes these files each time you go live. They are not added to drafts, and they are not served while the live site has password protection turned on. You do not set them up, and they do not use credits.
Caffeine manages these paths. If your app contains its own /llms.txt, the platform version replaces it, and files you upload under /.well-known/ are left out of the live app.
Frequently asked questions
Do I need to set this up?
No. It is included with your published app.
Does this cost credits?
No. Snapshots do not use build credits, Go Live credits, or any other credit charge.
How often is each page refreshed?
A saved snapshot is treated as current for 15 days. After that, it is refreshed automatically the next time a crawler requests that URL. Pages that crawlers never revisit are not refreshed on a schedule.
Will my latest Go Live show up in search right away?
Not necessarily. Going live does not immediately replace an existing crawler snapshot. Even if a crawler revisits sooner, it may continue to receive the previous snapshot until the 15-day window has passed. The next crawler request after that window receives the existing snapshot immediately while Caffeine refreshes it in the background.
People visiting in a normal browser always see the current live app, so you can share the updated URL immediately.
Can I force a refresh myself?
There is no setting for that. Caffeine refreshes a page when a crawler requests it after the snapshot is older than 15 days.
Does this work with a custom domain?
Yes. Snapshots follow the public URL crawlers use, including a connected custom domain.