Meta · Indexing
meta-webindexer
Identify meta-webindexer by its user agent. Check request identity, review robots.txt controls and track visits in Aiola Analytics.
Key facts
meta-webindexerWhat meta-webindexer is
meta-webindexer crawls content to improve Meta AI search results and enable citations and links in responses.
In Aiola's taxonomy, Indexing means crawled for ai or search indexing.
meta-webindexer user agent
Look for the match token inside the complete HTTP User-Agent header. Tokens can be spoofed, so use the network checks below for authentication.
meta-webindexer/1.1 (+/documentation/sharing/webmasters/web-crawlers)meta-webindexerOfficial IP ranges
Meta's network is identified here as AS32934. ASN ownership is a useful signal, not a bot-specific allowlist; combine it with the User-Agent and request behavior.
Verify a meta-webindexer request
For meta-webindexer, check AS32934 network ownership alongside the user-agent token and request behavior. An ASN cannot authenticate one bot.
Network ownership covers more than this crawler identity. Use your own request controls and review the vendor documentation before allowing traffic.
Control meta-webindexer with robots.txt
Meta documents crawler controls for this bot family. Robots.txt can direct cooperative crawling, but it cannot prove that a request is authentic.
User-agent: meta-webindexer
Allow: /User-agent: meta-webindexer
Disallow: /Aiola Analytics
See every meta-webindexer visit on your site
Aiola tracks 70 AI, search and link-preview crawlers next to your human traffic. See when meta-webindexer arrives, which pages it requests and how that changes over time. Hosted Analytics needs Pro or a trial.
Common questions
Build, ship and fix in one app
Errors, user feedback, GitHub and analytics live next to your coding agents in one desktop app. Every issue becomes a task an agent can take.