Allen AI · Training
AI2Bot
Identify AI2Bot by its user agent. Check request identity, review robots.txt controls and track visits in Aiola Analytics.
Key facts
AI2BotWhat AI2Bot is
AI2Bot finds web content used to train open language models. Ai2 publishes its complete request User-Agent and says it can be used to filter or reject crawler traffic.
In Aiola's taxonomy, Training means scraped for model training or dataset development.
AI2Bot user agent
Look for the match token inside the complete HTTP User-Agent header. Tokens can be spoofed, so use the network checks below for authentication.
Mozilla/5.0 (compatible) AI2Bot (+https://www.allenai.org/crawler)AI2BotOfficial IP ranges
Allen AI has not published official IP ranges that Aiola successfully fetched for AI2Bot. Aiola does not list cloud-provider ranges or community guesses as if they authenticated this crawler.
Verify a AI2Bot request
No IP authentication method is documented in this record for AI2Bot. Its user-agent token is a claim, not proof of identity.
Use the exact header token for classification and apply your own access controls. Check the vendor documentation for any newly published IP, token or header verification method.
Control AI2Bot with robots.txt
Allen AI documents crawler controls for this bot family. Robots.txt can direct cooperative crawling, but it cannot prove that a request is authentic.
User-agent: AI2Bot
Allow: /User-agent: AI2Bot
Disallow: /Aiola Analytics
See every AI2Bot visit on your site
Aiola tracks 70 AI, search and link-preview crawlers next to your human traffic. See when AI2Bot arrives, which pages it requests and how that changes over time. Hosted Analytics needs Pro or a trial.
Common questions
Build, ship and fix in one app
Errors, user feedback, GitHub and analytics live next to your coding agents in one desktop app. Every issue becomes a task an agent can take.