Baidu · Indexing

Baiduspider

Identify Baiduspider by its user agent. Check request identity, review robots.txt controls and track visits in Aiola Analytics.

Download AiolaAI crawler directory

Key facts

User-agent token: Baiduspider
Indexing: Crawled for AI or search indexing.
Verify requests with reverse DNS; Baidu documents the hostnames.

What Baiduspider is

Baiduspider discovers and indexes pages for Baidu Search. Baidu documents the Baiduspider robots.txt token and reverse-DNS hostnames under baidu.com or baidu.jp.

In Aiola's taxonomy, Indexing means crawled for ai or search indexing.

Baiduspider user agent

Look for the match token inside the complete HTTP User-Agent header. Tokens can be spoofed, so use the network checks below for authentication.

Full example User-Agent
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Case-insensitive match token
Baiduspider

Official IP ranges

Baidu supports reverse-DNS verification for this crawler family. Accepted suffixes in this snapshot: .baidu.com, .baidu.jp.

Use a double lookup: reverse-resolve the request IP and check the exact hostname suffix. Then forward-resolve that hostname and require the original IP. Checking only the visible hostname is vulnerable to attacker-controlled PTR records.

Verify a Baiduspider request

For Baiduspider, reverse-resolve the source IP recorded by your trusted edge or server, then check the documented suffixes.

Reverse- and forward-DNS double lookup
host REQUEST_IP
# Confirm the hostname ends in an official suffix
host RETURNED_HOSTNAME
# The forward lookup must return REQUEST_IP

Resolve the verified hostname forward and require the original IP in the result. A PTR hostname alone can be controlled by an attacker.

Control Baiduspider with robots.txt

Baidu documents crawler controls for this bot family. Robots.txt can direct cooperative crawling, but it cannot prove that a request is authentic.

Allow Baiduspider
User-agent: Baiduspider
Allow: /
Block Baiduspider
User-agent: Baiduspider
Disallow: /

Read official crawler documentation

Aiola Analytics

See every Baiduspider visit on your site

Aiola tracks 70 AI, search and link-preview crawlers next to your human traffic. See when Baiduspider arrives, which pages it requests and how that changes over time. Hosted Analytics needs Pro or a trial.

Download AiolaSet up bot tracking
Aiola Analytics showing AI crawler visits by bot and page

Common questions

Build, ship and fix in one app

Errors, user feedback, GitHub and analytics live next to your coding agents in one desktop app. Every issue becomes a task an agent can take.

Download Aiola