Huawei · Indexing

PetalBot

Identify PetalBot by its user agent. Check request identity, review robots.txt controls and track visits in Aiola Analytics.

Download AiolaAI crawler directory

Key facts

User-agent token: PetalBot
Indexing: Crawled for AI or search indexing.
Verify requests with reverse DNS; Huawei documents the hostnames.

What PetalBot is

PetalBot is the crawler for Huawei's Petal Search. Petal documents reverse DNS followed by a forward lookup confirming the original IP. Its platform instructions name petalbot.*.aspiegel.com. Its lookup instructions and example use petalsearch.com. These instructions conflict.

In Aiola's taxonomy, Indexing means crawled for ai or search indexing.

PetalBot user agent

Look for the match token inside the complete HTTP User-Agent header. Tokens can be spoofed, so use the network checks below for authentication.

Full example User-Agent
Mozilla/5.0 (compatible; PetalBot; +https://webmaster.petalsearch.com/site/petalbot)
Case-insensitive match token
PetalBot

Official IP ranges

Huawei supports reverse-DNS verification for this crawler family. Accepted suffixes in this snapshot: aspiegel.com, petalsearch.com.

Use a double lookup: reverse-resolve the request IP and check the exact hostname suffix. Then forward-resolve that hostname and require the original IP. Checking only the visible hostname is vulnerable to attacker-controlled PTR records.

Verify a PetalBot request

For PetalBot, reverse-resolve the source IP recorded by your trusted edge or server, then check the documented suffixes.

Reverse- and forward-DNS double lookup
host REQUEST_IP
# Confirm the hostname ends in an official suffix
host RETURNED_HOSTNAME
# The forward lookup must return REQUEST_IP

Resolve the verified hostname forward and require the original IP in the result. A PTR hostname alone can be controlled by an attacker.

Control PetalBot with robots.txt

Huawei documents crawler controls for this bot family. Robots.txt can direct cooperative crawling, but it cannot prove that a request is authentic.

Allow PetalBot
User-agent: PetalBot
Allow: /
Block PetalBot
User-agent: PetalBot
Disallow: /

Read official crawler documentation

Aiola Analytics

See every PetalBot visit on your site

Aiola tracks 70 AI, search and link-preview crawlers next to your human traffic. See when PetalBot arrives, which pages it requests and how that changes over time. Hosted Analytics needs Pro or a trial.

Download AiolaSet up bot tracking
Aiola Analytics showing AI crawler visits by bot and page

Common questions

Build, ship and fix in one app

Errors, user feedback, GitHub and analytics live next to your coding agents in one desktop app. Every issue becomes a task an agent can take.

Download Aiola