Crawler information
RhinoRankBot: Rhino Rank's web crawler
What it is
RhinoRankBot is Rhino Rank's web crawler. It reads the public pages of websites in the Rhino Rank publisher network so we can find relevant link placement opportunities for our clients. It only fetches publicly available pages. It does not log in, submit forms, or collect personal data.
How to recognise it
RhinoRankBot identifies itself with this User-Agent:
RhinoRankBot/1.0 (+https://www.rhinorank.io/bot)How it crawls
- It fetches one page at a time from any single website, with at least a short pause between requests.
- It honours the
Crawl-delaydirective in robots.txt (up to 30 seconds between requests). - If a site responds with HTTP 429 (Too Many Requests) or 503 (Service Unavailable), it treats that as temporary and tries again later.
Controlling RhinoRankBot with robots.txt
To slow it down:
robots.txt
User-agent: RhinoRankBot
Crawl-delay: 10To stop it crawling your site, or part of it:
robots.txt
User-agent: RhinoRankBot
Disallow: /Allowing RhinoRankBot
If your site uses Cloudflare or another firewall that blocks automated traffic, you can allow requests whose User-Agent contains RhinoRankBot. If you'd like help, contact us and we will work with you.