Real-time monitoring of search engine crawlers and bot activity across our platform
Last updated: loading... ยท Refreshes every 30s
Slurp crawler experiencing 200-400ms latency on /api endpoints. Root cause: upstream rate limiting. Mitigation in progress.
Crawl rate reduced to 8 req/min per rate policy. Bot respecting robots.txt directives. No impact on site functionality.
Malicious crawler from IP range 45.12.x.x flagged for aggressive scraping. Automatically quarantined. 3,412 requests blocked.
Crawl-delay increased from 1s to 2s for all non-essential bots during high-traffic window. Restored to default.
New content batch (247 pages) detected. Sitemap automatically updated and pinged to Google Indexing API.