Skip to content
Our crawler

ScanFreeBot

The crawler behind ScanFree's free AI visibility check. If you found this address in your server logs, this page tells you who we are and how to stop us.

robots.txt
# stop ScanFreeBot reading your site
User-agent: ScanFreeBot
Disallow: /

Who we are

ScanFree (a service of LMR Enterprises, Anza, California) checks whether AI assistants recommend a business. When someone enters a website address, our crawler reads that site, and the sites of a few competitors that the assistants named, to compare simple things like a sitemap, robots.txt rules, page titles and structured data. Contact: hello@scanfree.dev.

How to recognise us

User agent: ScanFreeBot/1.0 (+https://app.scanfree.dev/bot)

Requests come from the server at 2.24.222.13. We always send the user agent above and never pretend to be another bot or a browser.

What we do

  • We read /robots.txt first and follow it. If it disallows ScanFreeBot (or everyone) from a page, we do not fetch that page.
  • We read your sitemap and at most 12 pages of a site that was submitted, or 6 pages of a competitor's site. We wait at least one second between requests to the same site (longer if your Crawl-delay says so).
  • We fetch plain HTML only. We do not run JavaScript, submit forms, log in or follow links to other sites.
  • We keep derived facts only (for example "has a sitemap" or "title is 54 characters"), and the robots.txt and response headers we received, not copies of your pages.
  • We only fetch public web addresses.

How to stop us

Add this to your robots.txt:

robots.txt
User-agent: ScanFreeBot
Disallow: /

We will stop reading your site the next time we check. You can also write to us and we will add your site to a block list.

If we blocked you by mistake, or you were blocked

If your site answers our crawler with an error or a bot-protection page, ScanFree tells the person who asked that their site (or a competitor's) could not be read. That is a finding, not an attack. Nothing is retried in a loop.