Have you been visited by Mouse?
Mouse is an open-source search module for Aneamal, free software to run websites. Aneamal and Mouse are uploaded to the webspace where they are used. Hence there are multiple independent instances of the Mouse crawler controlled by different people.
The main purpose of Mouse is to enable webmasters to create a search function for their own websites, but the module can also be used to crawl and search other websites.
If your website is visited by a Mouse crawler and you do not want it to crawl your site anymore, add the following to a robots.txt file at https://your.domain/robots.txt
User-agent: Mouse
Disallow: /If no rules have been set for User-agent: Mouse, the crawler will also respect rules for User-agent: *. The Mouse crawler should then stop crawling your site within 12 hours.
Mouse also respects noindex and further directives in HTML meta elements and HTTP x-robots-tag headers.
Crawler specs
- Revisit robots.txt after
- 12 hours
- Maximum redirects to fetch robots.txt
- 5
- Maximum robots.txt size
- 512000 bytes
- Minimum pause between crawls
- 29 seconds
- Transfer timeout after
- 16 seconds
- Maximum general response size
- 8388608 bytes
- Maximum URL length without fragment
- unlimited (Mouse/1)
255 bytes (Mouse/1.0.1+) - Observed robots directives
- nofollow, noindex (Mouse/1+)
none, nosnippet (Mouse/2+) - Avoids extensions
- 7z, bmp, css, doc, docx, exe, flv, gif, gz, heic, heif, ico, jpeg, jpg, js, json, mid, midi, mov, mp3, mp4, mpg, ogg, ogv, pdf, png, ppt, pptx, rar, svg, svgz, swf, tif, tiff, wav, webm, webp, xls, xlsx, xml, zip (Mouse/1+)
avif, jxl (Mouse/2+)