updated bot rate
This commit is contained in:
parent
963d9209ca
commit
27203ff085
1 changed files with 1 additions and 1 deletions
|
|
@ -33,7 +33,7 @@
|
|||
<dd>It does not scrape page content, follow links, or index your site.</dd>
|
||||
|
||||
<dt>Rate</dt>
|
||||
<dd>At most 5 requests per domain, spread across the full scan.</dd>
|
||||
<dd>Downloads all the favicons listed on the site once a month.</dd>
|
||||
|
||||
<dt>User-Agent</dt>
|
||||
<dd><code>Mozilla/5.0 (compatible; EveryTabBot/1.0; +https://everytab.site/bot.html)</code></dd>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue