We have bot protection, in the form of cloudflare turnstile, to protect again malicious bots. However, there are "good bots", like Google Bot and Bing Bot, that are exempted from turnstile so that our site is searchable. Unfortunately, when good bots hit the download link, it will trigger remediate for a file that would not otherwise be looked at. In 5 minutes, these good bots used up our page count limit for the day.
We want to continue to allow these good bots to access our site, but not trigger remediation when they download a file.
We have bot protection, in the form of cloudflare turnstile, to protect again malicious bots. However, there are "good bots", like Google Bot and Bing Bot, that are exempted from turnstile so that our site is searchable. Unfortunately, when good bots hit the download link, it will trigger remediate for a file that would not otherwise be looked at. In 5 minutes, these good bots used up our page count limit for the day.
We want to continue to allow these good bots to access our site, but not trigger remediation when they download a file.