Is your feature request related to a problem? Please describe.
This will allow us to define the i want to crawl a search engine slowly but amazon with a better speed.
Describe the solution you'd like
make settings specific to individual crawlers instead of global setting
Describe alternatives you've considered
NA
Additional context
NA
Is your feature request related to a problem? Please describe.
This will allow us to define the i want to crawl a search engine slowly but amazon with a better speed.
Describe the solution you'd like
make settings specific to individual crawlers instead of global setting
Describe alternatives you've considered
NA
Additional context
NA