BrokerPaysBot collects public pages and documents about broker partner programs for the BrokerPays directory.
- User agent:
BrokerPaysBot/1.0 (+https://brokerpays.com/bot/) - Frequency: about once a week per page; no more than one request every 2 seconds per domain.
- Scope: only public pages listed in our source registry. The bot does not log in, submit forms or solve captchas.
- robots.txt: respected. To exclude pages, add a rule for
BrokerPaysBot. - Contact: [email protected] — for questions, corrections or to suggest the right source page for your program.