An update was pushed to Feedwhip's servers tonight which adds support for the robots.txt web standard. This means that webmasters can automatically ask Feedwhip to not access their pages.
This shouldn't happen very often, but if you've whipped a feed where this is the case, you'll now see a warning message instead of your usual updates. There's nothing we can do about this at our end -- we want to respect the wishes of the larger 'net community. We recommend you talk to the webmaster of the relevant pages and ask them to add an exception for "feedwhipbot" to their robots.txt file.
This shouldn't happen very often, but if you've whipped a feed where this is the case, you'll now see a warning message instead of your usual updates. There's nothing we can do about this at our end -- we want to respect the wishes of the larger 'net community. We recommend you talk to the webmaster of the relevant pages and ask them to add an exception for "feedwhipbot" to their robots.txt file.
As always, we'd love to hear your comments and suggestions.
Steve Leroux
Feedwhip