• FeedWhip
  • Home
feedwhip
Log in | Sign up
About Feeds Tools

Change_Is_Good

Bug reports, feature requests, and suggestions from Feedwhip's users.
re: Ignore Robots • Steve Leroux [Feedwhip admin] • Sat, Feb 03 at 07:38 PM No comments
This is an interesting suggestion, but I don't think I'm going to make the change.

If someone doesn't want Feedwhip crawling their site, then I need to respect that. The problem is that even though Feedwhip is what I think of as a "benign" crawler, most people just block *all* web crawlers.

The only thing I can suggest is to talk to the owners of the websites you're interested in and see if they'll add an exception for Feedwhip. It's easy to do, and that'll give you a pretty direct answer as to whether they want Feedwhip looking at their website.

Comments


about | contact us | ©2005-2007 Feedwhip
Feedwhip never sends email unless you ask us to.
0.074 seconds
Email Address:
Password:
Remember me on this computer
forgot my password
Your Email Address:
Choose a Password: