Can someone tell me the benefits of using a Proxy Server to crawl other servers? AS I understand it is used for security and enhancing performance by caching recently accessed sites.
Looking at my current Proxy Server Settings I have "Do Not Connect Using a Proxy Server". Will this hinder me from indexing external web sites?
Frustrated beyond belief in indexing external web site as I keep getting proxy server errors.
Do you know someone who can help? Share a link to this thread on twitter, or facebook.