Spider Not Working


Occasionally I get support emails from customers saying the spider isn’t working on Web Content Studio.

The typical error message in WCS is:
".. cannot continue with the spider because no search URLs were found in this term”

OK; there are a few reasons this can happen.  The main one is that you have not upgraded to the latest version.  If the auto-upgrade in WCS does not seem to be working, please see this post.

As per the instructions on the download page, please always run WCS as administrator. 

OK; if you are still having problems, here are the main reasons the spider can fail:

  1. Google has changed the way it displays it's results, and WCS needs an update.  This happens very infrequently (only once in the last 5 years).
  2. A firewall or anti-virus/anti-spam tool is blocking WCS from accessing Google.
  3. You currently have a temporary Google IP ban for automated queries.  Go to Google and do a search to make sure Google is providing your computer with results.  I should point out that I have never received a Google IP ban for using Web Content Studio, as it only queries Google once per search.  However, other "SEO" software is notorious for causing this type of problem, so using other SEO software can cause a temporary ban, and WCS is then unable to access Google results.

Leave a comment

Your email address will not be published. Required fields are marked *