Search Engine Spider Map Creator Frequently Asked Questions

Spider Map General

What's The Difference Between A Spider Map, A Site Map And A Links Page?

A "site map" is intended for humans, it may link to every page on a site or just some pages within each major section of the site.
A "spider map" or "search engine spider map" is intended for the search engine spiders, it links to every page on the site which you want indexed.
A search engine spider map may also be called a "links page" but that term is more commonly used to refer to a page with reciprocal links.

For a more detailed comparison of site maps and spider maps see site maps and spider maps compared.

Why Should I Use A Search Engine Spider Map?

The spider map links to every page on your site which means the search engine spiders can easily find and index your pages. Because all pages are only 1 level deep from the map, a spider map usually means that more pages will be spidered & indexed.

What Factors Should I Consider In Deciding To Add A Search Engine Spider Map?

How big is your site? The larger the site the more likely a spider map will help.
What is your PR on Google? Sites with low PR may benefit more from a site map because Google normally doesn't spider low PR sites deeply.
How deep is your site's hierarchy? The more clicks required to reach a page from your home page the more a spider map will help you.

Your Spider Map Creator Follows The Links To All My Pages, Why Won't Googlebot?

In theory Googlebot can find all the same pages as the spider map creator. In practice, Googlebot may not spider pages many clicks from your home page. Googlebot seems to determine how deep to spider a site based on the PR (Page Rank) of the site. The larger your site and the lower your PR, the more a spider map is likely to help you.

What Does A Spider Map Look Like?

Here are links to the spider maps for this site:

Those Spider Maps Are Ugly, Can I Change Them To Match My Site?

Feel free to copy and paste the spider map body in to a template for your site or modify the map format as you see fit (but be sure to leave the link to searchenginepromotionhelp.com at the bottom of the page). Each element of your map has a CSS style tag applied so you can easily change the appearance of the map.

How Do I Use The Spider Map?

After you have created your map you should link to it from your other pages. A link at the very bottom of the index page is OK, linking from every page is better for larger sites.

Depending on the search engine and your SEO strategy, you may also submit the spider map directly to the search engine.

Will The Search Engines Penalize Me If I Use A Spider Map?

No. Using a spider map is encouraged by Google:

"Every page should be reachable from at least one static text link" [1]
"Allow search bots to crawl your sites without session ID's or arguments that track their path through the site."[1]
"If your site's internal link structure does not provide a path to all your pages, our robot may not see all the pages on your site." [2]

A spider map (like any other SEO tool) can be abused and get you in trouble with the search engines. Use moderation and common sense and the spider map should improve the indexing of your site.

Spider Map Creator Is Free, What's The Catch?

Creating a map is a free service, in exchange we ask that you leave the link to our site at the bottom of the spider map.

Spider Map Creator Problems

Some Of My Pages Weren't Found

The spider map creator uses a simple spidering routine. If you have links using java or dynamic URLs then those pages won't be mapped.

My Page's Descriptions and/or Titles Aren't Right On The Spider Map

If the spider map spider can't find valid META tags it uses 'TITLE NOT FOUND' or 'DESCRIPTION NOT FOUND' in the spider map. You should repair your tags and confirm that the HTML is correct using http://validator.w3.org/

A tool for preparing your pages for the search engines is IBP's Top 10 Optimizer

How Can I Thank You For The Spider Map Creator?

Please Thank Us With A Link if you find the spider map creator useful.

I Have A Question/Suggestion/Complaint About Spider Map Creator

Please contact us if you are having problems with the spider map creator. If your mail is about a problem you have creating a spider map please provide as much information about your site (url, etc) and the problem as possible. Please note that we can't debug your HTML code for you. You should verify that your HTML is correct using http://validator.w3.org/

Search Engine Spider Map FAQ References

[1] http://www.google.com/webmasters/guidelines.html

[2] http://www.google.com/webmasters/2.html

Get A Top 10
Search Engine Ranking