Adding / Removing Your URL
Adding Your URL
Note! Before submitting any Web page to a search engine you should make sure it is at least somewhat "search engine friendly". You can do this using our Free Meta Tag Analyzer.
Submitting your Web page to Scrub The Web is 100% free. However no URL submission is guaranteed to be included in our index.
How do I submit to Scrub The Web?
You can use our Submit URL page to submit to Scrub The Web. You can also use our free online Easy Submit program to submit your Web pages to Scrub The Web as well as most all major search engines quickly and easily.
Why do you require an email address when submitting?
The email confirmation process helps us prevent invalid and spam submissions which consume valuable system resources. We do NOT sell or use your email address for any other purpose other than this confirmation email.
How long does it take to get my URL in Scrub The Web?
We are continually crawling our existing index as well as crawling new confirmed submissions around the clock. Usually you will see your site indexed within 7 to 14 days, but could be longer because we also need to re-index the existing index as well which all takes time.
I keep submitting, but I never find my site. Why not?
ScrubTheWeb.com is not obligated to index your submitted pages. Scrubby often randomly selects URLs to be included in the index. Do not email us asking Scrubby to index your site as these email will be ignored.
How do I update my listing once it is in Scrub The Web?
Scrub The Web will refresh the database automatically over time. There is no need for you to do anything. If the content changes, the URL has moved or the URL is no longer available, it will be updated when Scrubby next visits the Web page.
I have moved and my URL is no longer available. How do I delete the old listing?
Scrub The Web will refresh the database automatically over time. There is no need for you to do anything. If the content changes, the URL has moved or the URL is no longer available, it will be updated when Scrubby next visits the Web page.
How does Scrub The Web determine which sites come up first when I do a search?
Each record gets a grade based on the relevancy of the search terms it contains when a searcher searches. For more information on our search technology click here.
How do I get my page to come up higher when someone is doing a search?
The best way is to optimize your Website which will improve your rankings in all search engines.
How can I remove a page from Scrubby's index?
If you wish to exclude your entire Website or a specific section (directory) of your server from Scrubby's index, you can place a file at the root of your server called robots.txt.
To prevent Scrubby and other search engines from crawling your site, place the following robots.txt file in your server root:
User-agent: *
Disallow: /
If you wish to only exclude Scrubby from indexing your site, place the following robots.txt file in your server root:
User-agent: Scrubby
Disallow: /
You can also remove individual pages using this same method. This is the standard protocol that most web crawlers observe for excluding a web server or directory from an index. More information on robots.txt is available at 
http://www.robotstxt.org/robotstxt.html.
There is another standard for telling robots not to index a particular web page or follow links on it, which may be more helpful for some users, since it can be used on a page-by-page basis. This method involves placing a "Meta" element into a page of HTML.
Remember, changing your server's robots.txt file or changing the "Meta" elements on its pages will not cause an immediate change in what results Scrubby returns. It is likely that it will take a while for any changes you make to propagate to Scrubby's next index of the web.
For tips on how to improve your position on all major search engines including ScrubTheWeb.com, see our Members page for details.
How To Not Get Listed
- Typographical errors. Make sure you enter the Web address correctly. Double-check the spelling of the Web address before you submit. Enter it in your browser to make sure it works. Spaces are NOT allowed in a valid URL.
- The Website was inaccessible. Scrubby could not access the Website after several attempts. The Web server may be down or the site may have disappeared since you submitted it. Some host providers do a restart of servers and if this happens during the time Scrubby tries contacting your Website, you will not be indexed.
- Websites that employ practices such as unsolicited email may be excluded from our index without notice.
Important: Remember that Scrub The Web is under no obligation to accept Websites you recommend for inclusion in the index. We reserve the right to remove or exclude pages from the index at any time, for any reason, or for no reason at all.