Controlling where search engines go

I finally had reason to create a robots.txt file. This file implements a protocol used to tell search engine spiders to ignore certain pages on a website. I noticed that google was turning up a bunch of the meta-pages that are in the Sillysoft wiki, which was not desired. There are multiple meta-pages for every real wiki page, so they were drowning out the real content. By telling search engines to ignore the undesirable pages it focuses them on the pages that actually have useful content on them.

Posted by dustin on December 24, 2004 with category tags of shareware

Vorg — Tag Cloud

Written by dustin
skyfox.me
Who is Dustin "Quasar" Sacks?
I am a multi-media artist and producer. I work with video, gaming, photography, music, festival culture and personal development.

View my personal website skyfox.me for more details.
dustin's RSS feed
Latest Photo

[Dustin on instagram]

Quote of Now:
I love Orlando Bloom![quote database]
Friends
Amir Niroumand
Best Dad Ever
Blue Apples
Blue Lotus Creative
BoingBoing
Carolyn Coles
Coherent Earth
Dave Olson
DinoCats
Dream Director
Dustin Sacks
Edcon
Educational videos on YouTube
Erin Levine
Father's Day Gifts
Find Your True Nature
Graydin
Healing from Inside
Hot Grog
Integrate Therapy
Intent Health
Joffrey
Kate Pocock
Kindara Fertility
Koobee Kids - fun learning videos
Lady Zenaida
Lantern Park
Lindsey Rogerson
Patrick Adams
Paw Pros
PEMF Solutions
Said the Gramophone
Shala Yoga
Sillysoft Games
The Travel Files
Popular Posts
Create a Custom Avatar
Drupal forums
EA Games Interview
Explosion Animations
Healthcare - USA vs Canada
Hedonics - happiness economics
How-To Make your own devil sticks
Koodo Mobile
Littlest Hobo
MySQL limit in MS-SQL
Selling on Amazon.com
Vancouver Bloggers
Computer Games
Lux Alliance Diplomacy Game
Lux Risk Game
Meowzers Action Cats
Mission Me app
Turtoa Global Rhythm

Hey You! Subscribe to dustin's RSS feed.
Or get wider opinion in the Vorg All Author feed.