why is google so slow

tfgames

New Member
My site is one week old and msn hits it every days so far like 240 times, why hasnt google even hit me once yet and how long does it normally take? i have loads of other sites linking to it some with pr5 but still google hasnt come to me yet.
P.S howcome this forum is top of the list when you search for "web design forum" but it has a pr0?
 

Kiwiberry

New Member
I’m not sure if this is coincidence or not, but I was reading about google using a sitemap.xml to help index websites. So I read about creating the sitemap.xml file (which is very similar to robots.txt). Once I added it, I started getting hits from the googlebot, and I don’t have a lot of traffic to my site. I’ve got two hits this month, and I’m alright with that.
 

StephanieCordray

New Member
yes, they will still index you.. all a robot.txt does is make sure the robots index every page and block other bots you don't want on your site. Since you want google to index you, you won't have a problem as long as they can find your site. The easier it is to find your site, the better.
 

tfgames

New Member
Hey stephanie, some bot called LinkWalker hit my site around 180 times so i checked on google what linkwalker is and what i saw some guy was saying it downloads everything of your site and stuff. So if i make a robot.txt i can stop LinkWalker from coming back right?. Im still new and had never heard of robot.txt before but im gonna read up on it now thanks.
 
Top