How often does the google bot crawl your site ?

Discussion in 'Google' started by Tigger, Feb 9, 2010.

Thread Status:
Not open for further replies.
  1. Tigger New Member

    Member Since:
    Jan 10, 2010
    Message Count:
    32
    Likes Received:
    1
    Occupation:
    Hired Goon
    Location:
    Ireland
    I think on average the normal site gets crawled a couple of times a month but larger sites like digg etc get crawled loads of times a day , how do you know when its time for google to crawl your site more ?
  2. Adam H Administrator

    Member Since:
    Jan 7, 2008
    Message Count:
    2,892
    Likes Received:
    188
    Occupation:
    Head of Search
    Location:
    Norfolk Uk
    there is no real way to "tell the bot" when to visit or how many times it visits however you can turn up the crawl rate in your google webmaster tools panel which will increase how fast it crawls your site
  3. ~kev~ Active Member

    Member Since:
    Dec 5, 2008
    Message Count:
    432
    Likes Received:
    25
    Location:
    Texas
    On my main forum - google is on it all the time. I can post a new forum thread and within minutes its already indexed with google. But this forum is almost 4 years old and has thousands of backlinks.

    On my new forums, google might index a new thread every 24+ hours. These sites have only been opened about 1 1/2 years.
  4. jaikanth12 Member

    Member Since:
    Jan 9, 2010
    Message Count:
    491
    Likes Received:
    11
    Google’s spiders crawl the web on a regular basis to rebuild their index. Crawls are based on a number of factors, including Page rank, links to a page, and a web site’s structure. This is just a small list. There are a variety of factors that can affect the crawl frequency of individual sites.
  5. webtalk Member

    Member Since:
    Nov 11, 2009
    Message Count:
    133
    Likes Received:
    0
    it depends on allot of factors ,there are low PR sites crawled quick by google, also how quick and often you update your website with contents , google sits for long time on high PR sites
  6. AnnieSh New Member

    Member Since:
    Feb 13, 2010
    Message Count:
    10
    Likes Received:
    0
    Crawls are based on a number of factors, including Page rank, links to a page, and a web site’s structure, and ofcourse then there is google dance don't be freaked out just a part of the game.
  7. sazymi New Member

    Member Since:
    May 13, 2009
    Message Count:
    1,384
    Likes Received:
    57
    Occupation:
    SEO, SMO, Blogger
    Location:
    Los Angeles, CA
    Bots can will crawl a site every time it has a new content or post and every time that a particular keyword will search in a single querrie because it will look for the relevant site for that particular searches. Spiders usually read a site horizontally from left to right and from top to bottom so every time bots find a keyword or a link the bots will run into it. So its better to have a good structural linking for your site.
  8. joesmith New Member

    Member Since:
    Jan 7, 2010
    Message Count:
    132
    Likes Received:
    1
    We can't predict the no. of times the bots crawl our site. Its depends on several factors. Whenever the sites in which you have your links get crawled, your link also can get crawled.
  9. Red Paul New Member

    Member Since:
    Nov 24, 2007
    Message Count:
    236
    Likes Received:
    3
    Google crawls pages, not whole sites in one go. My sites get crawled daily - use Webmaster Tools to check googlebot activity
  10. Alfred New Member

    Member Since:
    Oct 21, 2009
    Message Count:
    186
    Likes Received:
    3
    Google crawling depends on Page Rank, links to a page, and crawling constraints such as the number of parameters in a URL. etc.. Google crawls famous site like Digg several times a day...
  11. brainberry New Member

    Member Since:
    Jan 2, 2010
    Message Count:
    20
    Likes Received:
    0
    Well said, It depends on a lot of factors. The factors mentioned are main things. Also it depends if its a blog site. A blog is crawled much quickly along with book marking sites.
  12. Barj10 New Member

    Member Since:
    Mar 16, 2010
    Message Count:
    27
    Likes Received:
    0
    There is no real schedule when Google release it's slaves and crawl your website's pages.
Thread Status:
Not open for further replies.

Share This Page