How to make Bing crawl you site faster
Crawl Control
The Crawl Control feature in the Configure My Site Section allows you to exert control over the speed at which Bingbot makes requests for pages and resources on your website. You can tell Bingbot to crawl your site faster or slower than the normal crawl rate for each of the 24 hours in the day. This way you can to limit Bingbot activity when your visitors are on your site and allow us more bandwidth during quieter hours.
| Note |
|---|
| If we find a crawl-delay: directive in your robots.txt file then it will take always precedence over the information from this feature. |
How to Set a Crawl Pattern Using a Preset
The easiest way to set a crawl pattern that optimizes your bandwidth for your visitors is by using the When do you receive most traffic during the local time day drop-down menu. This drop-down menu provides a number of preset options which you can use to indicate the busiest time on your site based on your local time of day. When selecting any of the presets, we will automatically optimize the crawl pattern by telling Bingbot to crawl more slowly during those hours and faster during the other hours. All you need to do to store the setting is clicking Save at the bottom of the page.
The following example shows a site that chose the 9AM - 5AM local time preset as the busiest time for the site, resulting in an inverse crawl pattern that allows faster crawl speeds for Bingbot outside of those hours:
How to Set Custom Crawl Pattern
Alternatively, you can use your mouse to create an entirely custom hourly crawl pattern by moving over the cubes holding the left mouse button down and dragging it to the desired number of quadrants for the timeline denoted on the x-axis. More blue quadrants indicate more visits from Bingbot (faster crawl), fewer quadrants mean the opposite (slower crawl). The default is 5 quadrants for each time slice. As a result, 5 blue quadrants represent the algorithmically determined baseline crawl rate for your site. Moving them up or down increases or decreases sets a higher or lower crawl rate based on the baseline crawl rate that was determined automatically for your site. When your happy with your pattern, simply click Save to store the setting.
Recent Comments
10
See more comments
Fantastic info! However I am just a wee bit lost. You mentioned "The Crawl Control feature in the Configure My Site Section" and then "When do you receive most traffic during the local time day" features.
Only problem is, I do not seem to be able to locate either of these.
Where am I going wrong?
Your other info dealing with "how to make Google crawl my site faster" was absolutely perfect. I followed the steps and now I am Indexed !
YaY ! :) Tom
I have added a screen shot of the bing webmaster tools dashboard so that you can see that the "configure my site" section is the first expandable menu and "Crawl Control" is found within it. Once you click on Crawl Control, you will find the graph and the drop down box pictured which has the "When do you receive the most traffic" dropdown box.