How do I speed up Google's crawl rate for my new website quicker?
I made a new website but Google is crawling my pages very slowly. How do I increase the number of crawled pages? My site is a pretty large site with over 100k pages.
If you have a large site, do not upload ALL your pages - focus on CONTENT PAGES only. (Pages that have lists / directories / search / similar content with different parameters should be avoided at the start)
Try your best to avoid having any downtime for your site - the moment Google detects the site is down for too long, it will PAUSE / REDUCE crawling significantly.
Get more quality backlinks for your site (1 - 2 quality a day is fine)
The rule of thumb is this: Google places a higher priority on websites that are more trustworthy, popular, and relevant.
Anything you do that falls into those 3 areas above will yield you good results.
Google sets aside a crawl budget for each site, and this value rises/falls depending on:
Number of pages indexed
Site speed
Quality of content
Site popularity (backlinks / trust score, etc.)
We've had scenarios where a few hundred thousand pages were crawled within a matter of weeks, and some other scenarios where very few pages (less than 10%) was crawled even after 2 months.
Fundamentally the focus should be to:
Create great content
Grow your site popularity organically (backlinks, media coverage, etc.)
Avoid duplicate content (if you duplicate content, add additional value on top of it)
Ensure all your pages are indexable/crawlable (proper linking, sitemaps)