Why Google Crawled but Currently Not Indexed Your Pages-How to Fix it

byBrendaMZ

7.5K followers

Updated 4 months ago

(cover image: Magnifying glass over the word Google color blue, red, yellow, and green lettering)

Hi Wealthy Affiliates,

If you are like me, you are puzzled as to why you see pages under Crawled-Not Currently Indexed in Google Search Console. When you click that, you can see that your posts are still nowhere to be found in the search results. It can be frustrating when your valuable content remains hidden from potential visitors.

"Crawling" is when search engines scan and analyze web page content using automated bots. These bots follow links, collect information, and constantly update web pages. When a page is crawled, its content is indexed and organized by Google for quick retrieval during user searches.

Let's discuss why this might happen and explore solutions to get your pages in the spotlight.

What do Crawling and Indexing Mean?

Have you ever wondered how search engines like Google find and organize information online? It all starts with crawling and indexing, which is how the search engine discovers and stores web pages on its servers. That process is essential for Google to provide relevant and accurate results when users enter a query in the search bar.

1. Crawling: Google's first step is crawling. Googlebot, an automated software program, visits websites to discover new or updated content by crawling through a list of known URLs, including popular sites, submitted sitemaps, and links found on previously crawled pages.

Googlebot follows links from one page to another and collects information about those pages. It then sends this information back to Google's servers for indexing.

2. Indexing: After crawling a website, Googlebot sends the collected information to be indexed. Indexing is the process of storing and organizing this information in Google's index or database. Think of it as organizing books in a library—without proper indexing, finding what you need would be chaotic.

Ensure
When your website is crawled but not indexed, it means that Google knows about your pages but hasn't deemed them worthy of being included in its index yet. This could be due to poor-quality content or technical issues hindering proper indexing. That is why you must constantly check Google Search Console; it will look like this.

Reasons Why Google May Not Index Your Pages After You Requested Indexing

You got your post indexed, but later, Google decided to de-index your articles. Why does that happen? As you can see, there are various reasons why Google may not index your pages even after you have requested it.

1. We are not focusing on links with /feed. What does /feed mean? It means that WordPress automatically creates these for blog posts. They're like individual RSS feeds. Those are not indexed, but they are still part of your site. You do nothing- there is nothing to fix when you see that; I am telling you right now, ignore it.

See this screenshot example.

1. We are looking for blog posts that have been indexed, but then later, Google decided to show those posts as crawled but not currently indexed. It will show your permalink as an example: yourdomianname.com/org/net /google-article-not-inde
2. We need to check those pages to ensure that Google crawls all the relevant pages on your site efficiently.

There are various reasons why Google might not index your pages even after you've requested them:

• Technical Issues: Google may have difficulty crawling your site due to slow loading speed, broken links, or a lack of quality content and relevant keywords.
• Low-Quality Thin Content: If your website has low-quality or thin content, Google may not index it. Ensure your content is original, high-quality, and relevant to your audience.
• Incorrect Robots.txt File: The robots.txt file tells search engines which pages or sections of your site they should crawl and which ones they shouldn't. If this file is set up incorrectly, it can prevent Google from indexing certain pages on your site.
• Broken Links and Faulty Redirects: Google won't index pages with broken links caused by URL changes or removed content. Faulty redirects can send users to incorrect pages and prevent Google from indexing your site.

Google might not index your pages after you request them. Therefore, it's essential to regularly check for technical issues, monitor the quality of your content, and ensure that your site meets Google's guidelines.

• Slow-Loading for Mobile Speeds: Remember, Google uses mobile page speed as a ranking factor. If your site is slow to load on mobile, it may not get indexed. Check your site's mobile loading speed using Google's PageSpeed Insights tool and make necessary optimizations.
• Improper Use of Noindex Tag: The noindex tag, like the robots.txt file, tells search engines not to index a specific page. Google will exclude them from search results if used accidentally on essential pages.
• Canonicalization Issues: “When multiple versions of the same page exist (e.g., with different URLs), Google may struggle to determine which one to index, potentially leading to duplicate content issues and causing some pages to be de-indexed by Google.
• Server Issues: Server errors or downtime can prevent Google from correctly crawling and indexing your site. Ensure your server is functioning correctly and there are no redirects or page loading speed issues.
• Domain Age and Authority: New websites with little authority may struggle to get indexed by Google. It takes time to build trust with search engines and establish yourself as a reliable source of information.
  

With patience and consistent effort, you can improve your site's chances of being indexed by Google and increase its visibility in search engine results.

How to Check and Fix Your Crawled-But Not Indexed Pages

So, you've noticed that Google has crawled your website, but for some reason, it's not indexing your pages. Don't panic just yet - there are steps you can take to troubleshoot this issue.

Step 1: Check if Google has crawled your website: You can type "site:yourwebsite.com" into the Google search bar. This will show you all the pages Google has indexed for your website.

Step 2: Pages are missing from the indexed list: Technical issues may prevent them from being crawled and indexed. Some common problems listed above may cause issues.

Step 3: If none of these issues seem to be causing the problem, You can use Google Search Console to request indexing again for specific URLs. This will prompt Google to recrawl and hopefully index those pages.

Step 4: Another potential issue could be a slow site speed: Google tends to prioritize faster-loading pages for indexing, so if your site takes too long to load, it may hinder the indexing process. Use a tool such as Google's PageSpeed Insights to analyze your site's speed and make improvements if necessary.

Step 5: Make sure your website has a sitemap. A sitemap is a file that lists all the pages on your website and helps search engines understand its structure. Creating and submitting a sitemap to Google can make it easier for them to crawl and index your pages.

Step 6: Check for duplicate content: If you have multiple pages with the same or similar content, Google may not index all of them. Make sure each page on your site has unique, quality content.

Step 7: Monitor for broken links: If Google encounters too many broken links while crawling your site, it may give up and not index certain pages. Regularly check for broken links on your website and fix them promptly.

Please do the following.

1. Ensure that your pages' content is unique and valuable. Google prioritizes high-quality content when indexing websites. Next, check for any technical issues blocking Google from indexing your pages correctly.
2. Use Google Search Console to request re-indexing of specific pages. This can prompt Google to revisit and hopefully index them accordingly. Additionally, ensure no meta robots tags or directives instructing search engines not to index certain pages.
3. Monitor your website's indexing status regularly to catch any discrepancies early on. By staying proactive and addressing any issues promptly, you can improve your chances of getting your crawled but not indexed pages back in the game.

Review and fix any pages that have been crawled but not indexed. Check for technical issues, ensure unique and valuable content, request re-indexing through Google Search Console, monitor for broken links and duplicate content, and regularly check your website's indexing status. If the problem persists, consider seeking help from an SEO specialist or asking for help in the WA community.

Final Thoughts

I used to have articles de-indexed, but I improved the content, and Google indexed them. Monitoring your website's indexing status is crucial. Check "Crawled-Not Currently Indexed" regularly. It's important to troubleshoot and improve your content for indexing. If it is not indexed, consider removing or using it for internal linking.

PS. I tried hard to make it short, but it's detailed information. Have you dealt with issues like this? Tell me what you did to solve those de-indexed posts.