Google Indexing Problem: Fixed!
Google Indexing Problem
Posts not being indexed by Google has been a problem experienced by a number of Wealthy Affiliate members for some time now.
Whenever I publish a new blog post, I submit it to Google Search Console and Bing Webmaster Tools and expect it to be indexed within a couple of days. Whether or not it gets ranked for particular keywords/search terms is another matter, of course and depends on a lot of factors, but unless I've done something really stupid, indexing should be a matter of course.
BUT...
I had one blog post on a particular site that Google just refused to index. It was a good keyword, all the content was original, it was quite long, all the SEO was set up properly, but GSC said the last crawl was back in mid-January (when the post was published) and it hadn't been crawled since. The only information was the useless "crawled, not indexed" and "but not because of an error". I had plenty of subsequent posts that had been indexed fine.
I tried getting other WA members to add comments and I tried on-site linking to and from it to other, related posts. Nothing worked. I must have resubmitted it to GSC twenty times, but the last crawl stubbornly remained at mid-January.
Now no one knows exactly how the Google algorithms work, so I created a mental model of what appeared to be happening.
It was as if Googlebot had decided that under no circumstances was it going to ever visit this post again. No changes to the post would ever be noticed because any request to include the post in the index would be ignored.
With that model in mind, clearly I had to make Googlebot think it was a new post, not the same old one it had taken such a dislike to.
So I gave the post a similar but different name (using Jaaxy to find a decent traffic but low QSR one), updated the slug and meta data to be consistent and changed the date to a recent one.
I then submitted the new URL to Google Search Console.
Index Success
The new/old post was successfully indexed within 12 hours!
What About Others?
OK, so I then checked ALL the blog posts on this site and found 11 others with the same problem.
I didn't really want to change the date on all of them as they appeared in a logical sequence so I left that part out, figuring that the name and slug change were probably the critical factors.
The first of the 11 was updated last night and Google had indexed it by this morning, so it appears to be on the right track. Now because these are old posts, I'm taking the opportunity to update some content, but I don't believe that's factor in the reindexing. It's just a bonus.
Incidentally, the second of the 11 also got reindexed along with the first. No change had been made to it, but I'm putting that down to internal links.
Now to do the remaining 9! And check out other sites.
I hope others find this useful.
UPDATE March 27
Of the original 11 that hadn't been indexed for the last couple of months, 10 have now been indexed by Google!
And it seemed to be getting faster and faster, with the last one being added to the Google database within an hour or so of being submitted.
Recent Comments
166
Thank you Phil,
Your post gives me some new perspectives :) This is so cool. I did change some post more than a few times, cos it has the red letter in the SEO section and that makes me worried. Still, that red T is there. Indexing from Google is another level ;) Do you know the way how I can ask for a support team to check my website and maybe fix some irregularities? Kyle mentions it in some of the lessons in the first level, I need to go back there but not sure where?
Nela
Your best bet is to click on help, then Site Support. Select your website and outline your issues/irregularities. They should be able to help.
I've had a few posts that refused to get indexed over time.
Some I got to index by tweaking the posts a little and changing the post titles, but I kept the slug as it was.
I may have to look into changing the slug on a few.
Haven't tried to do that yet.
Thanks!
John
Here's an update from today. Of the original 11 that hadn't been indexed for the last couple of months, 10 have now been indexed by Google!
And it seemed to be getting faster and faster, with the last one being added to the Google database within an hour or so of being submitted.
Wow, that sounds like a Win/Win and the best solution.
I won't hesitate to do that now, in some of the training they mention to not change the slug, it's why I never have.
Great stuff!
Phil,
Have been having the same issues, and have been resubmitting, getting a few in along. I am blaming the poor internet service I have. We are to have a fiber optic option in the next couple of months.
Indexing has always been a challenge, but until last year, would eventually happen.
I have not changed the title, but have done much editing to keep the posts current.
Every morning starts with URL submissions. Living in rural areas does come with issues. I do appreciate this discussion about this factor of my operation.
Sami
Sami,
I am starting out with WA and building out my 1st niche and live in a rural area with DSL not fiber. Do you find that a challenge in getting keyword posts indexed for rankings?
Paul
Paul, I do think this has had some bearing, however, once a group is finally indexed, they begin to build a bit of momentum. Not fast but eventually. So, I continue to work through the submitting URL thing, to get some recognition.
I do think the service I have causes some issues, but when it is all I can get at the present, I just have to work around the issues and keep going.
Best Wishes,
Sami
Internet speed should have nothing to do with Google indexing, Sami. Once it's submitted, it doesn't matter whether that took 5 seconds or 5 minutes, it's sitting there for Googlebot to process. The speed at which that happens depends of Google's servers and connections, no yours.
Here's an update from today. Of the original 11 that hadn't been indexed for the last couple of months, 10 have now been indexed by Google!
And it seemed to be getting faster and faster, with the last one being added to the Google database within an hour or so of being submitted.
Hi, Phil
It sounds like you are on the right path. I read that from time-to-time a post may be “hand reviewed” by an actual human at Google, as a measure of quality assurance.
Maybe that had something to do with it?
Luckily, so far I haven’t had your problem, but I’ll definitely keep that solution in mind.
When it comes to indexing and ranking, in the end you never really know what Google or any other search engine is going to do.
Frank
All very true, Frank, though I suspect my issues were with a Googlebot algorithm, rather than a person.
Here's an update from today. Of the original 11 that hadn't been indexed for the last couple of months, 10 have now been indexed by Google!
And it seemed to be getting faster and faster, with the last one being added to the Google database within an hour or so of being submitted.
See more comments
Great advice Phil!
I am now getting my articles indexed more frequently.
However, one quick question!
Is there a difference for when Google sends me an email letting me know that an article has been found and is indexed, vs. me going on GSC and seeing that it was indeed found and indexed???
Just curious...thank you!
-Mike
I am now making that a regular practice.
-Mike
How do you get an email from Google that an article has been indexed? Is that a Site Content thing (I don't use it)? I wouldn't think there's any difference.
It’s actually an email from WA’s Site Content (see attached). I then go to GSC to verify. I have found that the email doesn’t get sent for all — as there are posts that GSC has found and indexed, but Site Content hadn't sent the email notice.
-Mike
Thanks for the clarification, Mike. I was aware of the WA email but as you said it was from Google, I wasn't sure if that was what you meant. I thing GSC is a more reliable source.
I do too, and that’s why I now checkin every couple of days — and if I see a post hasn’t been found/ indexed…I’ll request that it does on the Google Search Console.
Thank you Phil!
-Mike