Updated July 3, 2024
Reading Time: 5 minutes
Search engine indexing Issues Everywhere
2022 has been an unpredictable marketing year. Companies, in particular, reported their web pages disappearing from search results. We’ve discovered that several of our clients are seeing multiple URLs tagged as not indexed on Google Search Console (GSC). If you’ve experienced the same issues, this post summarizes Google’s official response (and the online community’s uproar) about accidentally dropping pages from search. Plus, we’ll share the steps we’ve taken to reindex those pages.
Google Confirms a Deindexing Issue July 2022
Complaints started pouring last July 16th. Google announced the issue with search via this tweet:
“Yes, We are still experiencing the index issue. None of our sites are indexing, Please fix this major issue.”
There were reports that pages, previously ranking very poorly, suddenly appeared on the top of SERPs.
Luckily, Google replied with a follow-up tweet announcing they’d identified and fixed the issue the same day. You can view the complete thread on Twitter here. Unfortunately, once a page is deindexed, it’s up to the website owners to take steps to queue the page(s) for google to prioritize indexing it. Search Engine Journal’s post confirmed that the issue was about indexing, not SEO ranking.
Yet Another Deindexing Issue Arises in August
On August 9, Google had a significant outage from the Google data center fire that triggered another round of pages dropping out of the index. The errors were widespread, and it was impossible to send a request in GSC to index anything.
John Mueller of Google was quick to respond by making a post and replying to comments. By the next day, things were back to normal for users but not for site owners. There were reports of mixing keywords or new ones coming up on their Semrush account. In extreme cases, entire websites were completely de-indexed (making them invisible in search results). Needless to say, there’s been some scrambling to get important pages back into the index.
How do you get your pages indexed?
Google never promises to index all pages of a website. At some point, we may never really get a piece of content (either a web page or blog post) on the index at all. Still, we know there are things we can do to raise our chances. Here are eight ways to help your pages get indexed.
1. Fetch your URLs on GSC
It’s essential to have an account on Google Search Console (GSC). The first step when your page gets deindexed is to submit the URL to Google, so it is queued for indexing/crawled by bots. Submitting the request doesn’t automatically add the URL to the index but puts them in the queue. Google gives you step-by-step instructions on how to do it. Sometimes you have to be patient and manually recheck to see if the page has been reindexed
2. Add your pages to your sitemap
Another step is to add the page to your sitemap. This alerts Google by submitting it to GSC. Monitor if any URLs are not registering on the sitemap. You can even create a sitemap for all the photos and images on your site, especially if you want to rank on Image search.
3. HTTPS is invalid and might prevent it from being indexed
One reason your URLs might not be indexed is that Google might not see your site as secure. Google announced that they’re adjusting their search engine indexing system to look for more HTTPs pages. In general, if you haven’t upgraded your site to HTTPS, now is the time. It eliminates unnecessary headaches and shows your website visitors you take web security seriously.
4. Orphaned pages
Orphaned pages don’t have internal links pointing to them. Search engines use links to jump from page to page while crawling. Pages without internal links make it impossible for your web visitors to navigate to them, as there’s no way to arrive on the page other than by a direct link. You can quickly solve this by adding internal links from the main pages to other pages on the site. The goal is to have at least one link pointing to each page. This strategy is harder to implement on a big site, but it will pay off in terms of getting pages indexed. Bonus tip: have your popular posts point to important pages.
5. Remove low-quality pages to optimize the crawl budget
Google tries to minimize effort while maximizing results. One way they reward websites that provide value is to present them at the top of search results (SERPs). The opposite is also true. It doesn’t mean that pages with longer content are better. That’s why some pages can be as short as a paragraph or as long as a book and still rank.
Considering Google’s E.A.T. and the Helpful Content update, factors like your domain’s backlink profile, expertise on the topic, and online reputation play an essential part. And if you have pieces of content that you aren’t giving that value, maybe it’s time to unpublish it or revamp it
6. Share on Twitter
Google crawls Twitter and indexes tweets fast. Twitter also notes that the words you write on your Twitter profile and public tweets are sent to search engine indexes. So, they will come up whenever a search for those terms comes up. Be sure to share your pages on your Twitter account.
7. Share the page on high-traffic sites
Content distribution platforms like Medium, Entrepreneur and similar sites are great for sharing your content. Look for niche-specific websites where you can promote your content. By leaving links pointing to your site from that piece of content, web crawlers can follow them and better understand the relationship between web pages.
8. Backlinking
Although Google’s indexing considers many pages without backlinks, high-quality links are still valuable. According to Ahrefs, Google will likely crawl and re-crawl those pages faster. After all, if someone links to a page, that signals value, and valuable pages are indexed.
Need Help Getting Your Pages Indexed?
We hope this compilation of tips will help your pages return to the index. Search engines are not free from error. Sometimes even the most valuable blogs can be dropped. Make sure to submit links to search engines before considering your next options. If you need more information about indexing and deindexed URLs, contact us. We’re happy to help.
FAQs on Indexing
The average crawl time can take a few days to a few weeks. If the page hasn’t been indexed in four weeks, we recommend resubmitting it via Google Search Console. In our experience, some URLs require “fetching” more than once to get indexed.
Some website owners agree that newer websites take longer to index compared to older domains. Other factors, such as missing sitemaps, larger images, or duplicate content, can prevent pages from getting indexed. Ensure those pages aren’t tagged as no index, as it prevents crawl bots from finding your page.
Crawling is when search engines find new or updated content, including new websites, posts, pages, changes, and broken links. Meanwhile, Google indexing is the storage, analysis, and organization of the crawled content into the “index” or search engine database. To be eligible for organic search, a page must be crawled and indexed.
B2B Content Marketing Trends for 2023