I’ve been struggling with a particularly difficult link cleanup project lately. On the occasion of my 100th reconsideration request (4th for this particular client), I thought it might be helpful to share five not-so-common problems that you might run into during your own link cleanup project.
[Editor’s note: Since this post was originally published, it’s received praise from two Googlers: webmaster trends analyst John Mueller and webspam head Matt Cutts! Which just goes to show how important it is that you continue reading!]
1. Don’t Block Your Pages With Robots.txt
One good solution to get rid of some links that you don’t want any more is to take the offending page away completely. This obviously only works if the links point to an interior page, because of course you can’t 404 your home page. But let’s say all your links point to a page like /spampage.html. You can take spampage.html down completely, but don’t block it with robots.txt!
The search engines have to be able to index spampage.html to be able to see that it’s not there anymore, and if you block it in robots.txt or with a “noindex” command on the page itself, the search engines won’t be able to see that it is now a 404. For more on why this is the case, see this article on how search engines work.
2. Be Careful Of Redirects
Similar to the tip above, if your page goes through a redirect before it goes to a 404, it will take longer for Google to “see” the 404. If the redirect is a 301, Google will add it to their queue of links to crawl at some future date. They may not crawl this right away. If the redirect is any other form (302, 307, 304), Google will not add it to their queue of pages to crawl. Therefore, they may never see that your destination page is a 404.
It’s common to have a 301 redirect set in your .htaccess file to redirect site.com to www.site.com or vice versa. If you want Google to see your 404 quickly and discount the links that point to it, you’ll need to turn this off and make sure that any request to the page goes directly to 404.
3. Sync Your Disavow File
If you’ve been through a reconsideration request process, you know that you rarely get accepted on your first try. You will probably need to do multiple rounds of reconsideration. This means you’ll probably need to update your disavow file a few times. Make sure that each time you update it, you do two critical things:
- Include all of the domains you’ve previously disavowed! Remember, these domains you already disavowed are probably not going to show up in Google Webmaster Tools this time around. So don’t forget about them, or you’ll be locked in a continuous round-robin of disavowing links that you already disavowed and then forgot to add. The disavow file is processed anew each time it’s accessed by the search engine.
- De-duplicate the list! While the disavow file is processed by a machine and not a person, the manual reviewer usually does take a look at it. So make sure it’s clean; otherwise, it will look like you didn’t care enough to make sure your list was valid. And since a key element of getting your reconsideration request accepted is intent (Google’s got to believe that you care, that you put forth significant effort, and that you’re not going to break their rules again), anything you can do to show how seriously you take your files is a good thing.
Side Note: Make sure your disavow file is read with no errors. When you upload it to Google, if it has errors, they’ll let you know. Fix them and resubmit until it goes through with no errors!
4. Share Your Google Spreadsheet
You already know that you must show your work! Keep a spreadsheet that shows all the domains you’re aware of. Categorize them into logical categories (keep, not found, nofollow, contacted, removed, disavow, etc.) The specific categories you use are up to you, as long as the manual reviewer can tell what you did.
But, here’s the pearl of information: when you upload the spreadsheet to Google Drive, there are two key things that could go terribly wrong:
- You might not get all the data into the spreadsheet. Google Drive is picky. It only allows you to copy and paste a certain amount of data at a time. If you go too fast, you may not realize that some of your data has not been pasted in. You may have to add rows to the bottom of the spreadsheet. Check, double check, and triple check to make sure all your data is in that spreadsheet before you submit it in your reconsideration request.
- Share it the right way! There are only two ways to share a Google drive spreadsheet so that the manual reviewers can see it. The first is to make it public, which I don’t recommend. What you really need to do is set it so that “anyone with the link can view it.” Follow these instructions exactly:
With your spreadsheet open in Google Drive, select the “Share” button at the top right:
Under sharing settings, you’ll see “Who has access.” This is normally defaulted to “Private.” You need to select “Change” and adjust the setting to “Anyone with the link.”
Copy and paste the link into your reconsideration request. If you don’t use this link and put the settings this way, the manual reviewer will not be able to access your spreadsheet.
5. Know What To Expect After Your Reconsideration Request
Within a few minutes to a few hours after you submit your reconsideration request, you’ll get an automatic email from Google Webmaster Tools that says your request was received, like the one below.
If you do not get this auto-email, your request was not received and you should resubmit. This happens more often than you’d expect, so watch for that confirmation email to make sure that you’re not waiting for no reason. In my experience, you’ll get one of the following responses between 2 and 6 weeks:
Your request was accepted, and the manual action was revoked. This obviously is what you want to see, since it means you’ve succeeded! It’s really rare to see them on the first try, though.
Your request was not accepted; you have more work to do. This response is the most common one you’ll receive. It may or may not have some example URLs that you should look at. There’s a lot of insight you can take from your examples if you get them, but that is a whole different post.
There’s one final type of response that I’ve seen personally (all of the above are my own screenshots from live accounts), and that’s what I call the “non-answer”:
This one is particularly frustrating because there’s just no information here. In my experience so far, this response generally indicates that there was a problem somewhere: a spreadsheet the manual reviewer couldn’t access, an incomplete or error-filled disavow file, or something else that went wrong.
According to Top Contributors (as recognized by Google) on the webmaster tools forum, you should not resubmit. They say this response means your request is in queue. So, my advice is to check everything to make sure it’s right, but don’t resubmit. Presumably, you’ll get a response later. In the recent case that I had, the manual penalty was removed (a couple days ago), but we haven’t received another communication from Google.
There’s one last type of response that you may get, but I haven’t seen it personally. The response says something like, “We won’t review another request for a few weeks.” I think that means that Google basically thinks you’ve been spamming them — either that you’ve done too many requests in too short a time period, or that the effort you’ve put forth is not what Google wants to see. You might get a response like this, for example, if the day after you get a manual action notice, you submit a disavow file of all your links. In other words, you haven’t made any effort to remove links or make contacts. As mentioned above, Google wants you to make measurable effort.
Hopefully this list of five not-so-common errors with reconsideration requests will be helpful to you. If you’ve been hit with a manual Google penalty, don’t go it alone. Hire someone with experience to help you. It’s hard to come up with budget in such dire circumstances, but it’s well worth it to work with someone who’s seen these results and knows what’s needed for success.
Have you seen other response types not represented here? Let us know in the comments!
Contributing authors are invited to create content for Search Engine Land and are chosen for their expertise and contribution to the search community. Our contributors work under the oversight of the editorial staff and contributions are checked for quality and relevance to our readers. The opinions they express are their own.
