Google’s Disavow tool: when and how to use it

Written by Fernando Maciá

Google Disavow Tool: a consequence of Google Penguin

Google announced the implementation of its algorithm update called Penguin at the end of April 2012. This update focused, according to the search engine, on detecting websites that were violating Google’s quality guidelines for webmasters. Two fundamental aspects were targeted by Google Penguin: over-optimized websites (keyword density abuse, for example) and websites with artificial links (link farms, low quality directories, etc.).

While the over-optimization of a website is an aspect related to the relevance on page and is therefore entirely within our control (we can increase or decrease the keyword density by simply varying the wording of our content), as far as links are concerned, the possibility of a penalty for artificial links opened the door to “linking” actions.negative seo“. That is, a competitor could try to harm the positioning of my website simply by generating links that could be detected by Google as artificial links. For example, links with keyword as anchor text pointing to the home page, site-wide links (in blogrolls, in footers, etc.), links from websites that are not consistent with our theme or from low quality websites, such as certain directories, etc.

The possibility that competitors could take advantage of Penguin to damage another website’s ranking prompted Google to announce, just a few months later – October 2012 – the implementation of a tool to selectively disavow inbound links to a website: Google Disavow Tool. At least in theory, with Google Disavow Tool, webmasters could disavow or “disavow” those links that have been detected by Google as artificial or manipulative and thus avoid a penalty.

How to use Google Disavow Tool

Following the implementation of Penguin, Google began sending artificial link messages through Google Webmaster Tools and, since August 2013, it has added to this tool a specific section called Manual Actions as a subsection of Search traffic with which to warn of the identification of potential problems both at a global level – affecting the entire website – and at a partial level – affecting only certain sections of content.

However, Matt Cutts already indicated in this video that there is no problem in using Disavow Tool even if we have not received any manipulative link alert from Google, if we detect unnatural links or that we may be subject to negative SEO actions.

Google’s recommended path for Webmasters to disavow links includes the following steps:

1. Identify all suspicious links: this means not only looking at the links displayed by Google in your message or in the Manual actions Google Webmaster Tools, but also to identify other similar links that could also be identified – by their location on the page, by the domain of origin, by their anchor text, etc. – as artificial or manipulative links. The starting data for identifying these links can be found in section Search traffic > Links to your site in Google Webmaster Tools and in the New from MajesticSEO.
2. Try to remove all suspicious links: this means removing links that we have generated ourselves as well as links that, although created by others, meet any of the criteria used by Google to classify a link as manipulative. To do so, we should try to identify the webmasters responsible for those websites and request that they remove those links. And only after all this…
3. Generate a link disavow file: which we would then send to Google via its Disavow Tool. It is a simple .txt text file encoded as UTF-8 or 7-bit ASCII that we can generate with TextEdit (Mac) or NotePad (Windows). Later we will see exactly what the syntax of this file is.
4. Upload the file with Disavow Tool: easy, we go to Google Disavow Tool, select our domain, select our deauthorization file and upload it.
5. Please note: firstly, Google warns that it will take some time to process the link disavowal file we have submitted and apply the consequent changes to the index and ranking of the site; secondly, we can upload successive versions of link disavowal files. Each new version will replace the previously uploaded version. Finally, the disavowed links will still show up as Links to your site from Google Webmaster Tools (which does not mean that Google is ignoring our disavow file).
6. Request review: once all of the above steps have been completed, we can send a review request from the corresponding button in the section Search traffic > Manual actions Google Webmaster Tools in which we must include all the relevant information regarding the steps to identify problematic links and remove them.

Whether or not to use this tool has been written about extensively. Sujan Patel argues against its use in this post and there are also numerous threads on the Google forums about the effectiveness of its use. Although it is true that, if misused, this tool can also damage your positioning, in cases such as the one described below, we believe it is the best route to follow.

In any case, it is worth taking into account the recommendations and warnings contained in this document. Razvan Gavrilas in this great post, especially related to the risk of globally disallowing links at the domain level, gathering all the relevant information to write a complete reconsideration request as well as Marie’s advice. Haynes in this post about things that are usually ignored about Disavow ToolFor example, that disavowed links can be re-activated, that there is a size limit (2MB) for a disavow file, or that disavowing a link to a redirected destination usually has no effect (Marie Haynes recommends in this case to include both URLs: that of the link source and that of the 301 redirect source).

Using Google Disavow Tool in practice

Since telling it is not the same as living it, we are going to share how we detected an abnormal pattern of link growth towards our domain in July and how we have used Google Disavow Tool to try to avoid that these suspiciously artificial links can damage the positioning of our domain.

First notice of Manual Actions

In April 2013, we received from Google Webmaster Tools a Manual Actions notice > Partial Matches, where Google alerted us to the detection of links “artificial, misleading or manipulative links that direct to pages on this site”. The links in question referred, effectively, to links included within some of our posts that had been replicated on other websites. In this case, the posts were published on press release redissemination websites and dated back to 2004-05 no less!!!!

Although in some cases it was complicated, we managed to regain access to those sites to remove those links (by the way, many of these links at the time of that alert were already links pointing to 301 redirected URLs, as they were articles published under older versions of our website).

Removal and/or modification of links

Given that the warning referred to Partial Matches and that we had not detected any drop in traffic or positioning, we decided to limit our response to trying to remove the links that were appearing in the alert. And indeed, once these first links were removed, others appeared. In general, they corresponded to links of one of these types:

1. Links in old articles redistributed or plagiarized several years ago: the first case refers to posts that we actively redistributed in the past (2003 until perhaps 2008-09) through news redissemination portals, articles and press releases with little or no modification in their wording. Many of these portals had feeds that fed a number of other websites, so these articles ended up appearing on many other domains. The links contained in these articles were absolute links so that, if they were not modified by the webmaster, each new publication brought new links to our domain with very relevant anchor text and that, at that time, positively influenced our positioning.
   Along with these republishers, we also detected a large number of sites that simply copied and pasted our content, keeping the links in many cases. As these links have appeared in Manual Actions, we have been modifying them, removing them or requesting their removal, so far with success.
2. Site-wide links: these were links published in blogroll sections or in footers. In the first case, these were blogs of friends, colleagues, students, etc. that had included a link to our domain. In general, links were detected by Google as artificial when, in addition to being site-wide (being on all pages), they included a relevant anchor-text to the root of our domain. For example, links were detected from blogrolls with anchors such as “Search Engine Optimization”, but not when the anchor was “Human Level Communications”. As these links appeared in Partial Coincidences, we contacted by e-mail the person in charge of each web site so that they could eliminate or modify these links.
   In the second case, the links in the footer corresponded to some of the first websites developed by Human Level Communications and in which a credit line of the type “Website developed by Human Level…” with a link to our domain had been included, as was customary at the time.
3. Links in directories: the last type of “rare” links corresponds to links included in directories that were not created by us. Especially if they are links with optimized anchor text. Of these, in some cases it has been possible to identify the person responsible for the directory in order to request its elimination, while in other cases it has not.

From the first Manual Actions warning, we intensified the monitoring of inbound links to our domain, mainly with two tools: Google Webmaster Tools, through its Search Traffic section > Links to your site, and MajesticSEO.

Artificial link alert

It was precisely while reviewing inbound links in MajesticSEO that we came across the following graph in July 2014. An abnormal rise in both the Fresh and Historical indices:

On a more or less sustained pattern of new link discovery over the last few months, we suddenly detected from mid-June onwards a sudden increase in the number of new inbound links discovered each day. Had the popularity of our domain suddenly skyrocketed? Could we be being subjected to some negative SEO tactics? Or, what really counts in the end, could these new inbound links benefit us or hurt our positioning?

Link source domain identification

Reviewing which domains the new links came from, we discovered many legitimate domains that we have reason to thank for their links (thank you, María José Cachón), but also several directories that caught our attention. Among them:

• links-directory.com
• link-links.org
• web-directory.com
• informaticaautonomos.com
• directorioinformatica.com
• bigbozz.com
• Etc.

After visiting the different domains it became clear that:

• Some of the directories were developed with the same tool and practically the same template. The contact details confirmed that they did indeed belong to the same person and fortunately there was a contact telephone number. The main problem with these domains was an over-optimized anchor text (search engine optimization). An excessively high number of links with this anchor text pointing to the root of the domain could penalize it just for that search:
  
• Two more domains were also related to each other. In the domains themselves there is no owner’s name, email or phone number, only a contact form. So we did a quick Whois which revealed the name of the registrant of both domains and even his cell phone number. The main problem, anchor text with strange changes in an equally strange environment (see the expression “it has been working for 12.2 eyelets and this implies fearlessness for the lucky ones”) of the“scrape & spin” type that black hats like so much:
  
• As for Bigbozz, it is the Spanish version of a directory network with presence in several countries. There were no contact details and the Whois search only revealed the name of the owner, but no email or phone number.

Along with these directories, we also identified a domain (Dragonmoondesign.com) that had plagiarized an old article with absolute links and was cited as a sample of suspicious links in the Google Webmaster Tools Manual Actions report.

Link removal request

As recommended by Google, on August 1, we performed the following actions:

• We contacted the managers of the first three directories by telephone and they were very receptive and willing to immediately respond to our request to unsubscribe.
• In the case of InformaticaAutonomos and DirectorioInformatica, we also contacted their owner, who seems to have been quite surprised by our call. He was willing to remove the links “when he had time” but a month later the links still appear in the directory.
• We sent a message to DragonMoonDesign’s contact email also requesting the deletion of our domain. To date, we have not received any response.
• In the rest of the directories or domains, we did not obtain contact data or response to the request.

Disavow file generation

On August 19, and after checking the appearance of the BigBozz directory in Links to your site in Google Webmaster Tools, we decided to submit an updated disavow file including all these directory links.

The file is generated UTF-8 encoded using the TextEdit editor on a Mac, with the following content:

#Unsolicited links published in directories. Owner (xxxxxxxxxxx) contacted on August 1, 2014 by telephone. I requested the removal of all links to humanlevel.com. Deleted links confirmed on August 8, 2014

domain:links-directory.com

domain:link-links.org

domain:web-directory.com

#Unsolicited links published in directories. Owner (xxxxxxxxxxx) contacted on August 1, 2014 by telephone. I requested the removal of all links to humanlevel.com. He said he would delete them when he had time. On August 19 the links are still active

domain:informaticaautonomos.com

domain:directorioinformatica.com

#Unable to contact owner (xxxxxxxxxxx). There is no contact information or phone number in the domain registrations or on the web.

domain:bigbozz.com

#Contacted support to remove all links from this URL but no response. On August 19 the links are still active

http://www.dragonmoondesigns.com/indexable_websites_search_engine_ready_born_to_compete_and_win.php

Etc.

As you can see, the syntax of the Disavow file is very simple. Just be sure to include a hash (#) before the comment lines. Include independently each URL where the link pointing to our content that we want to disavow is located, or disavow in block all the outgoing links of a domain starting the line with the word “domain” followed by a colon and the domain to block. Once the file has been checked, we save it in UTF-8 format. When disavowing entire domains, we must be careful not to block legitimate links that may be providing value to our domain.

It is worth noting that, according to John Mueller, the comments are not read by Google supervisors but the file is processed automatically. We should, therefore, include in the comments only the information that for the purposes of monitoring our actions it is convenient for us to include. Basically, what we have done and when.

Although we managed to remove the links from the first three directories, we decided for security reasons to include them in the link disavowal file as well.

From the Disavow tool, all we have to do is select the domain to which we want to apply the deauthorization file, select the file itself on our hard drive and confirm. The tool loads the file and tells us if it is correct. We will also receive a message asking for confirmation that we really want to apply this deauthorization file on our domain. The message will reach the administrators of the Google Webmaster Tools account and can also be viewed in the Google Webmaster Tools Messages section of the site itself.

Request a review in Google Webmaster Tools

Once our link disavowal file has been submitted to Google, we have decided to wait and see what effect it will have. As of the date of publication of this post (August 25), we found that the DragonMoonDesign URL is no longer listed as a suspicious link sample in the Manual actions Google Webmaster Tools, although it now generically indicates that “Some inbound links” (without showing any in particular) still seem suspicious to it.

From now on, we will allow some time for the Google Webmaster Tools Links to your site data to be updated and we will recheck all listed inbound links to find out which ones may be detected by Google as manipulative.

We will continue to periodically check the MajesticSEO data to discover as soon as possible the new inbound links to our domain, checking the TrustFlow they provide, the anchor text used and the thematic classification of the source domain.

We will consecutively expand the number of domains and/or URLs in our Disavow file, including comments on when we add which URL or domain. We will use this information to complete the Review Request only when, at least on our part, we are confident that we have successfully removed all suspicious links.

How harmful can these inbound links be?

According to the consequences Penguin has had on other domains, the effect of a detection of artificial links with over-optimized anchor text usually harms the ranking of the website precisely for the keywords used as anchor text. We must, therefore, scrupulously and frequently check the anchor text of inbound links which could either be detrimental to us if they focus excessively on a specific keyword, or they could be detrimental to us if they deviate excessively from our positioning objective, as occurred in the two examples we have seen.

Some of the links of these directories correspond to the first type while those coming from DirectorioInformatica could be assimilated to the second type. This directory seems to work with some kind of robot that varies the landing pages and the anchor text pointing to them, so we see it as potentially harmful.

Given the potential for using links to negatively affect the positioning of other websites, it is worth keeping a close eye on new links pointing to your site to check that they meet the quality criteria required by Google. We’ve told you how we’re doing it at Human Level Communications, but we’d love it if you’d share how you do it and with what results.