Duplicated content and pages problem

Increasing the search attractiveness of the resource includes the fight against duplicate pages that negatively affect its search ranking, lower the rank in search results, reducing the effectiveness of the Internet project – the primary tool for promoting an online business.

Why is this happening?

Imagine the situation – the owner honestly prepared unique, high-quality thematic content for his target audience: useful, reliable, up-to-date descriptions, competent teaching materials, audio, and video recordings, excellent thematic graphics, unique authoritative author’s reports in media format, video tutorials, webinars, professional video clips with brand advertising, recorded in a prestigious studio – he provided the resource with everything necessary for trust, a stable reputation, and positive buying decisions. This approach deserves to rely on traffic and dividends.

What is the omission? There is an explanation.

We increase attendance and SERPs. You get sales and pay only for real results, only for targeted clicks from search engines

Search engines are robots that look for high-quality, original, useful content and increase the rating of such web resources and pages. Uniqueness is one of the fundamental criteria for the quality of content.

How does a search engine behave if someone dishonestly copied your flawless content and published it on his website – he lowers the rating of such a page, bans the resource for plagiarism, realizing that the authorship is violated. Unfortunately, if you copy on your website and post the same text a second time, the search engine will behave in precisely the same way: you are corny yourself competing with yourself.

It is this principle of attention to Internet users and the desire to select only the most unique, fresh, useful in the first places of search results that underlie the negative attitude of search engines to duplicates.

The principle is clear, but how surprised the owners of the sites when they discover a source of low rating – duplicates inside the site! The first reaction is: we do not have duplicates; each page is unique!

What is considered duplicates

Page duplicates on a site are pages whose content wholly or partially matches the content of another page that already exists on the web.

The addresses of such pages can be almost identical.

For instance:

  • with a domain starting at www and without www,
  • with a slash in the end
  • with .php or .html at the end.

The same page with several addresses with the indicated differences is perceived as several different pages – takes in relation to each other.

There are different duplicated content

If duplicate pages belong to the same site, these are internal duplicates.

Distinguish between full and incomplete internal duplicates (clear and fuzzy).

For example:  links

  • http://site.com/about
  • http://www.site.com/about
  • http://www.site.com/about.html
  • http://www.site.com/about/

there will be utterly identical content. There will be one page; only the URL will differ. These are full duplicates.

Incomplete duplicates are pages that partially repeat the content of other pages, for example, the main block of information on the start page exactly repeats the content of the About Us page, and the links for such pages will be different. Different sections will be active in the menu, and different parts may appear in the side columns blocks. The URL for these pages is different.

The reason for the appearance of duplicates

If incomplete duplicates are the result of a conscious choice of the structure of the site’s pages, then the owner is usually not even aware of the complete duplicates.

Reasons for full takes:

  • Features cms, allowing to form synonyms at the end of the page names. Writing at the end with / or without, with .html, .htm, php – we will always open the same page.
  • Additional domain with www for one site. Pages respond to both domain names.
  • Several different names in the site structure for identical pages, if any page of the site can be linked to a menu item several times.
  • The redesign that led to the change in the URL of the pages.

Reasons for incomplete duplicated content

As a rule, the owner’s conscious desire to display one information on different pages in order to attract more attention to it and increase usability. The same texts are displayed in different blocks and duplicated in many places. Within reasonable limits, this is justified.

What are the dangers of duplicated content website SEO promotion

  • Duplicated content is undesirable for the SEO since search engines impose sanctions on such sites, send them to filters, as a result of which the page rating and the entire site are downgraded until they are removed from search results.
  • Duplicated content interferes with the promotion of page content, affecting the relevance of promoted pages. If there are several identical pages, then the search engine does not understand
  • Duplicated content reduces the uniqueness of site content: it is sprayed between all takes. Despite the uniqueness of the content, the search engine perceives the second page as nonunique in relation to the first, reduces the rating of the second, which affects the ranking (sorting sites for search results).
  • Due to duplicates, the weight of the main promoted pages is lost: it is divided between all equivalent pages.
  • Search robots spend more time indexing all pages of the site, indexing duplicates.

 

How to find an clean duplicated pages

Based on the principle of the search engines, it becomes clear that only one link should correspond to one page, and one information should be on only one page of the site. Then there will be favorable conditions for promoting the necessary pages, and search engines will be able to evaluate your content adequately. To do this, duplicates need to be found and eliminated.

How to find duplicated pages on a site

1. In the search engine you can find all the pages of the site, if you enter the site in the search field: the domain name of the site, separated by a space fragment of text. In the search results, you will find all pages with this phrase only on your website.


2. For the analysis of sites, including for the search for duplicate pages, there are special programs. For example, Xenu is convenient and straightforward for such a purpose. The principle is this: the domain name is indicated, the search for all pages is launched, a report is compiled in which broken links, errors, and duplicates are displayed.
3. Search engines to help webmasters create the most effective and sought-after sites that will be perceived as similar by search engines, suggest using their site performance analytics services – “Webmaster Tools,” which have separate sections with reports on duplicate pages, there you can also get a list of problem pages.

How to get rid of duplicate pages

There are two ways to delete duplicate pages on a site: exclude duplicates and hide them from search robots.

If we proceed from the principle of one page – one link, then all variations of the same page should be excluded: one option opens the desired page, all the others do not find the page. If the page is not found, the user is given a 404 error. For aesthetics, it is often beautifully designed or laid out on its useful site material.

But for the user, this is inconvenient – to find a page using only one of several possible options. In this case, set up a redirect (automatic redirection) of all options for writing the page to one single existing page. This is called the 301 redirects, it is set on the server in a special htaccess file, and search engines treat it quite loyally.

You can prevent search engines from indexing duplicate pages in a special robots.txt file created for search engines as instructions on what to index and what to ignore. Using this file, we can close duplicate pages from indexing.

Plus, the situation is that your key asset – content – is not stolen by attackers, you do not need to defend intellectual property rights, sue, scribble complaints and letters to Google, Yandex, Digital Millennium Copyright with evidence and screenshots from the admin panel and blog posts: enough make some internal measures to eliminate an annoying misunderstanding, and soon after reindexing you can get a return on your invested budget, marketing efforts and increase profit from business.

SEO