And we are smoothly moving from semantics to technical optimization. The first step in this step is to define and configure the master mirror of the site.
Why is it important?
There are several reasons. Firstly, site mirrors are duplicates, which we need to get rid of by explaining search enginethat only one of them needs to be indexed. The second reason is that the links placed on the site are, as a rule, motley - someone wrote an Url with WWW, and another person indicated the address without it. As a result, the link mass is smeared between these two options. The actions described in this material are designed to completely solve such problems.
How to find the main mirror of the site?
If the site is no longer new, then first of all we need to determine which of the mirrors (with or without WWW) has been indexed. The method is quite simple - go to Yandex https://webmaster.yandex.ru/addurl.xml and, having registered the Url of our site, we get the following:
If the address you entered is the main mirror, then the result of the check will look like "The site www.site.ru has already been indexed".
Well, or the second way to determine the main mirror, which is fair for both Yandex and for - just enter the site address into the search and see the option that will be displayed on the first line.
What if the main site mirror in Yandex does not match with Google?
Better to leave the option that has taken root in Yandex, adjusting Google to fit it. According to my observations, Yandex has been experiencing the change of the main mirror much longer and much more painfully than Google. How to "fit" - will be described below.
Main site mirror: with or without WWW
There is no practical difference - choose what you like best. Out of old-school habits, I am impressed by the WWW option, but I can't help but admit that without WWW, the site address is shorter and attention is immediately focused on its name, and not on the prefix-anachronism (World Wide web) from the dawn global networkwhen it was required.
How to specify the main site mirror?
In general, this process consists of three stages. The first step is to add the host directive at the end of our file as follows:
User-agent: *
...
Host: www.site.ru
Sitemap: http://www.site.ru/sitemap.xml
This is primarily necessary for Yandex, since for Google given the directive means nothing. Then we move on to the second stage, which is different for each search engine and consists in setting up the webmaster's office.
Setting up the main mirror in Yandex
Next - "Indexing settings" - "Main mirror", where we choose the option that we prefer.
Setting up your main Google mirror
Add the site twice to google webmaster panel - that is, addresses with and without WWW, as if they were two different sites. Then, for each of them, select the desired option in the settings and save.
How to glue site mirrors?
The last step will be to write the server ones in the htaccess file, which will finally put an end to the inappropriate mirror by redirecting all its pages to the desired option.
Options + FollowSymLinks
RewriteEngine On
RewriteEngine off
RewriteCond% (HTTP_HOST) ^ www.site.ru $
RewriteRule (. *) Http://site.ru/$1
This example is true when the main mirror is without WWW. For the opposite situation, simply swap “www.site.ru” and “site.ru”. The "FilesMatch" section prohibits redirecting the robots.txt file of the secondary mirror - this is necessary for the host directive to point to the main one when the search engine addresses the non-primary alias.
That's, it seems, that's all. This small but important stage is over. Questions?
The main mirror of the site or the main domain of the site should be indicated to the search engines so that they know exactly which of the domains should be considered the main and which mirror.
Specifying the main mirror of the site is one of the points for the technical optimization of the site. This action can be done in three ways:
- By setting up a robots.txt file
- Htaccess file setup
- Setting up the appropriate sections in the tools for webmasters of the Yandex and Google search engines.
I will clarify one point. In this post, we will look at the most common case for determining the main mirror of a site - with or without WWW.
We will omit other situations that arise, for example, when a site is moved to another domain.
To be honest, the modern analytical level of search engine capabilities, in my opinion, can easily provide a solution to this problem.
But all the leading promotion experts still strongly recommend this kind of customization.
Moreover, they insist on fulfilling all three of the above points, despite the fact that the point with robots.txt is advisory in nature for search engines.
Setting up robots.txt
The robots.txt file is a text file with a set of specific directives. This file is located in the main directory of the site. Edited by the simplest editors such as "Notepad". 
The use of this file and its commands is described very well and in detail in the reference section of the Yandex Webmaster service. You can also broaden your horizons in this matter on the website robotstxt.org.ua.
In this post, we should point the search engines to the main mirror of the site. This is done using the Host directive as follows:
Please note that this directive is used only for search robot Yandex. The snippet looks something like this:
User-agent: Yandex
Crawl-delay: 4
Disallow: / cgi-bin
Disallow: / wp-admin
……………
Disallow: / tag
Disallow: / feedback
Host: site
You can check the correctness of filling in the robots.txt file in the same Yandex Webmaster. In the "Configuring Indexing" section, there is a subsection called "Robots.txt Analysis".
Cancellation of the Host directive in Yandex
The Yandex search engine announced that from March 2018, the Host directive in the robots.txt file will no longer be supported.
Is this good or bad? Rather good. After all, Google did not take this directive into account before. Now the rules for specifying the main site mirror are becoming more uniform.
Htaccess file setup
The definition of the main site mirror in the .htaccess file is carried out by setting up a 301 Redirect. 
How this can be done. Here are examples of customization for blogs on CMS WordPress.
The .htaccess file, like robots.txt, is also text file and is located in the main directory of the site. In this case, all of its directives must be followed for the entire site. If such files are found in other directories of the site, then its commands will operate only in this directory and in subdirectories subordinate to it.
Setting up the main site mirror in the webmaster tools.
To set up the main site mirror in Yandex and Google webmaster tools, you must have an account with these search engines.
Also, for this, you must confirm the rights to the site. I hope you already know how this is done.
I would like to draw your attention to one nuance. In the Google webmaster tools, you should verify the rights to both the domain with www and without www.
This is done quite simply by placing a small file in the main directory or by specifying special tags.
Setting up a mirror in Yandex
Specifying the main mirror in Yandex takes place in the "Setting up indexing" section. The subsection “Primary Mirror” gives three options to indicate. By default, the system selects - "At the discretion of the robot". 
Thus, the setting was made in old version Webmaster. In the new Yandex Webmaster, this parameter is configured somewhat differently. I think that Yandex determines the main mirror when registering a site in this service. You explicitly specify the protocol and URL of the main domain with or without www.
If necessary, the main mirror can be reconfigured in the Indexing section - the Site moving subsection.
Setting up a Google mirror
In Google Webmaster Tools, the customization option is located immediately upon entering the Site Settings section. But I will repeat once again that you will have to verify the rights to the domain twice. For a domain with www and without www.
These are the simple, but necessary steps to be taken so that search engines do not have any questions about the Main Mirror of your site or the sites of your clients.
Site mirror - a basic concept in SEO, the essence of the term is as follows: we have a site.ru site and it has several spelling options (we believe that the https protocol is connected):
- https://site.ru
- https://www.site.ru
- https://site.ru
- https://www.site.ru
Now the trick!
For a search engine, each site writing option is a separate site
That is, initially, when the site is launched, we have at least 4 copies of it, therefore duplicate pages are multiplied, and the site cannot be fully promoted.
Therefore, with basic optimization, you need to leave only one mirror (for example, using a 301 redirect). And in order for Yandex to correctly perceive the mirror we need, it must be added to Yandex Webmaster.
Add the main mirror (moving the site) to Yandex Webmaster
In the current Yandex Webmaster interface, you can specify the main mirror in the site move section (after).
Go to the section and see the following picture
Here you need to select the options for the main mirror (with https and / or www). You will first need to confirm the rights to the site.
After you add the main mirror, you need to configure it from the site side (for example, with a 301 redirect).
Here is the code that can do this (you need to place it in the .htaccess file at the root of your site). note that given code may not work on some hosting, the result of its work may be.
Options + FollowSymLinks RewriteEngine On RewriteCond% (HTTP_HOST) ^ www.site.ru $ RewriteRule ^ (. *) $ Https://site.ru/$1
Options + FollowSymLinks RewriteEngine On RewriteCond% (HTTP_HOST) ^ www. site. ru $ [NC] RewriteRule ^ (. *) $ Http: //site.ru/$1 |
This is a variant of a domain redirect from www to without www (you can also add https here instead of http and then your site will work on the https protocol)
We have released a new book “Content Marketing in social networks: How to get into the head of subscribers and fall in love with your brand. "
Site mirror - partial or full duplicate of the Internet project
The term "site mirror" most often means two copies of the same resource with and without the "www" prefix. Search engines treat yoursite.ru and www.yoursite.ru as two different domains... Accordingly, link ranking and ICS / PR indicators will be distributed to both addresses. To eliminate unnecessary duplication, the site mirrors are glued together.
More videos on our channel - learn internet marketing with SEMANTICA
A site mirror can mean a complete copy of an Internet resource that has a separate domain and is located on another server. A webmaster who knows how to set up a site mirror can create a duplicate for security purposes. If main internet the project will be temporarily unavailable, then traffic is redirected to a functioning server.
How to specify the main site mirror
Before starting search engine promotion, you need to check the main site mirror. PS robots do not always correctly determine the main domain. Therefore, the webmaster must know how to specify the main site mirror on his own.
You can pass your primary domain information to Google using webmaster tools. To do this, go to Search Console, add a resource and confirm the rights to it. Then you need to go to the settings (gear icon) and indicate the display of the main domain with or without the prefix "www".
Would you like to know how to determine the main site mirror for the Yandex robot? For this search engine, the site mirror is configured through robots.txt. Add the directive to it
Host: www.yoursite.ru (the main domain will be displayed with a prefix)
Host: yoursite.ru (in this case the domain without www will become the main mirror)
Important! Instead of "yoursite.ru" you should substitute the domain name of your Internet project.
The robots.txt file is a standard text Document, usually located in the root folderwebsite. It can be edited using the notepad application.
If everything is done correctly, then search engines will be able to glue site mirrors much faster, which will have a positive impact on promotion.
Checking the site mirror with online services
A webmaster who wants to know how to find a mirror of a site can use the recipdonor.com service from the RDS bar. In the RU domain zone, the search for site mirrors is performed using the seobudget.ru/mirrorlist/ service.
How to make a site mirror (backup take)
Why a website mirror might be needed by a web project owner? The need for a copy of the site on another domain may be due to the following reasons:
- Data protection from server failure;
- Security stable work internet project;
- The need to bypass the blocking, legal restrictions of certain countries;
- Removing the site from the sanctions of search engines.
Are you wondering how to create a site mirror by linking multiple domain names to a web project? The support service of the hosting company on whose server the site is located can help with this procedure.
The creation of a site mirror can be done using programs that can copy the structure and html-code of an Internet project. The most popular mirroring software is CVSup, wget, HT Track.
Hello everyone! Today is a detailed post on how I managed change the main mirror the blog and at the same time not lose positions, traffic, income from contextual advertising.
For a long time I was tormented by one question - how to get rid of the stupid one, which has been dragging along behind me for more than three years. Perhaps the age of the domain was the main reason why I did not dare to re-paste the domain earlier.
It so happened that from the very first days of the blog's existence, Yandex began to index the blog with www, at that time I decided not to interfere with this and put up with it.
Later, I consoled myself more than once, they say, what's the difference, the main thing is that the pages are in search and bring visitors. Was engaged in promotion,. And the more time passed, the more my hands itched to change the main mirror. In the end, my patience ran out and no matter what I decided. Today I will share my experience and tell you how it was.
Correct procedure for gluing mirrors
Throughout the entire time, I kept notes, which I constantly carried with me and wrote down all significant events in chronological order. So, at the beginning of September, my goal is to change the main domain mirror and remove www from the address.
09/06/2013 I am contacting Yandex.Webmaster support with a letter in which I describe the situation and ask for advice. I asked questions about how to properly re-glue the domain and so that the pages do not fall out of the index. Since my blog, I thought it would not be superfluous to contact his support and asked how the change of the main mirror would affect the presence in the catalog. Soon I got the answer:
“In this case, you need to change the main mirror using the Host directive, wait for the merging process to complete, then the address in the Catalog will change automatically. The TIC will also be moved to the address of the new main mirror and restored at this address after the TIC recount closest to the plywood. "
09.09.2013 I received a response from Platon Shchukin to the first letter:
“You shouldn't use a redirect to change the main mirror. Currently, the sites you specified are united in a group of mirrors, in which the main mirror is the domain with www according to the current instructions. In the Yandex search engine, only the pages of the main mirror are indexed and participate in the search ... the site, then first you need to remove it. Leave a redirect to current form and you will not be able to specify a domain without www as the main mirror, since then the instructions you specify to the main mirror will contradict each other. If you simply change the direction of the redirect, this will ultimately lead to a change in the main mirror, but in this case it should be understood that the robot will not index the pages of the current main mirror, but will follow the redirect. In turn, the target of the redirect will also not be indexed, since it is a non-main mirror that does not participate in the search.
Therefore, if you want your site to be indexed to the address of the current main mirror before merging the mirrors, the installed redirect should be removed. To change the main mirror, it is better to use the Host directive in the robots.txt files of all mirrors, and also select the required domain in the corresponding Yandex.Webmaster section. This will be quite enough, while you will avoid problems with site indexing due to redirection to a non-main mirror.
Thus, the most optimal way to get rid of the www prefix is \u200b\u200bto remove the redirect, change the main mirror using the Host directive and in Yandex.Webmaster, and then, when the main mirror changes, return the redirect if necessary. As for Yandex.Katalog and Yandex.Metrica, the data in them will be updated automatically, you won't have to add anything again. Please note that the process of gluing mirrors is automatic and can take several weeks, unfortunately it cannot be accelerated. "
On March 20, 2018 Yandex officially stopped supporting the Host directive. You can remove it from robots.txt, and if you leave it, the robot simply ignores it.
A very detailed answer, for which many thanks to the support of Yandex! Literally right away I started making changes:
- Changed the main mirror in the Yandex.Webmaster panel, specifying the address without www;
- Changed the Host directive for Yandex;
- Changed main domain in Google Webmaster Panel;
- Removed redirect from .htaccess file;
- Added new ones to the Google and Yandex webmasters panel;
- Fixed links in the main menu and in the sidebar so that search bots notice the changes as soon as possible.
Screenshot from the Yandex.Webmaster panel. In the Indexing Settings section, there is a Main Mirror item - here you need to specify your preferred domain:
Screenshot from Google's Webmaster Toolbar. The site settings are hidden under the gear. Here you need to specify the main domain:
The process has started! Along the way, I wrote down the main indicators of the blog for history:
- TIC 130 (last update 02.07.13)
- PR 2 (last update 04.02.13)
- Pages indexed in Google (according to RDS Bar) - 187
- Indexed pages in Yandex - 241
09/10/2013 Yesterday I undoubtedly checked the server response of both mirrors and it contained the code 200, as Platon Shchukin advised to do. But this morning, having come to work, I decided to check again that everything was in order and was surprised. Apparently on home computer the server responded, and now it gives out 301 codes when accessing the old address from www. As it turned out, WordPress has its own redirect, which takes into account the settings of the site address in the admin panel.
It was necessary to urgently find a solution to how to disable this redirection. One of the foreign forums came to the rescue, where a solution was found, by adding to the wp-config.php file following lines:
| 1 2 | define ("WP_HOME", "http: //". $ _SERVER ["HTTP_HOST"]); define ("WP_SITEURL", "http: //". $ _SERVER ["HTTP_HOST"]); |
define ("WP_HOME", "http: //". $ _SERVER ["HTTP_HOST"]); define ("WP_SITEURL", "http: //". $ _SERVER ["HTTP_HOST"]);
Let me add a few words about why you need to remove the redirect. The reason is that Yandex must have access to both mirrors. If there is a redirection from one mirror to another, then how can a mirroring robot determine that sites are identical to each other? Exactly…
This is the main one, as a result of which, in the process of pasting, the site completely drops out of the index long time does not participate in the search. And he does not index the purpose of the redirect due to the fact that only the main mirror can be in the results of the domestic search engine, and the plywood has not yet occurred. Vicious circle.
Therefore, in order to avoid such a fate, remember: never put a redirect to a new mirror and remove the existing one before the complete gluing / re-pasting of mirrors in Yandex, about which you will be notified by a message!
Then I proceeded to edit all my links to home pagefound in posts and comments. At first I decided to do everything manually, but I realized that I could not cope with such a volume. I took a chance and took advantage SQL query to the database:
| 1 2 | UPDATE wp_comments SET comment_author_url \u003d REPLACE (comment_author_url, "www.site", "site") |
UPDATE wp_comments SET comment_author_url \u003d REPLACE (comment_author_url, "www..ru")
09/13/2013 Update Yandex issue. No movement in the mirrors is observed. There is also silence in Google, although I reread many times on the forums that it reacts to changes almost instantly and literally the next day you can find new addresses in the search. I continue the agonizing wait.
09/19/2013 Another - everything is unchanged. I am convinced that I did the right thing by removing the redirect. Otherwise, the pages would have dropped out of search long ago, and with them the traffic would have seriously sagged and the income would have dropped to zero.
09/20/2013 Information about internal and external links for the new domain began to appear in Google Webmasters Tools, the data was previously unavailable.
09/26/2013 And again the issue update. The Yandex index contains several duplicate pages with and without www. The same day brought AP TIC, which added ten, became 140. In Yandex Catalog, the domain was glued to the new main one - at last, the progress began.
09/27/2013 Service xseo .. In the evening of the same day, another service cy-pr.com also confirmed the gluing.
10/1/2013 Yandex made a great gift for my birthday in the form of an update search base, and changed the main mirror, Hurrah! At exactly 8 am a letter from Yandex.Webmaster arrived, or rather, there were two letters:
- The main mirror has changed. The letter informed that www..ru and therefore it was automatically replaced in the list of sites.
- The rights to the site have been successfully verified. Everything is simple here, since the main mirror has changed, the site address has also changed. And since the confirmation file was already on the server, no additional action was required on my part.
10/02/2013 Google is in no hurry to change the main domain, its tool in the webmaster's panel does not seem to be so effective, or he dislikes me ... However, these feelings are mutual with us. I put it, and delete the previously added lines in the wp-config.php file. On Yandex, the mirrors are glued together, so it's no longer scary for indexing, and Google normally treats 301 redirects.
10/7/2013 Really? Xseo.in shows Google stitching. Although it has approximately 50/50 pages in and without www, many pages are duplicated. Judging by the crawl charts, Googlebot is actively surfing the blog. The server logs show that the robot first loads the page at the old address, receiving 301 responses, and then picks up the redirect target with the code 200.
The blog traffic fell by 50%, so we arrived, something that was so much feared happened. Although it had been decreasing from the first days of my experiment, it was happening smoothly, and on that day I saw a sharp decline. Hoping for the best, I continue to wait and follow the process, they say that time heals.
10/10/2013 People say the truth, time really works wonders! On this day, the next update of the Yandex search database took place. Everything is back to normal, I see traffic growth by about 30%. Oddly enough, but after a month Google still cannot give birth and finally change the main domain. Didn't expect this from him.
10/12/2013 Hmm ... however! A month later I publish for the first time new entry (before that I did not publish anything pending gluing) and lo and behold! By the evening, all pages containing the prefix disappeared from the Google search results.
Well, I finally got myself together and completely laid out my story with all the details. Today I can say with confidence that the replacement of the main mirror was successful, and I am very pleased with the result. I hope my experience will help many of you! 🙂
