By default, Wordpress automatically creates a robots.txt file for any site. But there are a number of nuances here. First, the user does not know where robots.txt is located in Wordpress. Secondly, he will not be able to edit it.
The standard file is not suitable for many people, so it must be edited, indicating which directories should be closed from indexing, and which should be left available for search engines. And for this you need to know everything about the robots.txt file in wordpress: where it is located, how to configure it, etc. Let's consider this issue in more detail.
We check the presence of the robots.txt file
So, the Wordpress system automatically creates a virtual robots.txt file. But there are exceptions. And before looking for it, you need to make sure that it exists.
The easiest way to check this is to add the line /robots.txt to the site address. It should look something like this:
https://wordpress-school.com.ua/robots.txt
If this file exists, a new page with approximately the following content will open:
The robots.txt file contains multiple lines, so it is considered invalid. It needs to be changed, specifying in detail what can and cannot be indexed. But it will not be possible to edit the virtual file, you need to create its physical version on the hosting.
Of course, here you need to understand in advance how and where to find robots.txt on the Wordpress site, because the new file must be located in the same place as the previous one. Otherwise, search engines will not find it and will assume that it simply does not exist (which is considered a critical error when promoting the site).
Where to find and how to work with the robots.txt file?
The robots.txt file is located in the root directory of the site. You can open it in different ways.
The first is through the hosting control panel. In this case, you need to go to the path: www/site_address.
- wp-admin;
- wp-content;
- wp-includes.
And below are various files, including robots.txt.
The second way is with the help of an FTP client installed on the computer. For example, through the Filezilla program.
- access hosting using Filezilla;
- open the folder www/site_address;
- find the robots.txt file.
By selecting it and clicking the right mouse button, you can edit it. Or create a new one if the file does not exist.
The third way is with the help of special Wordpress plugins (Yoast SEO or All in One SEO).
They perform many useful functions, including the ability to create a robots.txt file for a Wordpress site. This option is very simple and suitable for beginners. You only need to install the appropriate plugin and edit the content of the robots.txt file of the Wordpress site.
You don't need to enter the hosting control panel or install an FTP client on your computer. Plus, you won't have to figure out where to look for the file in Wordpress (the SEO plugin will create the file in the right place and save the changes).
And the last option is to consult with the managers of the company from which the website hosting was ordered.
This is an extreme measure that will help those who do not understand how to enter the control panel or install an SEO plugin. Managers will help to understand this issue and will explain in detail where to find and how to correctly configure the robots.txt file for your site.
Read also: How to create an online store on Wordpress?
Like the article? Tell your friends about it: 
