Harmful bots we block

In addition to good bots (user-agents) , such as GoogleBot, which scan the site for indexing in search engines, there are also malicious bots whose purpose is to collect statistics on the site for third parties, parse prices for goods and services, search for hidden pages, vulnerabilities, etc.

Malicious bots usually do not carry a payload for site owners, but they can cause many problems, such as generating an unreasonable load on the hosting account and spamming the site, so in order to prevent and anticipate such situations, the virtual hosting control panel blocks malicious bots by default to the site.

If necessary, use the following steps to unblock or block a malicious bot that is not on the default list on the site:

1. Open the cp.cityhost.ua control panel and go to hosting management.

2. Go to the settings of the desired site and click the Security button

3. In the Blocked user-agents section, click the Edit button

4. In the window that opens, you can manage the list of bots that will not have access to the site.

How to add a bot to the block list

You can add your own bot by clicking on the field at the end of the list, then enter the name of the bot (its user-agent) and apply the changes with the Save button. From this moment on, the bot does NOT have access to the site, you can make sure of this by executing the curl command from the example below.

The character case does not play a role when writing - writing testBot and TeStBoT will work the same. Allowed characters Az 0-9 / . , ( ) : ; + - (пробіл) . The correct bot user-agent can be obtained from the visit log of your site, or at user-agents.net/bots

How to remove a bot from the block list

Opposite the name (user-agent) of the bot, click on the cross icon and apply the changes with the Save button. From this moment on, the bot HAS access to the site, you can make sure of this by executing the curl command from the example below.

How to restore the default bot list

If you have accidentally or intentionally cleared the bot list, you can restore the default values and enable bot blocking by clicking the "Restore Defaults" button.

How to check the lock status

You can check the blocking status of a specific bot (user-agent) by executing the curl command through any terminal, for example, in WEB-SSH on the hosting.

Run the following command after substituting your own data:

curl -I "https://domain.zone" -A "user-agent"

https://domain.zone ? your site specifying the protocol
user-agent ? bot ID

If the bot is blocked, curl: (56) Recv failure: Connection reset by peer or curl: (92) HTTP/2 stream 0 was not closed cleanly will be returned. If the bot is not blocked, a response like HTTP/1.1 200 OK or any other positive status will be returned.

Please note

Monobank can initiate requests to your website using the go-http-client user-agent, which is blocked by default. In order to receive requests from Monobank, you should remove this user-agent from the blocked list.