Let’s see some simple examples:

Or, you can block any URLs, that start with a given string (path):

The “Allow” directive is not part of the original standard, but it is now supported by all major search engines. You can use it to specify exceptions to a disallow rule:

The aforementioned wildcard operator is supported by all major search engines and can be efficiently used to block pages when part of the given path is unknown or is a dynamic variable:

Also, if your site has any sitemaps, it’s always a good idea to include the appropriate sitemap directive(s) in the robots.txt file to specify the location of your sitemap(s):

Once you have created or edited your robots.txt file using any regular text editor, obviously, you’ll have to upload it into the root directory of your website.