The Location of robots.txt

Above is an example of how the remote file structure looks in FileZilla (FTP) on a Wordpress site.

It's important that the robots.txt file is located in the root directory (top-level directory of your web server).

In more detail:

When a search bot looks for the "/robots.txt" file for URL, it strips the path component from the URL (everything from the first single slash), and puts "/robots.txt" in its place.

Example:
For...

"http://www.your-site.com/blog/index.html"

it will remove the "/blog/index.html", and replace it with "/robots.txt", so it will end up as...

"http://www.your-site.com/robots.txt"

So, as a website owner you need to put it in the correct place on your web server for that resulting URL to work. That is usually the same place where your site's main "index.html" or "index.php" welcome page resides.

NB. use all lowercase for the filename: "robots.txt", and not "Robots.TXT" or "Robots.txt"