A robots.txt is simply a small file created using notepad which the Search Engines like Google, Yahoo, Bing recommend that you put in your website.

Why?

Because they want to know which webpages you want Search Engines to index, that is to show in the organic search when someone is typing in Google for your website.

For example, if you are promoting Strip That Fat product, and you have a specifc webpage for people to download a particular report after signing to your list, you really don't want anyone just Google your website name and find your report without opting in.

Plus. if you have bonuses for your email list, you strictly don't want that page to be indexed by Google, Why, simply because if you don't do that other people that have NOT bought from you will get it in the Search results and download them.

You don't want that! Right.

So the next lesson is how to create a robots.txt file.



Join the Discussion
Write something…
Recent messages
Drackonus Premium
Hey! Had to throw some GOLD at you for this! :)
Nice reminder, and guide! Thanx, mate!
Reply
Viju Dallon Premium
Thanks Jay!

Respect,
Viju Dallon
Reply
Jessies Dream Premium
Thanks Jay!

Respect,
Viju Dallon
Reply
magistudios Premium Plus
Awesome job Viju.

Perhaps you should elaborate the How to create page and let Mac users know how to create as they don't have notepad.
Reply
Lonewolf Premium
Awesome job Viju.

Perhaps you should elaborate the How to create page and let Mac users know how to create as they don't have notepad.
Reply
Top