Posts
44
Questions
29
I want to update my robots.txt to stop Google unnecessarily crawling /feed/ and to remove mentions of /feed/ from showing up in GSC.
I can't find the robots.txt using Filezi
it is normally a virtual file - some seo plugins will create one if you tell them to...and others have an editor....
all you need do is to create a text only file and place it in the root directory...
in the ftp you maybe - as i was - suprised that all the docs are under the http directory rather than the https - they still do a redirect on a lot of sites and this can cause issues occasionally...
but if you see index.php, wp-content, wp-admin, and wp-includes you should be in the right directory.
getting it wrong can crash your site, just as a warning....
Thanks for that info, Phil.
The thing is I use Rank Math which (as do other similar tools) provides a method for creating a robots.txt which I have updated.
This is what I've not figured out yet...
My Rank Math robots.txt is not working. Research implies that I need to delete the existing robots.txt from the root for the Rank Math one to work. But I can't find the old one to get rid of it. I can see what is in place when I render the robots.txt, but have no idea where it is rendered from...
;-)
Richard
i do wonder whether it is a script that wa have inserted as all the siterubix sites are now noindex and can't have a robots.txt added...
it maybe worth installing the robots.txt editor plugin as this can append info to the current one or has an option to overwrite ....
Thanks for that thought, Phil. I'll check it out. In the meantime I've contacted Rank Math support.
;-)
Richard
Hey Richard,
All you need is your FTP access to the site where the robot.txt file would be found, if the site has one.
Hope you find this helpful.
Thank you Trish.
I use FileZilla for that and can’t find the robots.txt. It’s time to contact Site Support.
;-)
Richard
Thank you Trish.
My reply to Phil (feigner) probably describes my dilemma best.
;-)
Richard
First, download a copy of your existing robots.txt file from your website. You can do this in a few ways:
Navigate to your robots.txt file directly (e.g., example.com/robots.txt) and copy its contents into a new text file on your computer.
Use a tool like cURL to download an actual copy of your robots.txt file. For example:
curl https://example.com/robots.txt -o robots.txt
Alternatively, you can use the robots.txt report in Google Search Console to copy the content of your robots.txt file and paste it into a local file on your computer.
Edit Your Robots.txt File:
Open the downloaded robots.txt file in a text editor.
Make the necessary edits to the rules. In your case, you want to prevent Google from crawling /feed/. You can achieve this by adding the following lines to your robots.txt file:
User-agent: *
Disallow: /feed/
Ensure that you use the correct syntax and save the file with UTF-8 encoding.
Upload the Updated Robots.txt File:
Upload your new robots.txt file to the root directory of your website. The process for uploading files depends on your hosting platform and server.
If you don’t have permission to upload files to the root directory, contact your domain manager or website administrator. They can make the necessary changes for you.
Refresh Google’s Robots.txt Cache:
Google’s crawlers automatically notice changes to your robots.txt file during their crawling process. The cached version is updated approximately every 24 hours.
If you need to update the cache faster, use the “Request a recrawl” function in the robots.txt report within Google Search Console
Thank you for your comprehensive reply Oscar.
I know most of this already... but nonetheless your reply is appreciated.
However, it seems that I will need to get Site Support to handle this since although I can copy and paste into a new text file I cannot upload via Filezilla. because I have no idea where to upload it within the folder structure.
;-)
Richard
Hi Richard it is a long time since I have changed a robots.txt. From memory this applies
The "robots.txt" file should be placed in the root directory of your website, which is the same directory where you can find the "wp-admin", "wp-content", and "wp-includes" folders.
For example, if your WordPress site is installed in the root directory of your domain (e.g., "https://www.example.com/"), the "robots.txt" file should be placed directly in that root directory:
/
├── index.php
├── wp-admin/
├── wp-content/
├── wp-includes/
├── robots.txt
└── ...
If your WordPress site is installed in a subdirectory (e.g., "https://www.example.com/wordpress/"), the "robots.txt" file should be placed in that subdirectory:
/wordpress/
├── index.php
├── wp-admin/
├── wp-content/
├── wp-includes/
├── robots.txt
└── ...
By placing the "robots.txt" file in the root directory of your WordPress installation, search engine crawlers and other bots will be able to easily locate and access the file to determine which pages or sections of your site should be crawled or ignored.
Thank you Catherine Unfortunately there is no robots.txt in the root directory.
;-)
Richard
See more comments
I want to update my robots.txt to stop Google unnecessarily crawling /feed/ and to remove mentions of /feed/ from showing up in GSC.
I can't find the robots.txt using Filezi
it is normally a virtual file - some seo plugins will create one if you tell them to...and others have an editor....
all you need do is to create a text only file and place it in the root directory...
in the ftp you maybe - as i was - suprised that all the docs are under the http directory rather than the https - they still do a redirect on a lot of sites and this can cause issues occasionally...
but if you see index.php, wp-content, wp-admin, and wp-includes you should be in the right directory.
getting it wrong can crash your site, just as a warning....
Thanks for that info, Phil.
The thing is I use Rank Math which (as do other similar tools) provides a method for creating a robots.txt which I have updated.
This is what I've not figured out yet...
My Rank Math robots.txt is not working. Research implies that I need to delete the existing robots.txt from the root for the Rank Math one to work. But I can't find the old one to get rid of it. I can see what is in place when I render the robots.txt, but have no idea where it is rendered from...
;-)
Richard
i do wonder whether it is a script that wa have inserted as all the siterubix sites are now noindex and can't have a robots.txt added...
it maybe worth installing the robots.txt editor plugin as this can append info to the current one or has an option to overwrite ....
Thanks for that thought, Phil. I'll check it out. In the meantime I've contacted Rank Math support.
;-)
Richard
Hey Richard,
All you need is your FTP access to the site where the robot.txt file would be found, if the site has one.
Hope you find this helpful.
Thank you Trish.
I use FileZilla for that and can’t find the robots.txt. It’s time to contact Site Support.
;-)
Richard
Thank you Trish.
My reply to Phil (feigner) probably describes my dilemma best.
;-)
Richard
First, download a copy of your existing robots.txt file from your website. You can do this in a few ways:
Navigate to your robots.txt file directly (e.g., example.com/robots.txt) and copy its contents into a new text file on your computer.
Use a tool like cURL to download an actual copy of your robots.txt file. For example:
curl https://example.com/robots.txt -o robots.txt
Alternatively, you can use the robots.txt report in Google Search Console to copy the content of your robots.txt file and paste it into a local file on your computer.
Edit Your Robots.txt File:
Open the downloaded robots.txt file in a text editor.
Make the necessary edits to the rules. In your case, you want to prevent Google from crawling /feed/. You can achieve this by adding the following lines to your robots.txt file:
User-agent: *
Disallow: /feed/
Ensure that you use the correct syntax and save the file with UTF-8 encoding.
Upload the Updated Robots.txt File:
Upload your new robots.txt file to the root directory of your website. The process for uploading files depends on your hosting platform and server.
If you don’t have permission to upload files to the root directory, contact your domain manager or website administrator. They can make the necessary changes for you.
Refresh Google’s Robots.txt Cache:
Google’s crawlers automatically notice changes to your robots.txt file during their crawling process. The cached version is updated approximately every 24 hours.
If you need to update the cache faster, use the “Request a recrawl” function in the robots.txt report within Google Search Console
Thank you for your comprehensive reply Oscar.
I know most of this already... but nonetheless your reply is appreciated.
However, it seems that I will need to get Site Support to handle this since although I can copy and paste into a new text file I cannot upload via Filezilla. because I have no idea where to upload it within the folder structure.
;-)
Richard
Hi Richard it is a long time since I have changed a robots.txt. From memory this applies
The "robots.txt" file should be placed in the root directory of your website, which is the same directory where you can find the "wp-admin", "wp-content", and "wp-includes" folders.
For example, if your WordPress site is installed in the root directory of your domain (e.g., "https://www.example.com/"), the "robots.txt" file should be placed directly in that root directory:
/
├── index.php
├── wp-admin/
├── wp-content/
├── wp-includes/
├── robots.txt
└── ...
If your WordPress site is installed in a subdirectory (e.g., "https://www.example.com/wordpress/"), the "robots.txt" file should be placed in that subdirectory:
/wordpress/
├── index.php
├── wp-admin/
├── wp-content/
├── wp-includes/
├── robots.txt
└── ...
By placing the "robots.txt" file in the root directory of your WordPress installation, search engine crawlers and other bots will be able to easily locate and access the file to determine which pages or sections of your site should be crawled or ignored.
Thank you Catherine Unfortunately there is no robots.txt in the root directory.
;-)
Richard
See more comments
If you have, do you have any recommendations for using AI to generate videos from posts?
Thank you
;-)
Richard
Great question.
synthesia .io, invideo.io and Canva are the ones I tried.
Thanks for sharing guys.
Mel B
I have the one that does blog posts to ai videos is from
pictory. ai
videotoblog.ai (reverse videos to blogs)
videogen.io
veed.io
Canva Pro does AI videos too
That the one I know of, I tried it a few times, not bad! I like it better than the others that Abie mentioned.
I have that bookmarked, too, Brenda :) When I did a search it was hiding lol Good job, you have it
Awesome. I didn't bookmark them. Just the ones I already started to use during the free trials.
I had an adapter failure last night... I lost broadband for a while, lol - it was driving me up the wall. I could not figure out how to fix it, so I kept rebooting and punching the 'F' keys on top... Whoops! All of a sudden, it came back on... A mind of its own, lol.
Go figure....!!!
I have those bookmarked in my resources:
capcut .com
invideo .io
synthesia .io
trymaverick .com
I would also reach out to Phil @Phil1944 for more info here :)
See more comments
Have you tried any ai video generation yet?
If you have, do you have any recommendations for using AI to generate videos from posts?
Thank you
;-)
Richard
Great question.
synthesia .io, invideo.io and Canva are the ones I tried.
Thanks for sharing guys.
Mel B
I have the one that does blog posts to ai videos is from
pictory. ai
videotoblog.ai (reverse videos to blogs)
videogen.io
veed.io
Canva Pro does AI videos too
That the one I know of, I tried it a few times, not bad! I like it better than the others that Abie mentioned.
I have that bookmarked, too, Brenda :) When I did a search it was hiding lol Good job, you have it
Awesome. I didn't bookmark them. Just the ones I already started to use during the free trials.
I had an adapter failure last night... I lost broadband for a while, lol - it was driving me up the wall. I could not figure out how to fix it, so I kept rebooting and punching the 'F' keys on top... Whoops! All of a sudden, it came back on... A mind of its own, lol.
Go figure....!!!
I have those bookmarked in my resources:
capcut .com
invideo .io
synthesia .io
trymaverick .com
I would also reach out to Phil @Phil1944 for more info here :)
See more comments
I'm experiencing a weird problem.
Suddenly today I am constantly getting a message that says:
"Updating failed. Sorry, you are not allowed to edit this post."
Ha
Richard, you have FANTASTIC responses here; kindly let us know if you require additional assistance.
Hi Abie
Of my activated plugins the following were updated in the last couple of days:
CookieYES
Editor's Kit
Kadence Blocks
MailPoet
RankMath
U+draft
I used WP Rollback to roll all of them back, one at a time, some of them several times and didn't find a culprit. I undid the rollbacks. So moving forward but currently stumped! Any ideas?
;-)
Richard
That has been resolved now, Richard :) I cannot update existing pages or sidebar widget?
Thank you. That was yesterday. Site Support told me that they adjusted some security parameters.
:-)
Richard
Hey Richard,
Kadence Block plugin got updated AGAIN today, so if you're using THAT plugin you may find the update solving this.
Updated all sites using that plugin yesterday and got notified of another update only a few hours ago now.
Hope this helps you.
Hi Trish
Thanks for responding!
Another Kadence Blocks update (and theme update) arrived overnight and unfortunately they've made no difference.
;-)
Richard
I rolled back Kadence Blocks and 5 other plugins and didn't find the culprit. The next stage has to be a patient check of combinations.
;-)
Richard
I did put a ticket into Kadence but it now seems that Site Support are working on a fix so it might be a local (server) issue.
;-)
Richard
Hi Troy
Site Support tell me that many people are having the same problem. It would be worth comparing plugins. I'll send you a PM it that's OK?
;-)
Richard
See more comments
I'm experiencing a weird problem.
Suddenly today I am constantly getting a message that says:
"Updating failed. Sorry, you are not allowed to edit this post."
Ha
Richard, you have FANTASTIC responses here; kindly let us know if you require additional assistance.
Hi Abie
Of my activated plugins the following were updated in the last couple of days:
CookieYES
Editor's Kit
Kadence Blocks
MailPoet
RankMath
U+draft
I used WP Rollback to roll all of them back, one at a time, some of them several times and didn't find a culprit. I undid the rollbacks. So moving forward but currently stumped! Any ideas?
;-)
Richard
That has been resolved now, Richard :) I cannot update existing pages or sidebar widget?
Thank you. That was yesterday. Site Support told me that they adjusted some security parameters.
:-)
Richard
Hey Richard,
Kadence Block plugin got updated AGAIN today, so if you're using THAT plugin you may find the update solving this.
Updated all sites using that plugin yesterday and got notified of another update only a few hours ago now.
Hope this helps you.
Hi Trish
Thanks for responding!
Another Kadence Blocks update (and theme update) arrived overnight and unfortunately they've made no difference.
;-)
Richard
I rolled back Kadence Blocks and 5 other plugins and didn't find the culprit. The next stage has to be a patient check of combinations.
;-)
Richard
I did put a ticket into Kadence but it now seems that Site Support are working on a fix so it might be a local (server) issue.
;-)
Richard
Hi Troy
Site Support tell me that many people are having the same problem. It would be worth comparing plugins. I'll send you a PM it that's OK?
;-)
Richard
See more comments
You have FANTASTIC responses here; I also leaned too :) I appreciate your question.
This is true, Abie!
;-)
Richard
Awesomely welcome Richard :)