Robots.txt and sitemap.xml

Need Help
#1

Hello

I placed both files into the root of the website. The files can not accessed: A 404 error occurs.
Probably a route or so is needed.

I tried this:

    ##
    ## Allow robots.txt and sitemap.xml
    ##
    RewriteRule ^robots.txt - [L]
    RewriteRule ^sitemap.xml - [L]

Something is missing yet or is wrong.
I don’t want to use a plugin for this. It is not needed to manage the files from the Backend.

Any idea?

Thank you!

#2

Self answer:

Add this to the “Allowed folder” or create a new section “Allow files” to the file .htaccess:

 RewriteCond %{REQUEST_FILENAME} !/robots\.txt
 RewriteCond %{REQUEST_FILENAME} !/sitemap\.xm

That’s all!

#3

If you would like to edit the files in the Editor, you can also set up pages for those files:

theme/[theme-name]/pages/robots.htm

url = "/robots.txt"

[resources]
headers[Content-Type] = "text/plain"
==
User-agent: *
Allow: /

theme/[theme-name]/pages/sitemap.htm

url = "/sitemap.xml"
title = "Sitemap"

[resources]
headers[Content-Type] = "application/xml"

Additional infos to set up the sitemap with Tailor: sitemap-plugin/UPGRADE.md at master · rainlab/sitemap-plugin · GitHub

2 Likes