SEO
Sep 1, 2025
How to access the robots.txt file
To ensure search engines can correctly navigate your site, a robots.txt file is necessary. In Framer, this file is automatically generated, and you can easily check it by appending /robots.txt to your domain.
Uploaded by
Translated by
Contents
Table of Contents
This document was created to address the challenges domestic users face due to the lack of Korean resources for learning Framer. We've translated content from the official blog into Korean and added practical information. We hope it provides some benefit to those using Framer.
What is robots.txt?
The robots.txt file is a set of guidelines that informs web crawlers (robots) like search engines about which pages on a website can be included in search results and which should not be crawled (collected).
For example, areas that shouldn't be public, like login or admin pages, can be excluded from search results using robots.txt. This file plays a crucial role in Search Engine Optimization (SEO), helping search robots navigate a website more efficiently and index only the necessary content accurately.
How to check robots.txt
Websites created with Framer automatically generate a robots.txt file, which can be accessed by simply appending /robots.txt to the end of the site address without any additional configuration.
When you access this address, you can see in text format the instructions the website gives to search engine crawlers.Note: The robots.txt file is a technical guide for search engines and robots, not for pages visible to general users. Incorrect settings can prevent crucial pages from being indexed, so caution is necessary.
This article is a translated and adapted version of ‘How to access the robots.txt file’ from the official Framer blog.
