Robots.txt
« Back to Glossary IndexA file that tells search engines which parts of a website to crawl and index.
« Back to Glossary Index
A file that tells search engines which parts of a website to crawl and index.
« Back to Glossary Index