Default directories NOT to include in robots.txt - Bugs & Issues - Bugs & Issues - Progress Community

Default directories NOT to include in robots.txt

 Bugs & Issues

Default directories NOT to include in robots.txt

  • Default directories NOT to include in robots.txt
  • Does anyone have a standard protocol for which directories to disallow in a robots.txt file for Sitefinity deployment?   For example, platforms like wordpress recommend certain directories to disallow (/wp-admin/, /wp-content/, /wp-includes/, etc.).   

    Obviously Sitefinity is far more advanced than wordpress, but the principle is the same.  

    So, I'm curious if such advice exists from fellow users or from Sitefinity themselves.  

     

    Thanks,

    Liz