# Served on every host this deployment answers for: siteo.io and every client # domain connected to it. # # /w/ is deliberately NOT disallowed. Those pages carry X-Robots-Tag: noindex # (vercel.json) and a noindex meta, and a crawler has to be allowed to fetch a # page in order to read either. Disallowing them here would instead freeze any # already-indexed preview URL in the index, which is the opposite of the point. User-agent: * Allow: / # /admin, /login, /client-login and /upload carry X-Robots-Tag: noindex # (vercel.json), for the same reason as /w/ above: a Disallow here would stop # crawlers from reading it. # The marketing pages only. Client sites and lead-magnet pages are deliberately # absent — see the comment in sitemap.xml. Sitemap: https://www.mentorshipboss.com/sitemap.xml