[Sitefinity Cloud] Domain Management: Preventing Search Engine Indexing of Internal Domains

Customers using Sitefinity Cloud may notice that their internal .sitefinity.cloud hostname appears in Google search results, even when a robots.txt file is configured to disallow crawling. This occurs because robots.txt controls crawling behavior but does not prevent indexing. The recommended solution is to apply an X-Robots-Tag: noindex, nofollow response header to the internal hostname and allow search engines to crawl the site so that they can detect the directive and remove indexed URLs over time.

Jul 9, 2026
Knowledge

Title

URL Name

sitefinity-cloud-domain-management-preventing-search-engine-indexing-of-sitefinity-cloud-internal-domains

Article Number

000302704

Environment

Product: Sitefinity Cloud

Question/Problem Description

A Sitefinity Cloud implementation uses both a public domain and an internal .sitefinity.cloud domain. The internal domain is discovered and indexed by search engines, causing internal URLs to appear in search results.

Administrators may want to:

Prevent the internal .sitefinity.cloud domain from being indexed.
Disable or remove the internal domain entirely.
Ensure only the public domain appears in search engine results.

Steps to Reproduce

1. Configure a Sitefinity Cloud project with both a public custom domain and the default .sitefinity.cloud domain. 2. Configure a robots.txt file on the internal domain that disallows crawling. 3. Allow the site to be crawled or discovered by search engines. 4. Search for content from the internal hostname in a search engine. 5. Observe that URLs from the internal domain may still appear in search results.

Clarifying Information

Error Message

No explicit platform error message is generated. Observed behavior: Internal .sitefinity.cloud URLs appear in search engine results despite a restrictive robots.txt configuration.

Defect Number

Enhancement Number

Cause

The internal .sitefinity.cloud hostname is a required platform domain used by Sitefinity Cloud infrastructure for operations such as routing, monitoring, caching, and other platform services. As a result, it cannot be disabled or removed.

Additionally, a robots.txt file that disallows crawling does not guarantee that URLs will not be indexed. Search engines can still index URLs if they are discovered through links or other sources, even when crawling is blocked.

Resolution

Apply an X-Robots-Tag: noindex, nofollow response header to the internal .sitefinity.cloud hostname.

Example:

X-Robots-Tag: noindex, nofollow

This header explicitly instructs search engines:

noindex – Do not include the page in search results.
nofollow – Do not follow links found on the page.

After the header is applied:

Update the internal domain's robots.txt file to allow crawling:

User-agent: *

Allow: /

Allow search engines to recrawl the internal domain so they can detect the noindex directive.
Over time, indexed URLs will be removed from search engine results.
Optionally, configure canonical tags that point from the internal domain URLs to the equivalent public domain URLs to reinforce the preferred URL structure.

Workaround

To accelerate removal of already-indexed URLs:

Verify ownership of the internal hostname in Google Search Console.
Use the Removals tool to request temporary suppression of indexed URLs.
Maintain the X-Robots-Tag: noindex, nofollow header until search engines permanently remove the URLs from their index.

This approach provides quicker removal of existing search results while the search engine recrawls and processes the noindex directive.

Notes

Keyword Phrase

Last Modified Date

7/10/2026 6:33 PM

[Sitefinity Cloud] Domain Management: Preventing Search Engine Indexing of Internal Domains

Files

Was this article helpful?

Was this article helpful?

[Sitefinity Cloud] Domain Management: Preventing Search Engine Indexing of Internal Domains

Information

Related Articles