Sitemap.xml Generation - Feature Previews & Betas - Feature Previews & Betas - Progress Community
 Feature Previews & Betas

Sitemap.xml Generation

  • Sitemap.xml Generation
  • Hello Everyone,

    We are planning to include Sitemap.xml generation, within a future release. The current considerations are based on the standards specified on Sitemaps.org and recommendations by search engines:

    • Allow a user to specify file name, directory and number of records per file
    • Option to compress and cache the Sitemap files(s)
    • The URLs of pages will be included by default
    • URLs of content items such as forum posts, news items etc. will be optional, which the user can choose to be included

    Could you please consider the list and provide us with feedback? 

    Thank you in advance.

    Charl

  • First of all this would be a great addition. In fact one that I feel should have been added  a long time ago.

    Would it be possible to allow the option to not include specific pages in the sitemap? For example, when we set up pages that are specific to a marketing event, I can keep them out of our sitemaps by checking the box to not have search engines index the page. 

     Also, will we have the ability to set how often the sitemap is regenerated?  Or will it be regenerated as we add new content?

     Finally, will we have the option to set specific items like change frequency and priority?

  • Hi Jeremy,

    We will consider the option to exclude a specific page and to set frequency/priority.

    Could you confirm by you question with regards to how often a sitemap is generated, that it implies a sitemap generation schedule for instance daily or weekly?

    The original idea was to only generate the file at a specific point, which is either manually or should you confirm in a schedule. Do you see a need to generate the sitemap, when the content changes?

    Best Regards

    Charl

  • Charl,

    Ideally it would be nice to have the option to set the sitemap to generate to:

    1. When pages/content are published
    2. Daily at a set time
    3. Weekly at a set time
    4. Manually

    This will provide smaller sites the ability to have the sitemap published quickly. Most of our smaller clients would never go in and manually update their sitemap. So having these option allows us the ability to make sure it is done for them. It would also provide sites that are aggressive in making sure their content is indexed quickly the ability to have their sitemaps updated instantly.

    However, some of our larger sites would want to limit when the sitemap is generated so they are not using up the site resources every time new content is updated. Plus they would be able to generate the sitemap at a point when their site usage is low and not impact site performance during peak hours. 

  • Shouldn't the sitemap generator take into consideration if a checkbox Allow search engines to index this pages is checked?

    Markus

  • I would like this scenario covered...

    /product/detail (Contains a dynamiccontent widget in Detail Mode)
    /product/detail/prod1
    /product/detail/prod2
    /product/detail/prod3

     ...okay so prod1,2,&3 should all be in the sitemap, but we need to make sure the root "detail" page should not be there (or browsable).

     Also would love a UI that would spider the sitemap and report errors, or just link to the webmaster tools which would do that too I think...let the users find the problems and fix them, not just trust the black box.

     Other than that, following a spec is great, thanks guys :)

  • Yep, I agree with Steve,

     All News, Blog, Custom Modules details pages should be inserted into the sitemap, except the root one.  But we also need to consider if the root page is containing the widget in list and details mode too. In this scenario it has to be contained in the sitemap.

     

  • The current idea is to have a page-level setting, which will indicate if the page URL is included in the Sitemap. An additional setting for the page will indicate if the URLs of the page contents are included in the Sitemap.

    @Steve & Federico:

    Would you consider that the settings will allow you to configure the scenario(s) you gave? The root page could be configured to not be included, but the content to be included for instance or only the page URL should be included without the content URLs.

    Spider the sitemap is added to the list for future considerations.

    @Markus

    Sitemap generation will consider “Allow search engines to index this page”, as well as list only pages which are visible to anonymous users.

    Please let me know if there are further considerations.

  • Apart from the ideas I posted along with the feature request, custom providers for content items should be supported. In my case I have a forum with a custom forums provider which generates a custom URL format.

  • We would like to see options for multiple xml sitemaps, for example video and mobile which have different schemas.

    A control on the frequency of site map production would be useful.

    Also manually setting exclusion rules based on URL/filename.

  • +50 for video sitemap 

    wistia.com/.../video-seo

     ...although a less gimped video module would have to go hand in hand with it (a module that's usable)

  • @Arno

    The features listed in your feature request are considered, which includes having HREFLANG attribute in the Sitemap file for language variations.

    We also consider the URL of content items for both standard and custom providers.

     @Adam

    Frequency of Sitemap and excluding pages/content are considered. Jeremy provided a list of suggested settings for frequency earlier in this thread which we will consider, so please feel free to add additional suggestions if you have :-)

    @Adam & Steve

     Video Sitemaps is on the list of considerations for future releases.

  • Hi Kurren,

    We are planning to implement the feature during the upcoming release.

    Best Regards

    Charl

  • Any update on this?