Allow search engines to index this page - General Discussions - General Discussions - Progress Community
 General Discussions

Allow search engines to index this page

  • Allow search engines to index this page
  • Hello,

    Editing a Page Title and Properties, under the Advanced Options, is a checkbox, "Allow search engines to index this page".

    I couldn't find anything in the documentation on this for 4.0.  In 3.7, it was a radio button that defaulted to yes.

    Should this default to checked?  Is there a global setting somewhere that overrides it?

    Thanks
    -Noah
  • 254012_advoptions.png
    Hello Noah,

    You can control this from the "Advanced" options of each page.

    Regards,
    Ivan Dimitrov
    the Telerik team
  • Dear Ivan

    When talking about search engines are we talking about google and friends?

    If yes what does it to to the page. Would not see a <meta tag?

    If this is ment to be that the page can be searched by SF internal then it should defenitely be renamed.

    Markus
  • Hi Ivan,

    I know where it is located, but it is defaulting to unchecked.  Do you have to go through each page and enable it?

    Thanks
    -Noah
  • Hello Noah,

    In SP1 the default value for Allow search engines to index this page is checked . You can set the value programmatically for each pages using the fluent API.

        App.WorkWith().Pages().ForEach(p =>
                   
                        p.Page.Crawlable = true;
                  
                    ).SaveChanges();

    Regards,
    Ivan Dimitrov
    the Telerik team
  • Dear Ivan

    Can you answer my questions as well?

    Markus
  • Hello Markus,

    I believe that we are talking about the Sitefinity's search capabilities. All meta tags for the public engines can be added through page properties sections, but if the page is published and it is navigable it will be shown on the public side - this page will be accessible by the public users.

    Regards,
    Ivan Dimitrov
    the Telerik team
  • Thanks Ivan

    So please let me make two feature requests

    1) Rename existing to "Allow Sitefinity search engines to index this page".

    2) Make a second one "Allow external search engines to index this page".  which will set an meta robot noindex

    I think 1) is important because when I hear someone talk about Search Engines I would assume he is talking about Google and freinds (SEO)

    Markus
  • Hi Ivan,

    I agree with Markus.  As the check box says search engines, plural, I would think it controlled external search engines such as Google.  I was under the impression that this was related to external search engines being able to index the page.  It would be helpful if the interface (and documentation which is missing on this topic), was updated to reflect which search engine(s) were affected.

    Thanks
    -Noah
  • @Noah

    Thanks for backing me up

    @Ivan

    When you write 'I believe ....' does that mean you are not sure?

    Markus
  • Hi Markus Berchtold,

    This option is only for external search engines like Google which means that if it is enabled the page will be indexed by the external engines. 


    Greetings,
    Antoaneta
    the Telerik team
  • Dear Antoaneta

    Thanks for clearing that up. So it acutally makes sense.

    This leaves the question how is indexing by search engines surpressed when using this option?

    I dont see a mate robots noindex there.

    How should it work?

    Markus

    PS: How about make an extrea feature make available for internal search engine then :-)
  • Hi Markus Berchtold,

    If you uncheck this option the page will not be indexed by the search engines, but will still be indexed internally.

    Kind regards,
    Antoaneta
    the Telerik team
  • Dear Antoaneta

    <meta name="robots" content="noindex" /> is really present when unchecked. Sorry missed that.

    Would it still make sense to have a secend option for SF internal search?

    markus