ASHISH PATIL (Customer) asked a question.

Excluded paths have no effect on an existing database, even after manual reindex and merge

MarkLogic Server 12.1.0, single-node, single-forest.

 

What we did:

1. Loaded ~1000 XML documents into a database with default indexing config.

2. Configured 5 index-exclusion-paths entries via the Management API.

3. Waited to see if the reindexer picked up the change automatically.

4. Manually triggered a reindex, then a database merge.

5. As a separate run, cleared the database, configured the same excluded

  paths on the empty database, then reinserted the same 1000 documents.

 

What we observed:

- After step 2, automatic reindexing did not trigger.

- After step 4 (manual reindex and merge), there was no effect - verified

 via before/after size metrics and via word search (content inside the

 excluded paths was still returned by cts:word-query).

- After step 5 (clear, configure, then reload), the universal index was

 reduced as expected: 71.5% smaller, with content size unchanged, and

 word search no longer matched content inside the excluded paths.

 

No data loss in either case.

 

Workaround: clear the database, apply the excluded-paths config first,

then load documents.

 

Question: Is there a supported way to apply excluded paths to an existing

database and have it actually take effect, without a full clear-and-reload?


  • Mads Hansen (Progress Software)

    Double check that your reindex test didn't have a mistake or issue with the XPath. I just did a quick test adding an excluded path on a 12.1.0 instance, and it triggered a reindex, and then my search for a word in an element matching that excluded path that previously returned the doc no longer did.

  • David Ennis (Clever Llamas)

    Hmm. Reindex enabled and setting the reindex to current timestamp should also trigger a full reindex.

  • James Kenwood (Progress Software)

    Hi Ashish, did Mad's response help answer your issue? If not please let us know. We could also probably handle this via a Support Ticket if needed. Feel free to reach out to me if you have any difficulties logging this.

  • ASHISH PATIL (Customer)

    Following up on this ticket, as requested we retested the behavior on MarkLogic 12.1.0, this time on two different systems with a higher configuration than our first attempt.

    There is definitely an issue with this feature. On both systems, adding excluded paths to a database that already holds documents triggered a reindex automatically, but the reindex never completed. It stayed stuck in a continuous merging / reindexing / refragmenting loop for more than 8 hours, on a database of only 1000 documents.

    • System 1: still stuck after more than 12 hours.
    • System 2: stuck for more than 8 hours, after which we cleared the database.

    A reindex/merge of 1000 documents should take minutes (seconds actually), not hours.