I’m using MarkLogic 12.0.1 + Data Hub Central 6.2.1 on localhost.

I have a Student Smart Mastering use case with two CSV sources. Currently I’m testing with around 10k rows from Source A and 10k rows from Source B, but the final target is around 200k total rows.

Current flow:

1. Load Program reference CSV into MarkLogic.
2. Map Program data into canonical Program records.
3. Load Student Source A CSV into MarkLogic.
4. Load Student Source B CSV into MarkLogic.
5. Map both student sources into canonical Student records.
6. Run Smart Mastering matching on Student records.
7. Matching result is split into:

• no match
• auto merge
• manual review notification

1. Run Smart Mastering merging for auto-merge candidates.
2. Store mastered/merged Student records in MarkLogic.
3. Store manual-review notification documents in MarkLogic.
4. Custom Manual Review UI reads pending notifications from MarkLogic.
5. User reviews uncertain pairs in the UI.
6. User clicks Merge.
7. UI calls a MarkLogic manual-merge resource.
8. MarkLogic creates/updates the mastered Student record.
9. UI deletes the handled notification.
10. Final mastered records are used for graph exploration.

Current issue:

At only around 20k total student rows, performance already feels slow:

• matching/mastering takes a long time
• pending notification list loads slowly
• merged record list loads slowly
• manual merge can take seconds per record
• dashboard counts are slow

Before I scale this to around 200k rows, what should I change in the Smart Mastering flow, matching/merging steps, indexes, batching strategy, or UI read pattern to avoid much worse performance?