This Article describes the enhanced IDXBUILD memory algorithm since OpenEdge versions 10.2B07, 11.2.x
- The initial algorithm when IDXBUILD was first enhanced to address more memory and threads (OpenEdge 10.2B06, 11.1), based merge thread memory (-TMB * -TM) on the number of Sort Groups (SG). This can result in over allocating initial memory, not needed for the merge buffers when all sort-groups are not used.
- The current algorithm assures that merge thread memory (-TMB * -TM) is allocated based on the -threadnum value rather than on the number of Sort Groups.
The amount of memory initially allocated by IDXBUILD can be
estimated with the following formula:
Data scan Overhead:
dataScanOverhead = -datascanthread * -TMB
Memory used by each Sort/Merge group:
eachSortGroup = ((-TMB * -TM) + TMB) * -mergethreads
Memory allocated for all Sort groups:
allSortGroups = eachSortGroup * -threadnum
-TF:
tfMemory = MIN((physicalMemory - InuseMemory), ((physicalMemory * -TF) / 100))
Total initial memory used:
totalMemory = dataScanOverhead + tfMemory + allSortGrups [+ some overhead 1-2 GB]
Example: 14 GB IDXBUILD memory requirement Assuming:
- 32 GB of memory and 2.5GB inuse memory
- We'd like the initial total Memory for IDXBUILD with these parameters to be around 45% of all physical memory.
PROUTIL -C IDXBUILD ALL -B 2048 -t -T ../tmp -rusage -SG 64 -thread 1 -threadnum 4 -TB 64 -TMB 512 -TM 32 -TF 40 -datascanthreads 24 -mergethreads 24
Data scan Overhead:
dataScanOverhead = 24 * 524288 = 12,582,912
Memory used by each sort/merge group:
eachSortGroup = ((524288 * 32) + 524288) * 24 = 415,236,096
Memory allocated for all sort groups:
allSortGroups = eachSortGroup * 4 = 1,660,944,384
-TF:
tfMemory = MIN((34,359,738,368 2,684,354,560), ((34,359,738,368 * 40) / 100)) = 13,743,895,347
Total initial memory used:
totalMemory = 12,582,912 + 1,660,944,384 + 13,743,895,347 = 15,417,422,643 or 14GB