Salesforce

How much memory will the enhanced IDXBUILD allocate initially ?

« Go Back

Information

 
TitleHow much memory will the enhanced IDXBUILD allocate initially ?
URL Name000037016
Article Number000160620
EnvironmentProduct: OpenEdge
Version: 10.2B07, 11.2.x and later, 12.x
OS: All supported platforms
Question/Problem Description
How to calculate the memory initially allocated for IDXBUILD?
How does datascanthread and -TMB affect the memory usage of IDXBUILD?
How do mergethread, -TM and -TMB contribute to an IDXBUILD memory consumption?
How much memory does the -TF % allocate to an idxbuild operation?
How much memory will IDXBUILD allocate when using less datascanthreads?
How much memory will IDXBUILD allocate when using more mergethreads?
Steps to Reproduce
Clarifying Information
Error Message
Defect Number
Enhancement Number
Cause
Resolution
This Article describes the enhanced IDXBUILD memory algorithm since OpenEdge versions 10.2B07, 11.2.x 
  • The initial algorithm when IDXBUILD was first enhanced to address more memory and threads (OpenEdge 10.2B06, 11.1), based merge thread memory (-TMB * -TM) on the number of Sort Groups (SG). This can result in over allocating initial memory, not needed for the merge buffers when all sort-groups are not used.
  • The current algorithm assures that merge thread memory (-TMB * -TM) is allocated based on the -threadnum value rather than on the number of Sort Groups.
The amount of memory initially allocated by IDXBUILD can be estimated with the following formula:
 
Data scan Overhead:
dataScanOverhead = -datascanthread * -TMB
 
Memory used by each Sort/Merge group:
eachSortGroup = ((-TMB * -TM) + TMB) * -mergethreads
 
Memory allocated for all Sort groups:
allSortGroups = eachSortGroup * -threadnum
 
-TF:
tfMemory = MIN((physicalMemory - InuseMemory), ((physicalMemory * -TF) / 100))

Total initial memory used:
totalMemory = dataScanOverhead + tfMemory + allSortGrups [+ some overhead 1-2 GB]
 
Example: 14 GB IDXBUILD memory requirement Assuming:
  • 32 GB of memory and 2.5GB inuse memory
  • We'd like the initial total Memory for IDXBUILD with these parameters to be around 45% of all physical memory.
PROUTIL -C IDXBUILD ALL -B 2048 -t -T ../tmp -rusage -SG 64 -thread 1 -threadnum 4 -TB 64 -TMB 512 -TM 32 -TF 40 -datascanthreads 24 -mergethreads 24 

Data scan Overhead:
dataScanOverhead = 24 * 524288 = 12,582,912
 
Memory used by each sort/merge group:
eachSortGroup = ((524288 * 32) + 524288) * 24 = 415,236,096
 
Memory allocated for all sort groups:
allSortGroups = eachSortGroup * 4 = 1,660,944,384
 
-TF:
tfMemory = MIN((34,359,738,368  2,684,354,560), ((34,359,738,368 * 40) / 100)) = 13,743,895,347
 
Total initial memory used:
totalMemory = 12,582,912 + 1,660,944,384 + 13,743,895,347 = 15,417,422,643 or 14GB
Workaround
Notes
Keyword Phrase
Last Modified Date11/3/2021 1:09 PM

Powered by