OpenEdge 10.1B02 Service Pack, 10.1C introduced a realtime memory consistency check as a proactive utility that enables memory overwrite checking within the buffer pool. This consistency check is a form of memory corruption detection which detects illegal operations in memory movement, such as a buffer copy with a negative length of data or a memory move to an incorrect location. Other consistency check parameters (-DbCheck, -TableCheck, -IndexCheck, -AreaCheck) check physical block contents whereas -MemCheck is a proactive parameter that applies memory overwrite checking to the buffer pool block boundaries (memory boundaries) in realtime before data is committed to disk. If -MemCheck is enabled and if it detects a memory overwrite issue within the buffer pool a fatal error will be generated, and database integrity will be preserved by forcing an Emergency database shutdown.
What is a Memory Overwrite ?
- When a user performs an operation like an insert, a delete or an update, the data within the associated buffer pools’ block(s) may need to be re-arranged within that block(s)
- A ‘memory overwrite’ is an erroneous operation occurring when data that should be rearranged within a given block is accidentally written to a location outside of that block. Such a scenario can arise for example as a result of a corruption in memory.
- -MemCheck is designed to check for a large variety of potential memory overwrite issues within the database buffer pool
- If -MemCheck is enabled and it detects a memory overwrite in the buffer pool, a fatal error will be generated and database integrity will be preserved by forcing an Emergency database shutdown.
- When the data that are expected to move around within the block is accidentally written outside of that block, caused by corruption in memory for some reason, this is called a “memory overwrite”.
What is Memory Overwrite Checking:
- The purpose of the Memory Consistency Check is to prevent memory operations that exceed the database block size and in effect, decrease the time needed to find and fix corruption when a crash issue results with the usual utilites: dbtool, idxcheck, dbrpr scans, idxfix, idxbuild.
- It enables memory overwrite checking which focuses on preventing potential memory overwrites.
- It is different to the Physical Block Consistency Checking (-DbCheck, -TableCheck, -IndexCheck,-AreaCheck) which validates index blocks and record block content. Memory Overwrite Checking checks the block boundary, which validates index blocks and record blocks in the Buffer Pool (memory).
- Enabling -MemCheck enhances the buffer pool manager to help identify block overwrites at crucial steps in the management of the life cycle of the block. Physical record and index block operations are checked for conditions that provide an opportunity to overwrite block buffers in the buffer pool after each write operation.
- It checks that the note offset plus note length of the intended operation does not exceed the database blocksize prior to the before-image note write and subsequent database block write.
- During RM block changes, this means the transaction note is prevented from being written at all and the database block if corrupt, is prevented from being written.
- During index expansion, insert operations are checked during the strategic points of the insertion of a key element. Leaf blocks are checked for the information byte size, and non leaf blocks for compression size changes, key element changes, after the data portion of the key is added.
- In addition, potential overwrite during index block split operations are checked during the strategic points of the insertion of a key element to ensure that the end of the index block is not overwritten and the key element is properly split in the new index block.
- Performance loss when using -MemCheck is expected to be nearly 0%.
Using Memory consistency check (-MemCheck) 1. Memory consistency check can be enabled for a single user client or for any offline utility:
- For single-user access, as a Client Connection parameter:
$ progres <dbname> -MemCheck
$ rfutil <dbname> -C roll forward -a dbname.a1 -MemCheck
$ proutil <dbname> -C idxbuild -MemCheck
If the utility is read-only, such as DBANALYS, IDXCHECK the -MemCheck parameter is ignored
2. Memory consistency check can be enabled for multi-user access:
- As a startup parameter on the database Primary Broker:
$ proserve <dbname> -S 5768 -B 100000 -L 100000 -MemCheck
- At runtime enable/disable with PROMON or the _DbParam VST:
$ PROMON [dbname] > R&D > 4. Administrative Functions > 8. Enable/Disable block level consistency check > 1. -MemCheck
Queried with the OpenEdge 11.5 _DbParam VST and enabled/disabled:
// Enable MemCheck
DO TRANSACTION:
FIND FIRST _DbParams WHERE _DbParams._DbParams-Name = "-MemCheck".
ASSIGN _DbParams._DbParams-value = "1". /* ENABLED */
END.
// Query the current value:
FIND FIRST _DbParams NO-LOCK WHERE _DbParams._DbParams-Name = "-MemCheck".
IF AVAILABLE _DbParams THEN
DISPLAY _DbParams-Desc SKIP
_DbParams-Name _DbParams-Msg-Num _DbParams-Value _DbParams-Is-Modifiable SKIP.
Memory Overwrite Messages:The advantage of Memory Overwrite Checking is that corruption are likely to be found at the time as opposed to later when next accessed.
When a “memory overwrite” is detected, the current operation is halted and messages are produced detailing the nature of the potential memory overwrite in the database lg file. This information needs to be provided to OpenEdge Technical Support for further evaluation:
The following messages confirm when Memory Overwrite Checking is enabled:
- Memory overwrite check (-MemCheck): [Not Enabled | Enabled] (14015)
- Memory overwrite check: Disabled (-MemCheck) (14023)
When Memory Overwrite Checking detects a possible memory overwrite, the block and supporting information correlating to the specific check that failed are dumped to the database lg file:
During RM block changes:
- SYSTEM ERROR: Internal Error: memory check fatal (14047)
- Internal ERROR: bkReplace: data too long! offset: len , dumping block dbkey: (14048)
During strategic points of the insertion of a key element to an Index:
- cxDoInsert: About to copy too much! (a), dumping block dbkey: (14043)
- cxDoInsert: About to copy too much! (b), dumping block dbkey: (14044)
- cxDoInsert: About to copy too much! (c), dumping block dbkey: (14045)
- cxDoInsert: About to copy too much! (d), dumping block dbkey: (14046)
During strategic points index block split operations:
- Internal ERROR: cxDoSplit(aa) data too long! offset: len (14038)
- SYSTEM ERROR: Internal ERROR: memory check fatal (14039)
- cxDoEndSplit: Index About to copy too much! (a), dumping block dbkey: (14040)
- cxDoEndSplit: Index About to copy too much! (b), dbkey: (14041)
- cxDoEndSplit: Index About to copy too much! (c), dumping block dbkey: (14042)
During single-user access:
- <function name>: Index <index-number> about to copy too much!() Dumping block dbkey: <area>/<dbkey> (15152)
