The Progress database manager will reuse space that has been freed up by means of deletions, depending on:
How much space each deletion left vs How much space is required by the create.
Types of Database Blocks:There are three types of database blocks involved, each block is equal to the size of the database blocksize.
1. RM Chain (Record Management) blocks
- Blocks on the RM Chain blocks are neither full nor empty and data.
- If a block has at least the Create Limit (150 bytes by default in OpenEdge 10 and higher version (75 for systems with 1K blocks size) or 75 bytes free in earlier versions), then the block is put on the RM Chain.
- When the RM Block is removed from the RM Chain, the database engine leaves the Toss Limit (300 bytes free by default in OpenEdge 10 and higher version or 150 bytes for 1K block size), in each block to allow for expansion in the event that the record is updated in the future. This is to reduce the number of records that are split across multiple blocks.
- The Create Limit is examined first before new records are created then the remaining Toss Limit determines eviction from the RM Chain. Otherwise, the remaining Toss Limit is examined for updates.
- Once an RM block, always an RM block; a RM block will never become a Free block even when all records are deleted from it. It will move on and off the RM chain based on the space allocation values set for it (RPB, tosslimit, createlimit)
2. Free Chain blocks
- Free Blocks are empty database formatted blocks below the Storage Area's High Water Mark (HWM).
- These blocks are available to be allocated as either index blocks or data blocks.
- For Type II Areas, when a new Cluster is added for an index, all the blocks in a new cluster are added to the Free chain of the index object except one block which is added to the index btree. When adding additional index blocks, they are removed from the free chain, formatted and added to the btree.
- For Type II Areas, when a new Cluster is added for a table from the Free Chain, all the blocks are formatted as RM blocks and added to the RM chain. This avoids having to remove an individual block from the Free Chain, format it and add it to the RM chain at runtime which would be costly when adding many records.
3. Empty Blocks
- Empty Blocks are completely empty and unformatted.
- They have never contained any data, they are above the Storage Area's High Water Mark (HWM).
- Empty Blocks are made available for use when the space allocation algorithm requires, at which time the HWM of the Area is raised and the Empty Blocks are added to the Free Chain as formatted blocks.
What does progress do when deleting records from the database ?The blocks on the RM Chain are there due to the creation of records leaving a block partially full, or the deletion of records leaving the block partially empty. Either way, this block can hold more data and is therefore left on the RM Chain. Once it is completely full (still allowing for expansion space) or completely empty it is popped to the RM blocks or the RM Free Chain respectively.
After many deletions, there are now many blocks on the RM Chain. When a new record is created, the database manager will search the RM Chain for space to accommodate new record creation.
To find space to store a record, or a record fragment, the database manager first looks at the RM Chain to see if an existing record block with unused space is available. If the block at the head of the RM chain contains enough space to store the fragment while still leaving expansion space (Toss Limit), and the block has unused record slots, then that block is used. The record is copied to the block and the amount of unused space in the block is updated.
If the block at the head of the RM chain cannot be used to store the record, and it has less unused space than the Toss Limit, or if all record slots have been used, it is removed from the RM chain. Otherwise it is moved to the tail end of the chain. In either case, the next block on the front of the RM chain is considered and this process is repeated until sufficient space is found or the search limits have been reached.
The algorithm needs to balance the search with performance. To limit the search time, no more than 100 blocks on the RM chain will be examined and removed from the chain at a time. No more than 3
consecutive blocks at a time will be examined and moved to the tail end. If either of these limits are exceeded during a search for space, the search for reusable space is abandoned and a free block is used instead by moving it from the from the Free Chain the RM Chain.
If no existing record block can be used, an unused block is allocated from the Empty blocks. The Storage Area's High-Water Mark is raised, the block is formatted as an RM block, and finally the record is stored in it. If the record is larger than 1 block, another block from the Free Chain is used to hold the overflow. The first block that was searched in the RM Chain is then moved to the tail end of the RM Chain so that the same blocks are not repeatedly searched.
The above discussion applies to Type I Storage Areas. With the introduction of Type II Storage Areas in OpenEdge 10, while the terminology remains, the behaviour or use and re-use changes. Refer to Article
VST's to monitor space in a Type II Storage Area? Additional Considerations:While the database chains can be rebuilt through DBRPR utilities, this is not considered normal production operation unless there is some very specific reason to do so. Even rebuilding the chains, will not re-order the blocks on the RM chain from 'most' to 'least' available space. In other words if there are many blocks at the head of the chain with little space (greater than the Toss limit), then it's likely that Free blocks and eventually Empty blocks are needed until the existing blocks are eventually used by smaller updates and tossed off the RM chain. It is recommended rather to perform an index rebuild or index compaction (online) on all tables which have had large amounts of data purged in order to convert Index Delete blocks to the appropriate Free Chain.
A view of the current chains can be obtained by running a chain analysis report :
$ proutil dbname -C chanalys
The Create and Toss limits can be manipulated (since OpenEdge 10.0B) as can the Space search algorithm (
-recspacesearchdepth). Changing the default values necessitate in depth knowledge of the Application usage and further monitoring to tune.
While the transaction that caused the record deletion is still active (or a higher level transaction if this was a sub-transaction) the space previously used by the record will not be reused. This is to ensure that the record can be restored during a rollback of the transaction should it fail to complete. So new records or updates will not be able to take advantage of the space freed by the delete until the transaction has committed successfully.
When objects share Type II Storage Areas, mass deletions of one database object will not benefit other objects. In Type I Areas the blocks added to the Free Chain after deletion, can be reused by other objects as well, i.e. records or indexes or LOBs. In a Type II Area the space freed will only be reused by the
same object until that object is deleted from or moved to another Storage Area.
When the records being created are all greater than the size of 1 block of the database, then the space on the RM Chain cannot be re-used, since the remaining space is less than or equal to the size of 1 block. This will cause Progress to grab blocks from the Free Chain to load records. When the number of Free Chain blocks becomes low, the database will extend to grab more blocks from the Empty Chain, which will cause the database to grow as the High Water Mark in that Storage Area is then raised as a consequence. Bear in mind, that a 100MB record deletion, does not equate to 100MB/databaseblocksize available blocks on the RM chain. The records may well have been fragmented in the first place.
The "
proutil <dbname> -C dbanalys" and "
prostrct statistics <dbname>" utilities are useful reports to get information about how well or poorly space is being utilized.
