Creating a LOB affects other clients performance:The reason that transaction processing against the database slows down, is because the LOB create is I/O bound.
When clients are connected through a tiered architecture, the agent / session can only handle one lob insert at a time which blocks other requests is coming from the throttle of the AppServer managing connections.
In addition, LOBS are more likely to pollute the MRU/LRU chain which affects other user's performance in having to re-read their blocks into bufers.
Creating a LOB uses the record subsystem:A LOB is saved in (almost) the same format as a regular record. It's not block based. The LOB data are cut (by the AVM runtime) into fragments that are stored separately in the same manner as normal data records are,
where each slice is stored as a record fragment. When the LOB value is larger than the database blocksize,
it will be fragmented across multiple blocks which will all need to be read to retrieve the data with a LONGCHAR or MEMPTR.
The amount of work the record manager is performing is exactly the same operation by operation to load the lob (with and without integrity). The reduction in performance is due to writing to the bi subsystem, raising the high-water mark and extending storage areas and record consolidation.
Creating a LOB is slower than Dumping a LOB:
- COPY-LOB to a file, dumps contents of a LOB to a file as a read operation and shared latched when assembled for reading. There are no bi notes on a read operation.
- COPY-LOB from a file, inserts the contents as a record write operation and as such bi notes are needed.
Creating a CLOB is slower than a BLOB:A CLOB takes more work to retrieve than a simple VARCHAR or LONGCHAR value due to code page conversion.
Equally sized CLOBs are slower than BLOBs, as:
- CLOBs are character based instead of bytes based;
- Characters can be in different encodings.
- CLOBS need to support more complicated operations like copy, truncate, and overlay (akin to a text editor that can support multiple languages, ccToUTF8)
The contention is not on TXE latching:
- TXE acquire/release are only in the rm layer for create and update as there are many record segments being created to populate the lob record.
- TXE acquire/release latches are held for the duration of the segmented lob creation and then released. They are not held until transaction commit
Creating BI notes needed for the LOB create with integrity (no -i) :Running reliable means completely logging all activity that goes into creating a lob object. A significant number of notes need to be written for:
- RM CHAIN manipulation
- The creation/update of record fragments (how lob segments are stored),
- Increasing the High Water Mark.
- All Physical and Physiological notes are logged.
- All purely Logical notes are written.
With a BIW running performance is slightly worse because the BIW is competing with the user for the same latch.
Creating BI notes needed for the LOB create with no integrity (-i) :There is a huge difference between note activity when LOBS are created with no-integrity (specifying -i) as there is a much lower degree of recovery:
- Only physiological notes are logged, which are significantly fewer.
- Most purely logical notes are not written (savepoints and logical record operation markers).
Creating a LOB in 11.5 or laterThe bi notes pre and post 11.5 are the same, there were minor code changes to support
partitionId in OpenEdge 11.5 and 11.6 which make creating notes more expensive.
OpenEdge 11.7.6, 12.1, bi cluster optimization was reworked to improve adding new clusters to the bi chain, which can improve performance at checkpoint time:
Performance improvement when new biclusters are added