Salesforce

DBTOOL Option 4 vs DBTOOL Option 3 reports discrepancies with the number of fields in a table

« Go Back

Information

 
TitleDBTOOL Option 4 vs DBTOOL Option 3 reports discrepancies with the number of fields in a table
URL NameP162223
Article Number000130604
EnvironmentProduct: Progress
Version: 9.1D06, 9.1E
Product: OpenEdge
Version: 10.x, 11.x, 12.x
OS: All supported platforms
Other: Database, DBTOOL
Question/Problem Description
What does DBTOOL: Record Version Validation mean?

DBTOOL Option 4 (Record Version Validation) vs DBTOOL Option 3 (Record Validation) when discrepancies with the number of fields in a table are reported

DBTOOL Option 4 and DBTOOL Option 3 report discrepancies with the number of fields in a table

Number of fields mismatch.  Expected #, got #
There were 1 discrepancies found in <dbkey> of tablename (tbl#).

Is there record corruption when DBTOOL Record Version Validation reports Number of fields mismatch discrepancies?

How does DBTOOL determine the current Schema Version of a record ?

Can DBTOOL Option 4 be used enforce a schema update to the new Schema Version across all existing records?

Does the "Record format errors found after upgrade: 0" message mean that DBTOOL has updated all records to the latest Schema Version?

Steps to Reproduce
Clarifying Information
Error Message
Defect Number
Enhancement Number
Cause
Resolution

A. Understanding the output of DBTOOL Option 4 "Record Version Validation":

Total records read: 1118

The total number of records found for the table referenced including the table schema template record itself, which is used for the comparison in the first run.

The total number or records are then reported in the next message:

Discrepancies found before record upgrade: 1117

After the first comparison, DBTOOL Record Version Validation updates the records read to the latest Schema Version in memory and does a new validation against the current record schema template. DBTOOL does not commit the records updated to the current Schema Version. This option simply tests if a schema upgrade will succeed in future when an out of date record is next accessed by a client. The result is reported in the final message:

Record format errors found after upgrade: 0

The fields mismatch messages are reported for each record read which does not match the current Schema Version of the table:
Number of fields mismatch. Expected 22, got 19
There were 1 discrepancies found in <recid> of <tablename>(<tablenum>).
  • These messages do not indicate record level corruption.  This is expected behavior after a schema update. 
For example, where the table being referenced has had fields that were added or purged from its schema definition. 
  • Fields may still exist in the database record even though they are no longer seen or new fields may not yet be added to the record in question.
  • There is nothing wrong with the record, just that its schema is out of date with the current Schema Version
  • The OpenEdge fast schema update feature will handle updating these records to the current schema's table template at runtime for the client request behind the scenes. This is done for performance reasons when Schema Versioning was introduced in Progress 9. For further information on the technique of this feature enhancement refer to Article What are fast schema change differences between Progress 8.x and 9.x?  

B. Understanding the output of  "Number of fields mismatch" errors specifically in DBTOOL Option 3: Record Validation

The same "Number of fields mismatch" messages are reported in both DBTOOL Options 3 & 4, but these discrepancies are not included in the Errors count in Option 3.  There is a subtle but very important difference between these two DBTOOL Record reports:
  • Option 4, Record Version Validation: Tests record validation before and after upgrading the schema version of a record in memory.
  • Option 3, Record Validation: Compares the Physical Storage of the record to the Schema Definition and reports errors which need to be fixed.  As such,  Record Validation will report additional errors about the physical record corruption which are recorded along with the fields mismatch message. These should not be ignored, they are indicative of record corruption which does need to be fixed.
Example: Option 3, Record Validation
 
doOption: Error - number of physical fields mismatch 19 in record version 0 but 22 found in rec <recid> of table <tablename>(<tablenum>)
ERROR - skip tables expected offset 0x3f01 found 0x138

Number of fields mismatch.  Expected 22, got 19
doPrefix: Number of physical fields mismatch 19 in record version 0 but 22 found in rec of table <tablenum>

There were 2 error(s) found in <recid> of <tablename>(<tablenum>).

C. Is there any way to view the current Schema Version?

  • No the current schema version cannot be queried.
  • Record version identity is completely transparent to the ABL/4GL layer, it cannot access the Schema Version of a record or table.
  • This information is held at the database level, only the database layers (and sql92 in Progress 9.x and above) see the "version identity" of a physical record.

D. Can Schema Versioning be turned off or the new Schema Version enforced on all records when changed? 

  • No, the Schema Version Update feature cannot be turned off.
  • This would be prior Progress 8 behavior where a schema change would potentially take a long time and cause large bi sizes as a result (one transaction all or nothing change).
While unnecessary, if every record (related to the table with schema change) needs to be updated to the latest version, then one of the following methods could be considered:

Method 1: A 'dummy' update to force a records Schema Version Update

By making any small change to the a record and reverting that update the record is passed back (with the updated record schema version) to be written to disc.
FOR EACH Customer:
   balance = balance + 0.1.
END.

FOR EACH Customer:
  balance = balance - 0.1.
END.

Method 2: Tablemove 

A tablemove operation to a new Storage Area would create these records again in the new Storage Area using the latest table Schema Template.

Method 3: Dump and Load

 A dump (delete) and load of all records would load the records against the latest table Schema Template.
Workaround
Notes
Example of errors in dbtool output file after running Dbtool option 4 (Record Version Validation):

Number of fields mismatch.  Expected 18, got 17
There were 1 discrepancies found in 795 of tablename (1501).
Number of fields mismatch.  Expected 28, got 21
There were 1 discrepancies found in 909 of tablename (1242).

Total records read: 1118
Discrepancies found before record upgrade: 1117
Record format errors found after upgrade: 0
Keyword Phrase
Last Modified Date12/24/2021 2:11 PM

Powered by