Salesforce

Roll forward fails with different sets of errors each PROREST baseline

« Go Back

Information

 
TitleRoll forward fails with different sets of errors each PROREST baseline
URL Name000046795
Article Number000169917
EnvironmentProduct: OpenEdge
Version: All supported versions
OS: All supported platforms
Other: Database, RFUTIL rollfoward
Question/Problem Description
Roll forward fails with varied errors each new PROREST baseline.
Some AI files are applied successfully, then start failing.
Roll forward retry fails with error 1028 again with the same BKUPDCTR and updctr numbers
BKUPDCTR and updctr values do not increase with subsequent roll forward retry.
No errors reported when restoring the new hotspare baseline.
Steps to Reproduce
Clarifying Information
OpenEdge version used to roll forward is the same OS, version and SP and bit-level as production.
PROSTRCT REPAIR run with a .st file pointing to the correct restored location.
3rd Party Software not accessing database files.
IDXCHECK / IDXFIX does not show index corruption on production.
Error MessageSYSTEM ERROR: Note leading and trailing counts don't match 86 -4 (1029)
SYSTEM ERROR: Rollforward Apply AI note, BKUPDCTR=74324347, note updctr=74422957. (1028)
SYSTEM ERROR: Roll forward of file <ai file name> did not complete normally (1046)
SYSTEM ERROR: End of last note read is at file offset <num> (1048)

Incomplete ai file detected, expected <blocks>, got <blocks>. (9333)

Extent <file-name> has a different last opened date <timestamp>, (9213)

Corrupt block detected when reading from database. (4229)
SYSTEM ERROR: wrong dbkey in block. Found 0, should be 165221312 in area 7 (1124)
Corrupt block detected when attempting to release a buffer. (4232)
Defect Number
Enhancement Number
Cause
When rolling forward a specific ai.n, the open database code does an index block clean up and writes two notes into the bi file of the backup database:
  1. One note updates the block and increments the update counter, for example, from 1211 to 1212.
  2. The other note updates the master block with a new time stamp.
The new time stamp then failed the roll forward (ai.n) because after it opened this ai.n, it found the time stamp stored in that file didn't match the new time stamp in the master block.
 
When the roll forward retry of the same ai.n was then executed, the retry operation ignored the time check, read notes from the AI file then applied each of these after the retry point. Once it reached the block mentioned in the 1124 errors, it found that the block update counter was already 1212, and this then triggered the (1028) error.
 
There is either a hardware problem this hotspare database is running on, evident from the fact that there are no corruption at source.
 
And/Or there's a problem with the AI file transfer, evident by the 1029 and 9333 errors in addition to the 1028 roll forward error:
SYSTEM ERROR: Note leading and trailing counts don't match (1029) 
Incomplete ai file detected, expected <blocks>, got <blocks>. (9333) 
SYSTEM ERROR: Rollforward Apply AI note, BKUPDCTR=<number>, note updctr=<number>. (1028) 
Resolution
  1. Consider using another machine for the hostpare database while investigating the current hotspare environment. 
  2. Ensure the hotspare machine is clean (memory/disks etc) 
  3. Investigate the AI file transfer routine between production and hotspare includes file transfer checks (like md5checksum for example). 
     
Workaround
Notes
Keyword Phrase
Last Modified Date2/3/2017 10:33 AM

Powered by