Salesforce

Errors (1028) and (8999), or (4554) occur during rfutil roll forward with endtime or endtrans

Printable View
« Go Back

Information

 
TitleErrors (1028) and (8999), or (4554) occur during rfutil roll forward with endtime or endtrans
URL Name000041245
Article Number000117300
EnvironmentProduct: Progress
Version: 9.x
Product: OpenEdge
Version: 10.x, 11 to 11.5.x
OS: All supported platforms
Other: RFUTIL
Question/Problem Description
Errors 1028 and 8999, or 4554 occur during RFUTIL rollforward with endtime or endtrans.
RFUTIL rollforward with endtime or endtrans fails with errors 1028 and 8999, or 4554.

Rollforward scripts process all AI files in a directory expecting that RFUTIL will only process AI extents that are appropriate to the endtime or endtrans provided.
Rolling forward with ENDTIME fails with error 4554 attempting to apply an AI extent that was created after the endtime.
Rolling forward with ENDTIME fails with error 1028 and 8999 attempting to apply an AI extent that was created after the endtime.

The problem scenario: RFUTIL rollforward with the endtime or endtrans option processes AI notes from the AI extent that follows the one that reached the endtime.
  • The endtime or endtrans specified was reached at the middle of AI extend a1. 
  • RFUTIL correctly stops processing transaction notes in the middle of a1 when endtime or endtrans is reached.
Roll Forward to Mon Aug 16 14:56:00 2021 has been located, further roll forward operations are invalid. (17923)
  • The rollforward script then processes the next AI file in the list provided. 
  • It is expected that RFUTIL should ignore this AI file due to the LAST AIMAGE NEW date within the extent being after the endtime or endtrans.
  • However RFUTIL processes these transaction notes.  
  • The roll forward operation has has in effect skipped half of .a1 and then starts to process a2 transaction notes.
  • Subsequent AI files then fail with:
Expected ai file number <n> but file specified is <n+2> in sequence. (8019) 

This causes two different types of errors when the next ai file is processed by rollforward:
rlaiextrd: Transaction table mismatch ptran: 0 note 1. (4554)

OR, when a transaction spans the AI extents, the BKUPDCTR and note updctr in error 1028 can be substantially different:
SYSTEM ERROR: Rollforward Apply AI note, BKUPDCTR=315, note updctr=318. (1028)
area 10, dbkey 512, record type 34 (8999)
Steps to Reproduce
Clarifying Information
RFUTIL rollforward without endtime or endtrans succeeds without error.

Example command line:
rfutil dbname -C roll forward endtime 2015:06:01:07:30:00 -a dbname.$i
rfutil dbname -C roll forward endtrans 30377 -a dbname.$i
Error Messagerlaiextrd: Transaction table mismatch ptran: 0 note 1. (4554)
SYSTEM ERROR: Rollforward Apply AI note, BKUPDCTR=315, note updctr=318. (1028)
area 10, dbkey 512, record type 34 (8999)
Defect NumberDefect PSC00259992 / OE00239082
Enhancement Number
Cause
RFUTIL roll forward does not prevent additional ai files from being applied when the endtrans or endtime has been reached.  

As a consequence, the database shared memory version of the transaction table finds that there are x live transactions (ptran) and the AI file's INMEM note's version of the table finds a different value, this check fails as they must be in agreement. Database corruption results in the hotspare database.
Resolution
Upgrade to OpenEdge 11.6.0 or later, where the next rollforward is prevented.

The fix involves identifying that the endtime or endtrans has been reached in whichever AI file it was found (even when the -ailist parameter since 11.3 is used) and further roll forwards are invalidated.  

The database recognizes that additional roll forwards are not allowed, by not incrementing the aiseq number up by 1 when endtime or endtrans are reached.  The next attempt to roll forward with the following ai file will be rejected.
$  rfutil dbname -C roll forward oplock [ endtime | endtrans ] -ailist alist.txt -MemCheck -DbCheck verbose -B n -r
$ echo y | rfutil dbname -C roll opunlock -crStatus 10 -crTXDisplay
Workaround
Start the roll forward operation again. Only roll forward all extents up to the one that the endtime time or endtrans number is within and no further.

Find the ENDTIME or ENDTRANS details needed from the AI extent:
$  rfutil dbname -C aimage scan -ailist listofais.txt
$  rfutil dbname -C aimage scan verbose -a dbname.aX
  1. For ENDTIME rollforward: Only use AI extents whose LAST AIMAGE NEW date is less than the endtime used when rolling forward ai files.
  2. For ENDTRANS rollforward:  Only use AI extents whose RL_TEND is less than the endtrans value used when rolling forward ai files. 

Example: Rolling forward to endtime 17 August 2021 @ 07:30:00

RFUTIL -C aimage scan shows:

dbname.a1: Last AIMAGE NEW Tue Aug 17 07:25:09 2021 (1641)
dbname.a2: Last AIMAGE NEW Tue Aug 17 07:30:57 2021 (1641)
dbname.a3: Last AIMAGE NEW Tue Aug 17 07:32:01 2021 (1641)

Stop rolling forward after the first ai file:
$ rfutil dbname -C roll forward endtime 2021:08:17:07:30:00 -a dbname.a1

Example: Rolling forward to transaction 30377

RFUTIL -C aimage scan verbose shows:

dbname.a1: 
Trid: 30373 code = RL_TEND version = 1 (12528)
Trid: 30374 code = RL_TEND version = 1 (12528)

dbname.a2: 
Trid: 30375 code = RL_TEND version = 1 (12528)
Trid: 30376 code = RL_TEND version = 1 (12528)
Trid: 30377 code = RL_TEND version = 1 (12528)

dbname.a3: 
Trid: 30378 code = RL_TEND version = 1 (12528)
Trid: 30379 code = RL_TEND version = 1 (12528)

Stop rolling forward after the second ai file: (endtrans is +1 of the actual transaction number needed).
$ rfutil dbname -C roll forward endtrans 30378 -a dbname.a1
$ rfutil dbname -C roll forward endtrans 30378 -a dbname.a2
Notes
Keyword Phrase
Last Modified Date8/17/2021 10:31 AM
Powered by