Mismatch between SourceDB and TargetDB by 500K ? - Forum - OpenEdge Pro2 - Progress Community

Mismatch between SourceDB and TargetDB by 500K ?

 Forum

Mismatch between SourceDB and TargetDB by 500K ?

This question is not answered

Hi All 

we started a Bullkload Proccess on a Table with 70 million Records which took about 7 days plus to complete,  but when the Bulk upload completed there is a difference of about 500K records missing from Target\SQL server, 

I hoped when we enabled the Table Triggers this would autmatically catch up\replicate the missiing 500K which after 4 days its only provided the Table Trigger updates, so the questions I have are....

1. why\how did this occur, is this beacuse we didn't enable the Database Triggers at the same time as the Bulkload ? and is this something we should do for furture tables ?

2. The missing 500K is probably the difference between the Bulkload starting and the 7 days it took to finish,  so that would imply that the Record count is taken at the start of the bulkload ? so everythin after that should be an update ? and should the Table trigger not have transferred the 500K ? 

3. There also in the "REPLQUEUE" Table 18,000 records that have all got the "Bulk-copy-err" status,  When I have tried to the bulkload for the Same Table,  it returns the "Bulkload for the this table is complete"  How do I move or load these records from PRO2 into the Target Database ?

4. the most important,  how do we rectify the missing 500K, would stating a Bulkload again for that table be a risk ? as it is 70 millions records in the table I would not want to have truncate the Target Table and start again from scratch especially just for 500K records

Many Thanks in Advance

Ade 

   

All Replies
  • 1. why\how did this occur, is this beacuse we didn't enable the Database Triggers at the same time as the Bulkload ? and is this something we should do for future tables ?

    Yes. This might be the reason for out of sync. The right sequence is, enable database triggers and then bulk-load the tables. Once bulk-load completes then turn on the replication threads to process the updates in the replqueue.

    2. The missing 500K is probably the difference between the Bulkload starting and the 7 days it took to finish,  so that would imply that the Record count is taken at the start of the bulkload ? so everythin after that should be an update ? and should the Table trigger not have transferred the 500K ? 

    This 500k is the difference between the time-gap of enable trigger vs bulk-load. More the gap among them will cause more records to be out of sync.

    3. the most important,  how do we rectify this, would stating a Bulkload again for that table be a risk ? as it is 70 millions records in the table I would not want to have truncate the Target Table and start again from scratch especially just for 500K records

    I am not aware of any solution to rectify this other than bulk-load of that particular table. However, if you can know those 500k rowid’s then you can rectify by creating replqueue records for those 500k records.
     
    Hope this Helps, Satya