Binary Load - Super Slow on 11.7.2 - Forum - OpenEdge RDBMS - Progress Community

Binary Load - Super Slow on 11.7.2

 Forum

Binary Load - Super Slow on 11.7.2

  • Libor is right.  I can't believe I didn't see that...

    (Of course that means that you should have a good backup first.  Or be willing to throw away the target.)  

    --
    Tom Bascom
    tom@wss.com

  • The Furgal Test:

    [root@hssmiplive-ind:ahmed]# /apps/mipusr/protop/bin/syncio.sh

    Procopy session begin for root on batch. (451)

    Database copied from /apps/dlc117/sports. (1365)

    Procopy session end. (334)

    OpenEdge Release 11.7.2 as of Tue Oct 24 18:20:59 EDT 2017

    Before-image cluster size set to 16384 kb. (1620)

    -rw-r--r-- 1 root root 8192 Mar  8 11:20 sports.b1

    Please wait...

    real    0m18.161s

    9 seconds =  10MB/sec -- which is barely acceptable, anything longer and your disk susbsystem is junk.

    5 seconds =  20MB/sec -- "ok"

    3 seconds =  30MB/sec -- "good"

    1 second  = 100MB/sec -- which is "excellent!"

  • So it is also inconsistent.  Or getting worse under load as the day goes on.

    --
    Tom Bascom
    tom@wss.com

  • > So it is also inconsistent.

    It's consistent:

    First run - 18.387 sec

    Run as syncio.sh - 18.161s.

    But 5 MB/sec is very slow.

  • I ran the dump and load using the Dictionary Dump, Bulk load and Index Re-Build and here are my results on the same DB and same server and same Disk Sub System.

    Dump - 15:36 - 17:47 - 2 Hours 11 Minutes.

    Load - 17:58 - 19:26 - 1 hour 28 Minutes

    Index Rebuild - 19:45 -20:31 - 46 Minutes

    Can somebody explain the difference in time for the Binary Load that took 11 Hours 49 Minutes compare to the Bulk Load that took 1 Hour 28 Minutes.?

  • Bulkload by uses –i.  Binary load does not, but –i can be specified.  If you use –i on binary load, how long does it take?

    Mike
    -- 
    Mike Furgal
    Director – Database and Pro2 Services
    PROGRESS
    617-803-2870 


  • Is your bulk load test using a new DB or did you procopy empty into the pre-grown shell of the first test? There may be a cost to growing the data files.

    Did you try the binary load with -r ?  -r gives you the benefits of -i (FS cached writes) without some of the drawbacks.

    Paul Koufalis
    White Star Software

    pk@wss.com
    @oeDBA (https://twitter.com/oeDBA)

    ProTop: The #1 Free OpenEdge DB Monitoring Tool
    http://protop.wss.com
  • @Mike - I will need to re-run the Binary Dump and Load again. Will try this now. Will run the load with -i and -r. I am using the same DB with the same data so this gives me consistency.

    @Paul - I created new DB's and the extents grew as I ran the load. But this could not have an impact on Binary Load of 11 Hours 49 Mins compared to Bulk Load of 1 Hour 28 Mins.

    My opinion is it seems to be a Progress 11.7.2 issue.!!

  • wrote that bulk load runs with -i so that seems to be the most likely explanation for the discrepancy.

    If you have plenty of time, I think that running a dictionary load with all indexes deactivated should simulate a bulk load without the -i.

    In summary, these are the various combinations:

    1. Binary load w/out -i / -r: 12 hours

    2. Binary load w/ -r: TO DO

    3. Bulk load (default w/ -i): 2h

    4. Bulk load w/out -i (simulated by dict load with inactive indexes): TO DO

    Paul Koufalis
    White Star Software

    pk@wss.com
    @oeDBA (https://twitter.com/oeDBA)

    ProTop: The #1 Free OpenEdge DB Monitoring Tool
    http://protop.wss.com
  • I added the -i -r option on the binary load and i have the following results.

    Binary Dump - 25 Minutes

    Binary Load - 50 minutes - Time down from 11 Hours 49 Minutes to 50 Minutes.

    Index Rebuild - 52 minutes

  • That makes a whole lot more sense!

    Paul Koufalis
    White Star Software

    pk@wss.com
    @oeDBA (https://twitter.com/oeDBA)

    ProTop: The #1 Free OpenEdge DB Monitoring Tool
    http://protop.wss.com
  • Thank you all for all your input. It has been a day of learning.