Any tools for cross-platform backup-restore (off of HP-UX Itanium)? - Forum - OpenEdge RDBMS - Progress Community

Any tools for cross-platform backup-restore (off of HP-UX Itanium)?

 Forum

Any tools for cross-platform backup-restore (off of HP-UX Itanium)?

This question is answered

Are there any third party tools that allow Progress customers to do a cross-platform "backup"/restore?  The intermediate "backup" file(s) would not need to be a standard format.  They could be in any format (perhaps simply consisting of a package of regular OE dump files).  The reason I ask is because Progress has left a big gap where this backup/restore feature should exist.  I was hoping that somebody had something to fill the gap.  (I'm thinking of vendors like riverside, roundtable, bravepoint, or any other vendor).

Another obvious option is to dump/load but that is soooo much work.  It involves numerous and very error-prone steps to be done manually by the DBA.  Moreover it is very time consuming from the a DBA's point of view and, depending on how good the DBA is, the process can under-utilize the available hardware (ie. CPU and disk can remain largely unused on either the source machine, target machine, or both if the DBA is busy organizing their manual work, or they are neglecting to operate on multiple tables concurrently).  If hardware is under-utilized then it can cause the outage window to be far longer than it needs to be.

Long ago we had asked Progress about a product of theirs, "PRO D&L", but they said that wasn't used for the same purpose (ie. simply automating a dump/load).  I think that product was for hooking into replication triggers and allowing a cross platform migration to be performed over several days without any database outages.  We are looking for a much more *simple* (but fast) operation that moves data from one platform to another during an outage window that is as short as possible.

I found an enhancement idea that is a couple years old now, and was wondering if this might be completed before HP-UX comes to its end-of-life date in 2025.

https://community.progress.com/community_groups/products_enhancements/i/openedge_database_enhancements_-_tell_us_what_youd_like_to_see/permit_cross-platform_database_restore

It will be nice to move out of HP-UX one day.  We'd go to another Progress platform like Windows or Linux.  But the prospect of doing the dump and load is not something we look forward to.  Long ago we did these dump/loads on a yearly basis, but these days we are a bit out of practice.

Speaking of migrating HP-UX databases, I thought I would point out that there was once a time when we ran the OE database on HP-UX PA-RISC.  When that architecture stopped being supported by Progress, they were able to waive a magical wand and allowed the OE database to be migrated from PA-RISC to Itanium *without* requiring a dump and load (see KB: https://knowledgebase.progress.com/articles/Article/P132598 ).   Is it reasonable to hope that they will do a similar thing when the time comes once again to stop supporting Progress on Itanium?  (Eg. OE 12 no longer supports Itanium)  

It would be really nice if Progress would consider doing all the heavy lifting for us.  Using their dump/load process is not a very happy prospect at all.

Verified Answer
  • HPUX on PA-RISC is Big Endian.  So is AIX and Solaris.  HPUX on Itanium is Little Endian, but when we did the initial port of HPUX to Itanium, the HPUX engineer suggested that we make the database Big Endian so the migration would be easier and since the Itanium chips were much faster any performance penalty for byte-swapping would be hidden by the faster speed.  This is why you can backup/restore between HPUX on PA-RISC and Itanium.

    Linux and Windows are Little Endian, so hence you must dump (to ascii or binary) to move from any Big Endian machine to Linux or Windows.

    Given modern day CPU speeds, if we were to start over, we would probably pick Little Endian and make databases compatible between all platforms, however there are always edge cases, where a customer would  require every last bit of CPU speed and they would not be happy with this compatibly cost since they likely would not be changing platforms.

    In the end, currently, Little Endian is taking over the world.  Windows and Linux are the largest growth platforms for deploying OpenEdge applications.  With HPUX going away and the decline of Solaris in deployment the database deployments will end up being AIX (Big Endian), Linux (Little Endian) and Windows (Little Endian).  

    It would not be too difficult to package up a migration tool that does the binary dump/load df export, user export, sequence export, etc to do this for you.  But if you want it to make effective use of the hardware at hand, then a generic tool becomes much harder.

    Mike
    -- 
    Mike Furgal
    Director – Database and Pro2 Services
    PROGRESS Bravepoint
    617-803-2870 


  • The only adequate solution today for migrating large databases from one platform to another is Pro2Pro.  it's really not difficult, and, most importantly, efficiently without long downtime.

    IMHO, something else, most likely, will not be implemented taking into account various architectural constraints.

  • Entire dump / create / load can be scripted quite easily with pct.

  • Hi:

    The Pro2 product can be purchased and implemented yourself, however 99% of the implementation are done by the Progress PS team.  If you wanted to use Pro2 strictly for a D/L purpose, then if you engage with Progress PS, we can do this for you where we include the “rental” of the software as part of the fee for performing the D/L.  Otherwise you would need to purchase the product.

    The Pro2Pro process is:
     - Install the software
     - Configure it
     - Add the triggers
     - Bulkload (This is the D/L portion)

    At cutover time, we just need to make sure the replqueue is empty, then remove the plumbing field (prrowid) from each table and the index on that field.  This is less than 1 hour of downtime regardless of the size of the database.

    If you are interested, contact me or your sales rep for a quote.

    Mike
    -- 
    Mike Furgal
    Director – Database and Pro2 Services
    PROGRESS Bravepoint
    617-803-2870 


  • I agree with Tom completely on this.  The MDBA business I run we dump and load a customer pretty much every weekend.  So we have canned scripts to start with.  If downtime is limited, then we start tweaking the scripts.   If the tweaked scripts still causes too much downtime, that is when we go down the Pro2Pro path for the dump and load.  We are probably dong a Pro2Pro Dump and Load every 6 weeks on average.

    Mike
    -- 
    Mike Furgal
    Director – Database and Pro2 Services
    PROGRESS Bravepoint
    617-803-2870 


All Replies
  • HPUX on PA-RISC is Big Endian.  So is AIX and Solaris.  HPUX on Itanium is Little Endian, but when we did the initial port of HPUX to Itanium, the HPUX engineer suggested that we make the database Big Endian so the migration would be easier and since the Itanium chips were much faster any performance penalty for byte-swapping would be hidden by the faster speed.  This is why you can backup/restore between HPUX on PA-RISC and Itanium.

    Linux and Windows are Little Endian, so hence you must dump (to ascii or binary) to move from any Big Endian machine to Linux or Windows.

    Given modern day CPU speeds, if we were to start over, we would probably pick Little Endian and make databases compatible between all platforms, however there are always edge cases, where a customer would  require every last bit of CPU speed and they would not be happy with this compatibly cost since they likely would not be changing platforms.

    In the end, currently, Little Endian is taking over the world.  Windows and Linux are the largest growth platforms for deploying OpenEdge applications.  With HPUX going away and the decline of Solaris in deployment the database deployments will end up being AIX (Big Endian), Linux (Little Endian) and Windows (Little Endian).  

    It would not be too difficult to package up a migration tool that does the binary dump/load df export, user export, sequence export, etc to do this for you.  But if you want it to make effective use of the hardware at hand, then a generic tool becomes much harder.

    Mike
    -- 
    Mike Furgal
    Director – Database and Pro2 Services
    PROGRESS Bravepoint
    617-803-2870 


  • One of the ways to read Mike's response is that scripting this process to suit your particular equipment, time frames, etc. is more tedious than difficult for someone with experience, so one approach is simply to hire an appropriate consultant to build an occasion specific tool.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • We had been looking at this issue for over a year and we just took the plunge on migrating off HPUX Itanium to Linux X86 last month.

    First let me agree with Thomas hire the appropriate consultant.

    The first shot at doing the migration took 10+ hours, after working their magic they got the complete migration down to just over 3.5 hours. They also reworked the database, tables, indexes, extents, file sizes as well as a whole mess of other ‘old practices’ which needed to be changed. The performance boost on the DB has been significant, and we are still working though some issues we found post conversion.

    When you choose to migrate hire someone who has done the project before.  It’s money well spent.

  • Given the lack of a shrink-wrapped software solution, we may have to hire a consultant.

    IMHO, a cross-platform backup/restore should not be so challenging as this.  I disagree that this is an "occasion-specific" requirement.  For development purposes I would have *frequently* used a backup-restore feature that would copy a database from HP-UX to my PDSOE machine (on Windows).   That would have been used dozens of times over the years, had it been easy, fast, and not so very error-prone.  My Windows workstation has a large SSD and a 4.2 GHz processor.  On the other hand, our HP-UX/Itanium hardware that we use for development is pretty outdated, with NAS for disk, and machinfo says the cpu is an Itanium  Processor 9350 (1.73 GHz, 24 MB).  The database is probably 10-20% slower than it would be on Windows, even for average CRUD operations.

    There is already an idea in the "product enhancements" forum that has quite a lot of votes, relatively speaking.   It is about two years old.  I suspect those are votes from people who would also do backup/restore on a regular basis.

    It sounds like there is an opportunity for someone to create a tool to facilitate the migration of a database away from HP-UX.  I think a lot of Progress customers will be ready to make the switch once they learn that OE 12 won't be certified to run on HP-UX.  (But most of them may wait until after 2025, if I had to guess).  I would bet that even the consultants would rather buy a software tool for doing these migrations - rather than wrestle with the current dump/reload procedures.  A software tool might decrease their billable hours, but it would otherwise be a much better option.  It would avoid a ton of manual efforts, and home-grown scripts.

    As intimidating as it is to contemplate the dump/reload, there was even a suggestion from a team member to abandon the whole idea.  Instead, we could force our clients just start over with empty databases on a new platform!

    I wish that Progress would announce a plan for HP-UX customers who need to move their databases.  The HP-UX platform has reached the end of the road, but I'm guessing that Progress wants to keep those customers just the same (albeit on a different platform).  It would be unfortunate to lose some of the customers altogether (possibly to another database vendor).  Doesn't it make sense to give us a well-defined strategy for moving our OE databases?  Must we really fend for ourselves?  

  • It might be unfamiliar at first, but once a D&L process has been run a couple times it is fairly painless.  

    When I have been involved in moving from HPUX Itanium to Linux, the database migration was just a small part of the effort.  The custom code, interfaces, printing, ...  were more of an issue.

  • The only adequate solution today for migrating large databases from one platform to another is Pro2Pro.  it's really not difficult, and, most importantly, efficiently without long downtime.

    IMHO, something else, most likely, will not be implemented taking into account various architectural constraints.

  • Entire dump / create / load can be scripted quite easily with pct.

  • Maybe you can use CDC (Change data capture).

    - It can be activated online

    - Handled on a very Low level (Almost no performance cost)

    - You can handle the changes on separate area's

    Then you can synchronize every change marked with a "change-sequence"id to the other database. (Your own Implementation)

    I think it's pretty straightforward to implement it. But never did it before of course :)

  • I didn't realize that pct would have all the dump/load options for data.  I know that ABL compilation would require the schema, but not necessarily the data.  Thanks for the pointer.  This may be a good reason in itself to start learning pct.

    Thanks for pointing out the option of using pro2pro.  I will look into that as well. I'm wondering if it can be purchased on a temporary basis (eg as a subscription) since we might not need it anymore after our migration is complete.  Does anyone know if there is a trial download available for pro2pro, so I can investigate the feasibility of doing a full database migration using that product?  Also, I'm curious if pro2pro does some type of initialization or "seeding" of the target database in some way (along with the installation of the long-term replication infrastructure).

    Pro2pro uses CDC internally once the database is upgraded, right?  That is my understanding.  (I think the database needs to be up to 11.7.x or so.)  We already use our replication triggers so I'm not that keen on using a product that may compete for use of them.

  • Hi:

    The Pro2 product can be purchased and implemented yourself, however 99% of the implementation are done by the Progress PS team.  If you wanted to use Pro2 strictly for a D/L purpose, then if you engage with Progress PS, we can do this for you where we include the “rental” of the software as part of the fee for performing the D/L.  Otherwise you would need to purchase the product.

    The Pro2Pro process is:
     - Install the software
     - Configure it
     - Add the triggers
     - Bulkload (This is the D/L portion)

    At cutover time, we just need to make sure the replqueue is empty, then remove the plumbing field (prrowid) from each table and the index on that field.  This is less than 1 hour of downtime regardless of the size of the database.

    If you are interested, contact me or your sales rep for a quote.

    Mike
    -- 
    Mike Furgal
    Director – Database and Pro2 Services
    PROGRESS Bravepoint
    617-803-2870 


  • Thanks Mike, I will definitely talk to our salesrep.  Previously when I had heard about this option the only part that seemed unappealing (other than the cost) is the fact that pro2 would start hijacking and sharing database triggers.  

    But my understanding is that newer versions of pro2 should not need to interfere with ABL triggers since they use CDC, right?  And, similarly, my understanding is that newer versions of pro2 would work regardless of whether our database CRUD is happening in ABL or SQL92, right?

  • Hi,

    >> is that newer versions of pro2 should not need to interfere with ABL triggers since they use CDC, right?

    Yes you are right. But you must have either a separate license for the OE CDC, or an Advanced Enterprise RDBMS license.

    >>is that newer versions of pro2 would work regardless of whether our database CRUD is happening in ABL or SQL92, right?

    Yes, to track changes made through SQL, you can. This requires the OE CDC license. In this case, you do not need to do anything else on your part.

    Without CDС, it is necessary to implement java-triggers in the source database.

  • >  machinfo says the cpu is an Itanium  Processor 9350 (1.73 GHz, 24 MB).  The database is probably 10-20% slower than it would be on Windows, even for average CRUD operations.

    You are vastly underestimating the performance benefit of migration.  Even a run of the mill Wintel server will be 3x to 5x faster for any server-centric operations.  Obviously stuff that has to go over the network will not get all of those benefits and operations where the server side of things is a small percentage of the overall execution will only benefit by the pro-rated percentage that the server side stuff represents.  Single record CRUD operations on user screens you probably won't notice nearly as much of an improvement.  But operations like large inquiries & reports or any batch/app server stuff that runs on the server will be crazy faster than it was.  Users will probably think it is broken.

    --
    Tom Bascom
    tom@wss.com

  • A platform neutral backup & restore would be great.

    Meanwhile...

    Dumping and loading is very scriptable and most of us consultants have a set of base scripts that we use over and over.  So, in a sense, we already have the tooling that you are looking for.

    The particular scripting does often need to be tweaked to fit the circumstances - especially if you have a home-built or highly customized system or other special requirements.  But once it has been done it is very repeatable.

    Taking the Pro2 "minimal downtime" approach is sometimes sensible, sometimes not.  It really depends on how big your db is, how much downtime you can actually afford and how often you think you are going to do this kind of thing.

    --
    Tom Bascom
    tom@wss.com

  • One point of Clarity.  Pro2 Version 5 includes a CDC license for use with Pro2 only.

    Mike
    -- 
    Mike Furgal
    Director – Database and Pro2 Services
    PROGRESS Bravepoint
    617-803-2870