The history of VST changes by versions - Forum - OpenEdge General - Progress Community

The history of VST changes by versions

 Forum

The history of VST changes by versions

  • Off-topic question: how to differentiate the undocumented information and the information that Progress does not like to share publicly?

    For example, there are the 4GL functions that are not KEYWORD'ed. Is it done intentionally?

  • The side result of VST investigation was a collecting information about schema table changes. The database schema is not changing in the service packs – only the major Progress versions are allowed to change database schema. Column “Change” in the Excel file contains the values: “new”, “dropped” or the number of fields added (or dropped) to a table in the given release. Zero can mean the changes of field types (e.g. integer -> int64).

    Update: the uploaded files were moved to the first post.

  • The values in _Database-Feature records:

    10.0A _Database-Feature[-16427]
    
    From  To     # Feature Name                    Ver   Command to enable/disable a feature
    ----- ----- -- ------------------------------- ----- -----------------------------------
    10.0A 10.1A  0 V100A_STORAGE
    10.0A 10.1A  1 FATHOM_REPLICATION              9.1D  proutil -C enablesitereplication / disablesitereplication
    10.1A 10.1A  2 FAILOVER_CLUSTERS               10.0A _dbutil prostrct cluster enable / disable
    10.0A 10.1A  3 PEER_DIRECT                     10.0A proutil -C enablepdr / disablepdr
    10.0a 10.1A  4 *** Invalid Feature ***
    10.0A 10.1A  5 LARGE_FILES                     9.1C  proutil -C enablelargefiles
    10.1A 10.1A  6 AUDITING                        10.1A proutil -C enableauditing / disableauditing
    10.1A 10.1A  7 AI_ARCHIVER                     10.1A rfutil  -C aiarchiver enable
    
    10.1B        1 OpenEdge Replication            10.1B proutil -C enableoerepl / disableoerepl  a.k.a. 9.1D enablesitereplication / disablesitereplication
    10.1B        2 Failover Clusters               10.0A _dbutil prostrct cluster enable / disable
    10.1B        3 DataXtend Remote Edition        10.1B proutil -C enableDXRE / disableDXRE
    10.1B        4 Reserved
    10.1B        5 Large Files                     9.1C  proutil -C enablelargefiles
    10.1B        6 Database Auditing               10.1A proutil -C enableauditing / disableauditing
    10.1B        7 JTA                             10.1A proutil -C enablejta / disablejta
    11.7         8 After Image Management/Archiver 10.1A aiarchiver enable / disable
    10.1B        9 64 Bit DBKEYS
    10.1B       10 Large Keys                      10.1B proutil -C enablelargekeys (for 4K or 8K dbblocksize)
    10.1B       11 64 Bit Sequences                10.1B proutil -C enableseq64
    10.1B       12 DataXtend Integration Edition   10.1B proutil -C enableDXIE / disableDXIE
    10.2B       13 Encryption                      10.2B proutil -C enableencryption / disableencryption
    11.0        14 Multi-tenancy                   11.0  proutil -C enablemultitenancy / disablemultitenancy
    11.2  11.7  15 Concurrent JTA and Replication
    11.4        16 Reserved
    11.3  11.7  17 MT Index Rebuild                11.3  proutil -C enablemtidxbld / disablemtidxbld (retired since 12.0)
    11.3  11.7  18 MT Data Move                    11.3  proutil -C dbrestrict datamove enable / disable
    11.3        19 Roll Forward Restricted Mode    11.3  proutil -C dbrestrict rollforward enable / disable
    11.4  11.7  20 TP Index Rebuild                11.4  proutil -C enabletpidxbld / disabletpidxbld (retired since 12.0)
    11.4        21 Table Partitioning              11.4  proutil -C enabletablepartitioning / disabletablepartitioning
    11.5  11.7  22 Read-only Partitions            11.5  proutil -C enablereadonlypartitions / disablereadonlypartitions (retired since 12.0)
    11.5  11.7  23 New VST Tables                  11.5  proutil -C enablenewvsttables / disablenewvsttables (retired since 12.0)
    11.5  11.5  24 TP Partition Move                     proutil -C enablepartitionmove / disablepartitionmove (was not implemented)
    11.6        25 Partition Copy                  11.6  proutil -C dbrestrict partitioncopy enable / disable
    11.6        26 Authentication GATEWAY          11.6  proutil -C enableauthgateway / disableauthgateway
    11.6        27 Change Data Capture             11.6  proutil -C enableCDC / disableCDC (since 11.7)
    11.7        28 Backup Counter Extension        11.7  proutil -C enablebackupcounterextension / disablebackupcounterextension (mb_bkincr_HIGH, mb_buseq_HIGH)

    The features that seemed to be not implemented:

    15 Concurrent JTA and Replication
    24 TP Partition Move

    The features that are retired since 12.0:

    17 MT Index Rebuild
    20 TP Index Rebuild
    22 Read-only Partitions
    23 New VST Tables

    "Save Key Events" feature is not reported by _Database-Feature but reported by proutil -C describe.

    Since V10.1B the enablekeyevents command creates the _KeyEvt table.
    Proutil -C disablekeyevents does not remove the _KeyEvt table from metaschema.
    4GL sessions are unable to check if the "Save Key Events" feature is currently enabled.

    _Database-Feature VST gets data from the fields in Master Block:

    mb_fmaDBContent (8 bytes) Active  feature list
    mb_fmaDBEnabled (8 bytes) Enabled feature list

    Old features are still stored as the single-byte fields in Master Block but these values are not used except mb_SaveKeyEvents:

    mb_largeFilesEnabled  enablelargefiles
    mb_replEnabled        enablesitereplication / disablesitereplication
    mb_pdrEnabled         enableDXRE / disableDXRE
    mb_clustEnabled       procluster enable / disable
    mb_XAsupportEnabled   enablejta / disablejta

    "Large Keys" feature is duplicated in _Db._Db-res1[1]

  • Since V11 Progress by default calculates what seems to be an alternate (SQL) CRC: _File._Fil-res1[1]
    It’s an addition to 4GL CRC: _File._CRC.

    Article: Format of field _File._Fil-res1 is too short for values in OpenEdge 11
    https://knowledgebase.progress.com/articles/Article/000043719

    FOR EACH _File:
      DISPLAY 
        _File._Tbl-Type
        _File._File-Name
        _File._Fil-res1[1] FORMAT ">>>>>>9"
        _File._CRC.
    END.

    Both values are evenly distributed in the range from 0 to 32K.

    Both are not applied to SQL Views (_Tbl-Type = “V”).
    Both are changing if we are modifying a table structure.
    But they are not equal.

    I guess these CRCs are calculated based on the different sets of field properties. For example, 4GL CRC uses 4GL data type (_Field._Data-Type) while SQL CRC uses SQL data type (_Field._Fetch-Type). Is it correct?

    Can we use SQL CRC to track the changes of the tables that are ignored by 4GL CRC? Or do they always change synchronously? For example, if we change a field type from “integer” to “int64” then SQL type will be automatically changed from “integer” to “bigint”. So both CRCs are changed.

  • > On Dec 16, 2019, at 12:57 PM, George Potemkin wrote:

    >

    > Since V11 Progress by default calculates what seems to be an alternate (SQL) CRC

    dont know about this particular item, but historically, various _fil-res*Star schema items (and simlarly named ones in other schema tables) have been used by the openedge dataserver code.

    the openedge sql server does not care about default 4gl display formats.

  • Gus, can I upload here your "Metadata" (Engine Crew Monograph Number 17) in pdf format? It's sad that nowadays we can find on the Internet only the shadows of these monographs.

  • > various _fil-res* schema items (and simlarly named ones in other schema tables) have been used by the openedge dataserver code.

    _Db._Db-res1[1] stores the "Large Keys Support" flag, I guess, since proutil -C enablelargekeys was introduced in 10.1B 

    Update: enablelargefiles command was mentioned by mistake

  • > On Dec 17, 2019, at 10:06 AM, George Potemkin wrote:

    >

    > _Db._Db-res1[1] stores the "Large Keys Support" flag

    these "res" fields were put in so that if we needed to add something we could do it without a database version change. at times we used them for things like the large keys flag. the intent was that in the next major release, they would be replaced by properly named fields. this did not always happen.

    the dataservers were orginally on a different release cycle than the main product so they used a lot of the "res" fields for their own purposes. a given field might be used for one thing by the oracle dataserver and something else by the sybase dataserver.

    later we added the feature bitmaps so we could add new things without waiting for the next (major) release that included a database version change.

  • > On Dec 17, 2019, at 3:07 AM, George Potemkin wrote:

    >

    > Gus, can I upload here your "Metadata" (Engine Crew Monograph Number 17) in pdf format?

    yes. feel free to post that and any of the others you wish.

    i also have a newer metadata document. it isn't quite up to date since i wrote it in the 10.2 time frame.

  • All files are moved to the first post.

  • New finding:

    Since V12.0 proutil -C dbauthkey does not update _Db._Db-revision (integer) anymore. Now it updates _Db._Db-auth-key added in V12.0.

    The _Db-auth-key value looks like: oakb1::qKgHqEYTQ3AXhvXquJN0bW+Zz2WEfOQffDqngiCO9EMD7Pad2D+TN/rc5hIrEze

    The message is changed as well:

    Before 12.0: Database authorization key changed to <newkey>. (1973)

    Since 12.0: Database authorization key has been changed successfully. (19415)