Database corruption caused by improper protrace generation on Linux. Errors 9214 9212 9216 605.

Database fails to start after normal shutdown with 9214 9212 9216 605
Offline PROBKUP fails with 9214 9212 9216 605
Single User access fails with Creation date mismatch.
Cannot access database due to timestamp mismatch.
uname -a output for Sysname, Nodename, Release, Version is around or larger than the 80 character mark

Database shutdown normally without error
Online PROBKUP suceeded during uptime
During uptime, a processes crashed generating a PROTRACE file
PROTRACE files created during database uptime are empty, apart from the Progress header.
The the wrong creation date year is cannot be accounted for in the database history, viz 1987

 

This is a defect related to the length of the uname -a output for Sysname, Nodename, Release, Version (related to Linux only).  

• Due to this defect, when a session on Linux crashes and creates a protrace, during the protrace creation due to wrong file descriptor resulting from a buffer overflowing, the remainder of the stack gets written (it overwrites, or could overwrite) to any file on the system, including a database extent.
• The first line of the protrace file is in the protrace file, but the remainder of the file might be written to a database extent, corrupting the database.
• A 'cat' against the database extent referred in the 9212 message with wrong time stamp, will show up the content of a protrace.
• Similarly, forcing a protrace with proGetStack or kill -10 results in the same corruption. The difference being a runtime crash condition where we automatically generate a protrace and one where a protrace file is explicitly generated. For further information refer to Article:  Incomplete protrace on Linux due to the length of the hostname  

Upgrade to OpenEdge 11.3.3, 11.4.0 or later where the handling of hostnames when protraces are generated has been reworked.

To initially recover from the corruption, the database will have to be restored using site DR procedures. (restore backup | AI roll forward | Replication Target)


 

Ensure that the Linux OS uname -snrvm output is shorter than 80 characters (eg <= 79).

uname --snrvm is the concatenation of : kernel name, nodename, release, version, and machine  

As the hostname is part of the output for uname -snrvm and the hostname is often artificially long because the domain is part of the hostname (as seen from the output of the hostname call), removing the domain from the hostname can workaround the issue.

If FQDN is used as a part of hostname, split it into "hostname" and "dnsdomainname" in /etc/sysconfig/network , eg:
...
instead of:

Progress Articles:

Cannot start proserve, procopy, prodb or prorest a database when hostname exceeds 28 characters on Linux   
proGetStack causes memory leak in AppServer   
proGetStack or kill creates an incomplete protrace on Linux   
 ABL client hangs. Protrace shows malloc function making subcall to malloc on Linux.