Generated protrace contains no information about a _sqlsrv2 crash
protrace incomplete after a _sqlsrv2 crash with no C-stack or SQL diagnostics to understand why the SQLSRV2 remote server dies

PROTRACE reports the epoch start time not the time it was generated

 

ulimit -c unlimited or ulimit -c 2048
protrace files are complete for ABL and _mprosrv remote server processes

On Solaris each connected client will get:

•  1 protrace file + 1 protrace file for the main thread, and 
•  1 protrace file for the Signal Handling thread of that Server. 

Example: Dead _sqlsrv2 process 621 had a user 17674 that dies during a microtransaction:

SRV 621: (8873) Login usernum 17674, remote SQL client.
...
BROKER 0: (1153) BROKER detects death of server 13754
WDOG 2991: (2526) Disconnecting client 17674 of dead server 621.
WDOG 2991: (2256) SYSTEM ERROR: User 17674 died during microtransaction.

• The user 17674 is a SQL client and connected to remote server 621 .
• The WDOG process detects that server 621 is dead and tries to clean it from shared-memory
• When disconnecting the orphan user 17674, their context had died during a microtransaction
• Since shared-memory cannot be safely cleaned, the database triggers an abnormal shutdown
• A protrace for the _sqlsrv2 process (PID 13754) is created printing the epoch time and otherwise the content is empty
• PROTRACE.13754 for the _sqlsrv2 process contains no information about the OpenEdge version, no command-line argument, no startup parameters, no C-stack or SQL diagnostics

=============================================
PROGRESS stack trace as of Thu Jan 01 00:00:00 1970U

command line not initialized
=============================================

When generating the C-stack unsafe async thread signal corrupts the stack causes the database engine to crash 

1. epoch time - is printed due to the uttime() call on big endian systems (Solaris) with 64bit time_t ,which leads to a the "timeholder" variable holding a value of 0 for some codepaths. It does not cause crashes or other unsafe behaviour.
2. empty protrace - the stack tracing mechanism is not thread safe. This is only for database processes that use dbut_uttime() as part of the C stack generation [ _sqlsrv2, rpagent, rpserver, rputil, dbtool ]. Other database executables (_mprsrv) handle SIGUSR1 through ABL protrace generation which don't have this issue.

Upgrade to OpenEdge 11.7.6, 12.2.0 or later where protraces created by database processes has been fixed.

• The timestamp in protrace file reports the correct time
• Changes to the Solaris C stack generation use Solaris APIs getcontext() and walkcontext(), which are guaranteed to be thread safe and async signal safe. 

Progress Articles:
AIX protrace files sometimes don't contain full call stack for remote Server processes
Can KILL -SIGUSR1 cause a process to crash?