10.2B08 Windows 2008R2: Database not started complete and appserver (every process) hungs which tries to connect - Forum - OpenEdge RDBMS - Progress Community

10.2B08 Windows 2008R2: Database not started complete and appserver (every process) hungs which tries to connect

 Forum

10.2B08 Windows 2008R2: Database not started complete and appserver (every process) hungs which tries to connect

  • Hello,

    databases are started by _mprosrv.

    If they are started by script at boot (adminserver -> appserver -> starts db and other appserver) _mprosrv and _proapsv are visible but somes db's never accepts connections.

    Wed Mar 12 08:46:49 2014
    [2014/03/12@08:46:49.466+0100] P-4192       T-4292  I BROKER  0: (333)   Multi-user session begin.
    [2014/03/12@08:46:49.901+0100] P-4192       T-4292  I BROKER  0: (5326)  Begin Physical Redo Phase at 0 .
    [2014/03/12@08:46:52.307+0100] P-4192       T-4292  I BROKER  0: (7161)  Physical Redo Phase Completed at blk 119 off 3366 upd 388.
    [2014/03/12@08:46:52.308+0100] P-4192       T-4292  I BROKER  0: (13547) At end of Physical redo, transaction table size is 128.
    [2014/03/12@08:46:52.325+0100] P-4192       T-4292  I BROKER  0: (452)   Login by winsure on batch.
    [2014/03/12@08:46:52.326+0100] P-4192       T-4292  I BROKER  0: (4234)  Progress OpenEdge Release 10.2B build 1907 SP08 on WINNT .
    [2014/03/12@08:46:52.327+0100] P-4192       T-4292  I BROKER  0: (4281)  Server started by winsure on batch.
    [2014/03/12@08:46:52.328+0100] P-4192       T-4292  I BROKER  0: (-----) Started using pid: 4192.
    [2014/03/12@08:46:52.329+0100] P-4192       T-4292  I BROKER  0: (4235)  Physical Database Name (-db): C:\ProgramData\Progress\eNora\winsureuk\winprint\db\2014\wp2014.
    [2014/03/12@08:46:52.331+0100] P-4192       T-4292  I BROKER  0: (4236)  Database Type (-dt): PROGRESS.
    [2014/03/12@08:46:52.332+0100] P-4192       T-4292  I BROKER  0: (4237)  Force Access (-F): Not Enabled.
    [2014/03/12@08:46:52.332+0100] P-4192       T-4292  I BROKER  0: (4238)  Direct I/O (-directio): Not Enabled.
    [2014/03/12@08:46:52.334+0100] P-4192       T-4292  I BROKER  0: (-----) LRU mechanism enabled.
    [2014/03/12@08:46:52.335+0100] P-4192       T-4292  I BROKER  0: (-----) Number of LRU force skips (-lruskips): 0
    [2014/03/12@08:46:52.336+0100] P-4192       T-4292  I BROKER  0: (-----) Number of LRU2 force skips (-lru2skips): 0
    [2014/03/12@08:46:52.336+0100] P-4192       T-4292  I BROKER  0: (4239)  Number of Database Buffers (-B): 3000.
    [2014/03/12@08:46:52.337+0100] P-4192       T-4292  I BROKER  0: (-----) Number of Alternate Database Buffers (-B2): 0.
    [2014/03/12@08:46:52.338+0100] P-4192       T-4292  I BROKER  0: (9422)  Maximum private buffers per user (-Bpmax): 64.
    [2014/03/12@08:46:52.339+0100] P-4192       T-4292  I BROKER  0: (4240)  Excess Shared Memory Size (-Mxs): 23.
    [2014/03/12@08:46:52.340+0100] P-4192       T-4292  I BROKER  0: (10014) The shared memory segment is not locked in memory.
    [2014/03/12@08:46:52.340+0100] P-4192       T-4292  I BROKER  0: (4241)  Current Size of Lock Table (-L): 8192.
    [2014/03/12@08:46:52.341+0100] P-4192       T-4292  I BROKER  0: (13953) Maximum Area Number (-maxArea): 32000.
    [2014/03/12@08:46:52.341+0100] P-4192       T-4292  I BROKER  0: (4242)  Hash Table Entries (-hash): 887.
    [2014/03/12@08:46:52.342+0100] P-4192       T-4292  I BROKER  0: (4243)  Current Spin Lock Tries (-spin): 24000.
    [2014/03/12@08:46:52.342+0100] P-4192       T-4292  I BROKER  0: (6526)  Number of Semaphore Sets (-semsets): 3.
    [2014/03/12@08:46:52.343+0100] P-4192       T-4292  I BROKER  0: (13924) Maximum Shared Memory Segment Size (-shmsegsize) 128 Mb.
    [2014/03/12@08:46:52.343+0100] P-4192       T-4292  I BROKER  0: (4244)  Crash Recovery (-i): Enabled.
    [2014/03/12@08:46:52.344+0100] P-4192       T-4292  I BROKER  0: (6573)  Database Blocksize (-blocksize): 8192.
    [2014/03/12@08:46:52.344+0100] P-4192       T-4292  I BROKER  0: (4245)  Delay of Before-Image Flush (-Mf): 3.
    [2014/03/12@08:46:52.345+0100] P-4192       T-4292  I BROKER  0: (4247)  Before-Image File I/O (-r -R): Reliable.
    [2014/03/12@08:46:52.349+0100] P-4192       T-4292  I BROKER  0: (4249)  Before-Image Truncate Interval (-G): 0.
    [2014/03/12@08:46:52.350+0100] P-4192       T-4292  I BROKER  0: (4250)  Before-Image Cluster Size: 524288.
    [2014/03/12@08:46:52.350+0100] P-4192       T-4292  I BROKER  0: (4251)  Before-Image Block Size: 8192.
    [2014/03/12@08:46:52.351+0100] P-4192       T-4292  I BROKER  0: (4252)  Number of Before-Image Buffers (-bibufs): 20.
    [2014/03/12@08:46:52.351+0100] P-4192       T-4292  I BROKER  0: (-----) Record free chain search depth factor 5 (-recspacesearchdepth)
    [2014/03/12@08:46:52.352+0100] P-4192       T-4292  I BROKER  0: (9238)  BI File Threshold size (-bithold): 0.0   Bytes.
    [2014/03/12@08:46:52.353+0100] P-4192       T-4292  I BROKER  0: (6552)  BI File Threshold Stall (-bistall): Disabled.
    [2014/03/12@08:46:52.354+0100] P-4192       T-4292  I BROKER  0: (4254)  After-Image Stall (-aistall): Not Enabled.
    [2014/03/12@08:46:52.354+0100] P-4192       T-4292  I BROKER  0: (4255)  After-Image Block Size: 8192.
    [2014/03/12@08:46:52.355+0100] P-4192       T-4292  I BROKER  0: (4256)  Number of After-Image Buffers (-aibufs): 20.
    [2014/03/12@08:46:52.356+0100] P-4192       T-4292  I BROKER  0: (8527)  Storage object cache size (-omsize): 1024
    [2014/03/12@08:46:52.356+0100] P-4192       T-4292  I BROKER  0: (4257)  Maximum Number of Clients Per Server (-Ma): 5.
    [2014/03/12@08:46:52.357+0100] P-4192       T-4292  I BROKER  0: (4258)  Maximum Number of Servers (-Mn): 5.
    [2014/03/12@08:46:52.359+0100] P-4192       T-4292  I BROKER  0: (4259)  Minimum Clients Per Server (-Mi): 1.
    [2014/03/12@08:46:52.360+0100] P-4192       T-4292  I BROKER  0: (-----) Server network message wait time (-Nmsgwait): 2
    [2014/03/12@08:46:52.360+0100] P-4192       T-4292  I BROKER  0: (-----) Use pollset mechanism for client/server (-pollset): Disabled
    [2014/03/12@08:46:52.361+0100] P-4192       T-4292  I BROKER  0: (-----) Delay first prefetch message (-prefetchDelay): Disabled
    [2014/03/12@08:46:52.361+0100] P-4192       T-4292  I BROKER  0: (-----) Prefetch message fill percentage (-prefetchFactor): 0
    [2014/03/12@08:46:52.362+0100] P-4192       T-4292  I BROKER  0: (-----) Minimum records in prefetch ms (-prefetchNumRecs): 16
    [2014/03/12@08:46:52.362+0100] P-4192       T-4292  I BROKER  0: (-----) Suspension queue poll priority (-prefetchPriority): 0
    [2014/03/12@08:46:52.363+0100] P-4192       T-4292  I BROKER  0: (4260)  Maximum Number of Users (-n): 21.
    [2014/03/12@08:46:52.363+0100] P-4192       T-4292  I BROKER  0: (4261)  Host Name (-H): Not Enabled.
    [2014/03/12@08:46:52.364+0100] P-4192       T-4292  I BROKER  0: (4262)  Service Name (-S): Not Enabled.
    [2014/03/12@08:46:52.364+0100] P-4192       T-4292  I BROKER  0: (4263)  Network Type (-N): Not Enabled.
    [2014/03/12@08:46:52.406+0100] P-4192       T-4292  I BROKER  0: (4264)  Character Set (-cpinternal): ISO8859-1.
    [2014/03/12@08:46:52.406+0100] P-4192       T-4292  I BROKER  0: (4282)  Parameter File: Not Enabled.
    [2014/03/12@08:46:52.407+0100] P-4192       T-4292  I BROKER  0: (9426)  Large database file access has been enabled.
    [2014/03/12@08:46:52.407+0100] P-4192       T-4292  I BROKER  0: (9336)  Created shared memory with segment_id: 39190528
    [2014/03/12@08:46:52.407+0100] P-4192       T-4292  I BROKER  0: (12813) Allowed index cursors (-c): 84.
    [2014/03/12@08:46:52.408+0100] P-4192       T-4292  I BROKER  0: (12814) Group delay (-groupdelay): 10.
    [2014/03/12@08:46:52.408+0100] P-4192       T-4292  I BROKER  0: (12815) Lock table hash table size (-lkhash): 1237
    [2014/03/12@08:46:52.408+0100] P-4192       T-4292  I BROKER  0: (12818) Message Buffer Size (-Mm): 1024
    [2014/03/12@08:46:52.408+0100] P-4192       T-4292  I BROKER  0: (12821) Use muxlatches (-mux): 1
    [2014/03/12@08:46:52.409+0100] P-4192       T-4292  I BROKER  0: (12823) Semaphore Sets (-semsets): 3
    [2014/03/12@08:46:52.409+0100] P-4192       T-4292  I BROKER  0: (13870) Database Service Manager - IPC Queue Size (-pica) : 64.0  KBytes.
    [2014/03/12@08:46:52.409+0100] P-4192       T-4292  I BROKER  0: (13896) TXE Commit lock skip limit (-TXESkipLimit): 10000.
    [2014/03/12@08:46:52.410+0100] P-4192       T-4292  I BROKER  0: (15219) Encryption enabled: 0
    [2014/03/12@08:46:52.410+0100] P-4192       T-4292  I BROKER  0: (15218) Encryption cache size (-ecsize): 1000

     

    Then database log finished, at least this


    [2014/03/12@11:44:58.576+0100] P-12728      T-12564 I BROKER  0: (10471) Database connections have been enabled.

    is missing.


    If i try proshut on this "nearly started" db it hungs, mpro hangs too, like the appserver which tries to connect the database through shared memory.

    Any suggestions?

  • This sounds very similar to a bug that is being worked on in development as we speak.
    In that situation, there is a remote server that was spawned by the broker prior to completion of database initialization that resulted in a dead lock situation (an OpenEdge bug).
    Identifying and killing the hung remote server allowed the broker to complete the startup processing.
     
    This might work in your situation as well.
     
    _________________________________
    Richard Banville
    Fellow, OpenEdge Development

    PROGRESS SOFTWARE CORPORATION

    14 Oak Park | Bedford, MA 01730 | USA
    DIRECT  +1 781 280 4875
    richb@progress.com
     
     
    From: Stefan Marquardt [mailto:bounce-Marquardt@community.progress.com]
    Sent: Wednesday, March 12, 2014 8:38 AM
    To: TU.OE.RDBMS@community.progress.com
    Subject: 10.2B08 Windows 2008R2: Database not started complete and appserver (every process) hungs which tries to connect this databases
     
    Thread created by Stefan Marquardt

    Hello,

    databases are started by _mprosrv.

    If they are started by script at boot (adminserver -> appserver -> starts db and other appserver) _mprosrv and _proapsv are visible but somes db's never accepts connections.

    Wed Mar 12 08:46:49 2014
    [2014/03/12@08:46:49.466+0100] P-4192       T-4292  I BROKER  0: (333)   Multi-user session begin.
    [2014/03/12@08:46:49.901+0100] P-4192       T-4292  I BROKER  0: (5326)  Begin Physical Redo Phase at 0 .
    [2014/03/12@08:46:52.307+0100] P-4192       T-4292  I BROKER  0: (7161)  Physical Redo Phase Completed at blk 119 off 3366 upd 388.
    [2014/03/12@08:46:52.308+0100] P-4192       T-4292  I BROKER  0: (13547) At end of Physical redo, transaction table size is 128.
    [2014/03/12@08:46:52.325+0100] P-4192       T-4292  I BROKER  0: (452)   Login by winsure on batch.
    [2014/03/12@08:46:52.326+0100] P-4192       T-4292  I BROKER  0: (4234)  Progress OpenEdge Release 10.2B build 1907 SP08 on WINNT .
    [2014/03/12@08:46:52.327+0100] P-4192       T-4292  I BROKER  0: (4281)  Server started by winsure on batch.
    [2014/03/12@08:46:52.328+0100] P-4192       T-4292  I BROKER  0: (-----) Started using pid: 4192.
    [2014/03/12@08:46:52.329+0100] P-4192       T-4292  I BROKER  0: (4235)  Physical Database Name (-db): C:\ProgramData\Progress\eNora\winsureuk\winprint\db\2014\wp2014.
    [2014/03/12@08:46:52.331+0100] P-4192       T-4292  I BROKER  0: (4236)  Database Type (-dt): PROGRESS.
    [2014/03/12@08:46:52.332+0100] P-4192       T-4292  I BROKER  0: (4237)  Force Access (-F): Not Enabled.
    [2014/03/12@08:46:52.332+0100] P-4192       T-4292  I BROKER  0: (4238)  Direct I/O (-directio): Not Enabled.
    [2014/03/12@08:46:52.334+0100] P-4192       T-4292  I BROKER  0: (-----) LRU mechanism enabled.
    [2014/03/12@08:46:52.335+0100] P-4192       T-4292  I BROKER  0: (-----) Number of LRU force skips (-lruskips): 0
    [2014/03/12@08:46:52.336+0100] P-4192       T-4292  I BROKER  0: (-----) Number of LRU2 force skips (-lru2skips): 0
    [2014/03/12@08:46:52.336+0100] P-4192       T-4292  I BROKER  0: (4239)  Number of Database Buffers (-B): 3000.
    [2014/03/12@08:46:52.337+0100] P-4192       T-4292  I BROKER  0: (-----) Number of Alternate Database Buffers (-B2): 0.
    [2014/03/12@08:46:52.338+0100] P-4192       T-4292  I BROKER  0: (9422)  Maximum private buffers per user (-Bpmax): 64.
    [2014/03/12@08:46:52.339+0100] P-4192       T-4292  I BROKER  0: (4240)  Excess Shared Memory Size (-Mxs): 23.
    [2014/03/12@08:46:52.340+0100] P-4192       T-4292  I BROKER  0: (10014) The shared memory segment is not locked in memory.
    [2014/03/12@08:46:52.340+0100] P-4192       T-4292  I BROKER  0: (4241)  Current Size of Lock Table (-L): 8192.
    [2014/03/12@08:46:52.341+0100] P-4192       T-4292  I BROKER  0: (13953) Maximum Area Number (-maxArea): 32000.
    [2014/03/12@08:46:52.341+0100] P-4192       T-4292  I BROKER  0: (4242)  Hash Table Entries (-hash): 887.
    [2014/03/12@08:46:52.342+0100] P-4192       T-4292  I BROKER  0: (4243)  Current Spin Lock Tries (-spin): 24000.
    [2014/03/12@08:46:52.342+0100] P-4192       T-4292  I BROKER  0: (6526)  Number of Semaphore Sets (-semsets): 3.
    [2014/03/12@08:46:52.343+0100] P-4192       T-4292  I BROKER  0: (13924) Maximum Shared Memory Segment Size (-shmsegsize) 128 Mb.
    [2014/03/12@08:46:52.343+0100] P-4192       T-4292  I BROKER  0: (4244)  Crash Recovery (-i): Enabled.
    [2014/03/12@08:46:52.344+0100] P-4192       T-4292  I BROKER  0: (6573)  Database Blocksize (-blocksize): 8192.
    [2014/03/12@08:46:52.344+0100] P-4192       T-4292  I BROKER  0: (4245)  Delay of Before-Image Flush (-Mf): 3.
    [2014/03/12@08:46:52.345+0100] P-4192       T-4292  I BROKER  0: (4247)  Before-Image File I/O (-r -R): Reliable.
    [2014/03/12@08:46:52.349+0100] P-4192       T-4292  I BROKER  0: (4249)  Before-Image Truncate Interval (-G): 0.
    [2014/03/12@08:46:52.350+0100] P-4192       T-4292  I BROKER  0: (4250)  Before-Image Cluster Size: 524288.
    [2014/03/12@08:46:52.350+0100] P-4192       T-4292  I BROKER  0: (4251)  Before-Image Block Size: 8192.
    [2014/03/12@08:46:52.351+0100] P-4192       T-4292  I BROKER  0: (4252)  Number of Before-Image Buffers (-bibufs): 20.
    [2014/03/12@08:46:52.351+0100] P-4192       T-4292  I BROKER  0: (-----) Record free chain search depth factor 5 (-recspacesearchdepth)
    [2014/03/12@08:46:52.352+0100] P-4192       T-4292  I BROKER  0: (9238)  BI File Threshold size (-bithold): 0.0   Bytes.
    [2014/03/12@08:46:52.353+0100] P-4192       T-4292  I BROKER  0: (6552)  BI File Threshold Stall (-bistall): Disabled.
    [2014/03/12@08:46:52.354+0100] P-4192       T-4292  I BROKER  0: (4254)  After-Image Stall (-aistall): Not Enabled.
    [2014/03/12@08:46:52.354+0100] P-4192       T-4292  I BROKER  0: (4255)  After-Image Block Size: 8192.
    [2014/03/12@08:46:52.355+0100] P-4192       T-4292  I BROKER  0: (4256)  Number of After-Image Buffers (-aibufs): 20.
    [2014/03/12@08:46:52.356+0100] P-4192       T-4292  I BROKER  0: (8527)  Storage object cache size (-omsize): 1024
    [2014/03/12@08:46:52.356+0100] P-4192       T-4292  I BROKER  0: (4257)  Maximum Number of Clients Per Server (-Ma): 5.
    [2014/03/12@08:46:52.357+0100] P-4192       T-4292  I BROKER  0: (4258)  Maximum Number of Servers (-Mn): 5.
    [2014/03/12@08:46:52.359+0100] P-4192       T-4292  I BROKER  0: (4259)  Minimum Clients Per Server (-Mi): 1.
    [2014/03/12@08:46:52.360+0100] P-4192       T-4292  I BROKER  0: (-----) Server network message wait time (-Nmsgwait): 2
    [2014/03/12@08:46:52.360+0100] P-4192       T-4292  I BROKER  0: (-----) Use pollset mechanism for client/server (-pollset): Disabled
    [2014/03/12@08:46:52.361+0100] P-4192       T-4292  I BROKER  0: (-----) Delay first prefetch message (-prefetchDelay): Disabled
    [2014/03/12@08:46:52.361+0100] P-4192       T-4292  I BROKER  0: (-----) Prefetch message fill percentage (-prefetchFactor): 0
    [2014/03/12@08:46:52.362+0100] P-4192       T-4292  I BROKER  0: (-----) Minimum records in prefetch ms (-prefetchNumRecs): 16

  • Yes, it looks like a timing issue because the startup works if we do this one by one manually.

    Is it possible to get a hotfix for this for 10.2B08?

    BTW: With which version/SP this bug was "implemented"?

  • Yes, it is believed to be a timing issue during database initialization.  If the request to start a remote server are not made prior to completion of database initialization then the problem does not occur.
     
    The root cause and fix are still being worked on so I can’t answer questions 2 or 3.
     
     
    From: Stefan Marquardt [mailto:bounce-Marquardt@community.progress.com]
    Sent: Wednesday, March 12, 2014 8:55 AM
    To: TU.OE.RDBMS@community.progress.com
    Subject: RE: 10.2B08 Windows 2008R2: Database not started complete and appserver (every process) hungs which tries to connect
     
    Reply by Stefan Marquardt

    Yes, it looks like a timing issue because the startup works if we do this one by one manually.

    Is it possible to get a hotfix for this for 10.2B08?

    BTW: With which version/SP this bug was "implemented"?

    Stop receiving emails on this subject.

    Flag this post as spam/abuse.

  • In our case there should be no remote servers (-S not enabled) but perhaps we hit the same bug within other circumstances.

    How i can check on W2k8R2 what "our" _mprosrv is doing (dump/trace//debug/protrace)?

  • The usual was is progetstack <pid> but I don’t think that will work for a broker process.
    Outside of attaching with a debugger I’m not sure. Tech Support might have some other tricks.  It is worth contacting them for that as well as telling them you have this problem so it can be properly tracked
     
     
    From: Stefan Marquardt [mailto:bounce-Marquardt@community.progress.com]
    Sent: Wednesday, March 12, 2014 9:08 AM
    To: TU.OE.RDBMS@community.progress.com
    Subject: RE: 10.2B08 Windows 2008R2: Database not started complete and appserver (every process) hungs which tries to connect
     
    Reply by Stefan Marquardt

    In our case there should be no remote servers (-S not enabled) but perhaps we hit the same bug within other circumstances.

    How i can check on W2k8R2 what "our" _mprosrv is doing (dump/trace//debug/protrace)?

    Stop receiving emails on this subject.

    Flag this post as spam/abuse.

  • progetstack won't obtain C stack on Windows. adplus - support.microsoft.com/.../286350 or windbg - msdn.microsoft.com/.../ff551063(v=vs.85).aspx

    make sure you have pdb file available - _mprosrv.pdb

  • Where i can get the pdb files (10.2B08 win32)?

  • progress.thruinc.net/ExDn.aspx

  • Hi Richard,

    is there already a bug fix 10.2B08 available?

    Stefan

  • Now i have an environment where i can test all things, just restarted the server with automatically startup of database and appservers.

    The last line from db log is:

    [2014/10/14@09:53:13.865+0200] P-3672       T-3604  I BROKER  0: (12815) Lock table hash table size (-lkhash): 13063

    The next lines should be:

    [2014/09/15@10:28:30.098+0200] P-20324      T-9804  I BROKER  0: (12816) Maxport (-maxport): 5000

    [2014/09/15@10:28:30.099+0200] P-20324      T-9804  I BROKER  0: (12817) Minport (-minport): 3000

    but they are missing, _mprosrv stops working.

    Client connect hangs.

    After some struggling with windbg i got these stacks from __mprosrv:

    Thead 0:

    ntdll!ZwWriteFile+0x15

    kernel32!WriteFile+0x4a

    _mprosrv!bkWriteMessage+0x32

    _mprosrv!dbLogMessage+0x335

    _mprosrv!drFetch_DB+0x134

    _mprosrv!msgout+0x201

    _mprosrv!msgnCB+0xb9

    _mprosrv!msgCB+0x1d

    _mprosrv!dsmMsgnCallBack+0x59

    _mprosrv!dbContextWriteOptions+0x1568

    _mprosrv!dsmContextWriteOptions+0x20

    _mprosrv!doserve+0x73a

    _mprosrv!main+0xb2

    _mprosrv!__tmainCRTStartup+0x10f

    kernel32!BaseThreadInitThunk+0x12

    ntdll!RtlInitializeExceptionChain+0x63

    ntdll!RtlInitializeExceptionChain+0x36

    Thread 1:

    ntdll!ZwWaitForMultipleObjects+0x15

    kernel32!BaseThreadInitThunk+0x12

    ntdll!RtlInitializeExceptionChain+0x63

    ntdll!RtlInitializeExceptionChain+0x36

    Thread 2:

    ntdll!DbgBreakPoint

    kernel32!BaseThreadInitThunk+0x12

    ntdll!RtlInitializeExceptionChain+0x63

    ntdll!RtlInitializeExceptionChain+0x36

  • Additional info:

    I stops here and there (watched with WinTail - after reboot)), now it's:

    [2014/10/14@10:42:15.915+0200] P-3744       T-3648  I BROKER  0: (12814) Group delay (-groupdelay): 10.

    Perhaps it has to do with the filesystem blocks.

    procexp shows this:

    ntoskrnl.exe!KeWaitForMultipleObjects+0xc0a

    ntoskrnl.exe!KeAcquireSpinLockAtDpcLevel+0x732

    ntoskrnl.exe!KeWaitForMutexObject+0x19f

    ntoskrnl.exe!PoStartNextPowerIrp+0xba4

    ntoskrnl.exe!PoStartNextPowerIrp+0x1821

    ntoskrnl.exe!KeAcquireSpinLockAtDpcLevel+0x93d

    ntoskrnl.exe!KeWaitForMutexObject+0x19f

    ntoskrnl.exe!NtCreateFile+0x4cd

    ntoskrnl.exe!NtWriteFile+0x7e3

    ntoskrnl.exe!KeSynchronizeExecution+0x3a23

    wow64cpu.dll!TurboDispatchJumpAddressEnd+0x6c0

    wow64cpu.dll!TurboDispatchJumpAddressEnd+0x2ac

    wow64.dll!Wow64SystemServiceEx+0x1ce

    wow64.dll!Wow64LdrpInitialize+0x42a

    ntdll.dll!RtlUniform+0x6e6

    ntdll.dll!MD5Final+0x21e83

    ntdll.dll!LdrInitializeThunk+0xe

    ntdll.dll!ZwWriteFile+0x15

    kernel32.dll!WriteFile+0x4a

    _mprosrv.exe!dbut_utmalloc+0x1e2

    _mprosrv.exe!dbut_utmalloc+0x7e5

  • That's the stack from ProcessHacker:

    0, ntoskrnl.exe!KiDeliverApc+0x1c7

    1, ntoskrnl.exe!KiCommitThreadWait+0x3dd

    2, ntoskrnl.exe!KeWaitForSingleObject+0x19f

    3, ntoskrnl.exe!IopSynchronousServiceTail+0x2a9

    4, ntoskrnl.exe!NtWriteFile+0x7e2

    5, ntoskrnl.exe!KiSystemServiceCopyEnd+0x13

    6, wow64cpu.dll!CpupSyscallStub+0x9

    7, wow64cpu.dll!ReadWriteFileFault+0x31

    8, wow64.dll!RunCpuSimulation+0xa

    9, wow64.dll!Wow64LdrpInitialize+0x42a

    10, ntdll.dll!LdrpInitializeProcess+0x17e3

    11, ntdll.dll! ?? ::FNODOBFM::`string'+0x280c0

    12, ntdll.dll!LdrInitializeThunk+0xe

    13, ntdll.dll!ZwWriteFile+0x15

    14, KernelBase.dll!WriteFile+0x113

    15, kernel32.dll!WriteFileImplementation+0x76

    16, _mprosrv.exe!dbut_utmalloc+0x1e2

    17, _mprosrv.exe!dbut_utmalloc+0x7e5

  • Today I run a test with 11.5.1 (32bit)  on win2008 , still the same problem!

    Startup through the tool delivered by software supplier during reboot of the windows server.

    OE databaseserver ist not up and not down, something in the middle.

    pro says: multiuser, mpro hungs (endless)

    Log:

                   Fri Jun 19 08:43:19 2015

    [2015/06/19@08:43:19.215+0200] P-4868       T-4872  I BROKER  0: (333)   Multi-user session begin.

    [2015/06/19@08:43:19.219+0200] P-4868       T-4872  I BROKER  0: (5326)  Begin Physical Redo Phase at 64 .

    [2015/06/19@08:43:19.412+0200] P-4868       T-4872  I BROKER  0: (7161)  Physical Redo Phase Completed at blk 126 off 4789 upd 3064.

    [2015/06/19@08:43:19.412+0200] P-4868       T-4872  I BROKER  0: (13547) At end of Physical redo, transaction table size is 256.

    [2015/06/19@08:43:19.459+0200] P-4868       T-4872  I BROKER  0: (452)   Login by SYSTEM on batch.

    [2015/06/19@08:43:19.465+0200] P-4868       T-4872  I BROKER  0: (5644)  Started for 21010 using TCP IPV4 address 0.0.0.0, pid 4868.

    [2015/06/19@08:43:19.467+0200] P-4868       T-4872  I BROKER  0: (4234)  Progress OpenEdge Release 11.5 build 1190 SP01 on WINNT .

    [2015/06/19@08:43:19.467+0200] P-4868       T-4872  I BROKER  0: (4281)  Server started by SYSTEM on batch.

    [2015/06/19@08:43:19.468+0200] P-4868       T-4872  I BROKER  0: (6574)  Started using pid: 4868.

    [2015/06/19@08:43:19.469+0200] P-4868       T-4872  I BROKER  0: (9426)  Large database file access has been enabled.

    [2015/06/19@08:43:19.470+0200] P-4868       T-4872  I BROKER  0: (15219) Encryption enabled: 0

    [2015/06/19@08:43:19.471+0200] P-4868       T-4872  I BROKER  0: (15824) Multi-tenancy enabled: 0

    [2015/06/19@08:43:19.472+0200] P-4868       T-4872  I BROKER  0: (15824) Table Partitioning enabled: 0

    [2015/06/19@08:43:19.472+0200] P-4868       T-4872  I BROKER  0: (-----) LRU mechanism enabled.

    [2015/06/19@08:43:19.472+0200] P-4868       T-4872  I BROKER  0: (4282)  Parameter File: \\dexw4160\C$\ProgramData\Progress\eNora\winsure\swinsureN.pf.

    [2015/06/19@08:43:19.472+0200] P-4868       T-4872  I BROKER  0: (9336)  Created shared memory with segment_id: 40566784

    [2015/06/19@08:43:19.473+0200] P-4868       T-4872  I BROKER  0: (4250)  Before-Image Cluster Size: 524288.

    [2015/06/19@08:43:19.473+0200] P-4868       T-4872  I BROKER  0: (4251)  Before-Image Block Size: 16384.

    [2015/06/19@08:43:19.474+0200] P-4868       T-4872  I BROKER  0: (13873) After-image Management Archival Directory List (-aiarcdir): Not Enabled

    [2015/06/19@08:43:19.474+0200] P-4868       T-4872  I BROKER  0: (13874) Create After-image Management Archival Directory(s) (-aiarcdircreate): Not Enabled

    [2015/06/19@08:43:19.475+0200] P-4868       T-4872  I BROKER  0: (13872) After-image Management Archival Interval (-aiarcinterval): -1

    [2015/06/19@08:43:19.475+0200] P-4868       T-4872  I BROKER  0: (4256)  Number of After-Image Buffers (-aibufs): 20

    [2015/06/19@08:43:19.476+0200] P-4868       T-4872  I BROKER  0: (4254)  After-Image Stall (-aistall): Not Enabled

    [2015/06/19@08:43:19.477+0200] P-4868       T-4872  I BROKER  0: (17555) Starting index number for statistics range (-baseindex): 1

    [2015/06/19@08:43:19.477+0200] P-4868       T-4872  I BROKER  0: (17554) Starting table number for statistics range (-basetable): 1

    [2015/06/19@08:43:19.478+0200] P-4868       T-4872  I BROKER  0: (4252)  Number of Before-Image Buffers (-bibufs): 20

    [2015/06/19@08:43:19.478+0200] P-4868       T-4872  I BROKER  0: (6552)  BI File Threshold Stall (-bistall): Disabled.

    [2015/06/19@08:43:19.479+0200] P-4868       T-4872  I BROKER  0: (9238)  BI File Threshold size (-bithold): 0.0   Bytes

    [2015/06/19@08:43:19.479+0200] P-4868       T-4872  I BROKER  0: (6573)  Database Blocksize (-blocksize): 8192

    [2015/06/19@08:43:19.480+0200] P-4868       T-4872  I BROKER  0: (12812) BIW writer delay (-bwdelay): 0

    [2015/06/19@08:43:19.481+0200] P-4868       T-4872  I BROKER  0: (12813) Allowed index cursors (-c): 244

    [2015/06/19@08:43:19.482+0200] P-4868       T-4872  I BROKER  0: (12265) SSL Certificate Store Path (-certstorepath): Not Enabled

    [2015/06/19@08:43:19.483+0200] P-4868       T-4872  I BROKER  0: (4264)  Character Set (-cpinternal): ISO8859-1

    [2015/06/19@08:43:19.483+0200] P-4868       T-4872  I BROKER  0: (4235)  Physical Database Name (-db): C:\ProgramData\Progress\eNora\winsure\sure_db\winsure

    [2015/06/19@08:43:19.484+0200] P-4868       T-4872  I BROKER  0: (4238)  Direct I/O (-directio): Not Enabled

    [2015/06/19@08:43:19.485+0200] P-4868       T-4872  I BROKER  0: (4236)  Database Type (-dt): PROGRESS

    [2015/06/19@08:43:19.485+0200] P-4868       T-4872  I BROKER  0: (15218) Encryption cache size (-ecsize): 1000

    [2015/06/19@08:43:19.485+0200] P-4868       T-4872  I BROKER  0: (12814) Group delay (-groupdelay): 10

    [2015/06/19@08:43:19.486+0200] P-4868       T-4872  I BROKER  0: (4242)  Hash Table Entries (-hash): 6661

    [2015/06/19@08:43:19.488+0200] P-4868       T-4872  I BROKER  0: (4244)  Crash Recovery (-i): Enabled

    [2015/06/19@08:43:19.489+0200] P-4868       T-4872  I BROKER  0: (17557) Number of indexes included in statistics collection (-indexrangesize): 50

    [2015/06/19@08:43:19.489+0200] P-4868       T-4872  I BROKER  0: (14268) TCP/IP Version (-ipver): IPV4

    [2015/06/19@08:43:19.490+0200] P-4868       T-4872  I BROKER  0: (12263) SSL Key Alias Name (-keyalias): Not Enabled

    [2015/06/19@08:43:19.490+0200] P-4868       T-4872  I BROKER  0: (12815) Lock table hash table size (-lkhash): 13063

    [2015/06/19@08:43:19.491+0200] P-4868       T-4872  I BROKER  0: (17805) Original Lock Release Algorithm (-lkrela): Not Enabled

    [2015/06/19@08:43:19.491+0200] P-4868       T-4872  I BROKER  0: (17560) Number of LRU force skips (-lruskips): 0

    [2015/06/19@08:43:19.492+0200] P-4868       T-4872  I BROKER  0: (17561) Number of LRU2 force skips (-lru2skips): 0

    [2015/06/19@08:43:19.492+0200] P-4868       T-4872  I BROKER  0: (13953) Maximum Area Number (-maxArea): 32000

    [2015/06/19@08:43:19.493+0200] P-4868       T-4872  I BROKER  0: (12540) Size of JTA transaction table (-maxxids):  100

    [2015/06/19@08:43:19.493+0200] P-4868       T-4872  I BROKER  0: (5649)  Maximum Port for Auto Servers (-maxport): 5000

    [2015/06/19@08:43:19.494+0200] P-4868       T-4872  I BROKER  0: (5648)  Minimum Port for Auto Servers (-minport): 3000

    [2015/06/19@08:43:19.494+0200] P-4868       T-4872  I BROKER  0: (17564) Multi-tenancy partition cache size (-mtpmsize): 1024

    [2015/06/19@08:43:19.495+0200] P-4868       T-4872  I BROKER  0: (12821) Use muxlatches (-mux): 1

    [2015/06/19@08:43:19.496+0200] P-4868       T-4872  I BROKER  0: (4260)  Maximum Number of Users (-n): 61

    [2015/06/19@08:43:19.496+0200] P-4868       T-4872  I BROKER  0: (17566) Minimum time to nap at first -spin exhaustion (-nap): 10

    [2015/06/19@08:43:19.497+0200] P-4868       T-4872  I BROKER  0: (17565) Maximum time to nap at -spin exhaustion (-napmax): 250

    [2015/06/19@08:43:19.498+0200] P-4868       T-4872  I BROKER  0: (12273) No SSL Session Cache (-nosessioncache): Not Enabled

    [2015/06/19@08:43:19.499+0200] P-4868       T-4872  I BROKER  0: (17807) Disable LRU mechanism (-nolru): Not Enabled

    [2015/06/19@08:43:19.499+0200] P-4868       T-4872  I BROKER  0: (16689) Login Governor (-nGovernor): 0 of 61

    [2015/06/19@08:43:19.500+0200] P-4868       T-4872  I BROKER  0: (8527)  Storage object cache size (-omsize): 1024

    [2015/06/19@08:43:19.501+0200] P-4868       T-4872  I BROKER  0: (13870) Database Service Manager - IPC Queue Size (-pica): 64.0  KBytes

    [2015/06/19@08:43:19.501+0200] P-4868       T-4872  I BROKER  0: (17802) Shared memory segments locked (-pinshm): Not Enabled

    [2015/06/19@08:43:19.502+0200] P-4868       T-4872  I BROKER  0: (16953) Use pollset mechanism for client/server (-pollset): Not Enabled

    ... Finish!