I've had this happen on a number occasions. Our databases are restarted every weekend. And this obviously impacts all directly connected database clients. But the PASOE instance is running on a different server with client/server connections.
I would expect that PASOE would fail to process the first set of requests that arrive the next day, but would eventually recover and start up *new* MS-agents and/or new sessions within existing agents that would be properly connected to the database. However what I'm finding is that PASOE does not recover on its own from these situations and I'm required to go stop the ms-agents manually via OEE. After the ms-agents are stopped, things start working again.
The client typically detects STOP conditions and they don't go away, no matter how many retries are attempted:
[09:29:45.244] CONNECTING to: serv001:8810/.../apsv ... [09:29:45.249] RUN: persistent procedure - app/Production/AppServer/Shift.p [09:29:45.252] RUN: internal procedure - GetShiftData Exception thrown: 'Progress.Open4GL.DynamicAPI.FGLErrorException' in Progress.o4glrt.dll Exception thrown: 'Progress.Open4GL.Exceptions.RunTime4GLStopException' in Progress.o4glrt.dll Exception thrown: 'Progress.Open4GL.Exceptions.RunTime4GLStopException' in Progress.o4glrt.dll Exception thrown: 'Progress.Open4GL.Exceptions.RunTime4GLStopException' in Progress.o4glrt.dll Exception thrown: 'Progress.Open4GL.Exceptions.RunTime4GLStopException' in UFP.LumberTrack.AppServer.dll
The failures in the PASOE logs (agent) look like so:
[18/01/03@09:27:13.599-0500] P-005432 T-005464 1 AS-4 -- (Procedure: 'GetShiftData app/Production/AppServer/Shift.p' Line:192) Disconnect from server; database is being shutdown. (2659) [18/01/03@09:29:45.273-0500] P-005432 T-005464 1 AS-4 -- (Procedure: 'GetShiftData app/Production/AppServer/Shift.p' Line:192) app.Production.Shift Database lumbertrack not connected. (1006)
The database not connected error will repeat indefinitely.
I'm hoping someone else knows how to configure PASOE to recover from this situation. I'm betting that if I had a "resource timeout" in place then perhaps I wouldn't be affected as frequently because the chances would be higher that the ms-agent process would be automatically trimmed out of memory during the late hours when the database is restarted. However, even that isn't a guarantee. Ideally PASOE would have some special understanding of this type of problem (database not connected) and kill off the affected ms-agents so that he PASOE instance would recover on its own.
Any tips would be appreciated. I guess the next step will be to start writing scripts to stop all PASOE sessions/agents whenever any database is restarted. I was hoping to hold off on this type of approach as long as possible...
I once had to write some code to handle cases were the database was disconnected. The top-level .p program had no DB access code, it checked if the db's were connected, and if they weren't it ran another .p to connect them. The routine was in an ON STOP block in the top level .p in order to catch the untrappable stop generated by a db disconnect.
I once had to write some code to handle cases were the database was disconnected. The top-level .p program had no DB access code, it checked if the db's were connected, and if they weren't it ran another .p to connect them. The routine was in an ON STOP block in the top level .p in order to catch the untrappable stop generated by a db disconnect.
I have a little more information about the problem where the database is disconnected. When this happens, the rug is pulled out from under PASOE and the agent sessions become useless for anything other than generating error messages.
After opening a call with tech support I've learned that the issue didn't have anything to do with the migration to PASOE as such. The problem was related to the migration away from shared memory connections. With "shared memory" connections the agent(thread) is recycled. However if the databases is restarted and their are "client/server" connections, then the problem will occur, and the errors won't go away unless a custom programming solution is implemented.
I am new to client/server programming and this stuff surprised me. I wouldn't have guessed that any of this stuff would require custom logic, since we had already placed the database connectivity information in the PASOE appserver configuration properties. It seems to me that the configuration properties should indicate to PASOE that an active connection to the database is a necessary "pre-condition" for the agent to start, and for client's ABL code to run. It seems to me that the responsibilities should lie with PASOE, and that it should destroy sessions which don't continue to have their pre-configured database connection(s).
In any case, whether it is up to PASOE or the ABL developer, something needs to be done to recover database connections. Given that it is currently the ABL developer's responsibility (in the context of client/server connections) then there is sample code that you can use to recover your database connections (see here: knowledgebase.progress.com/.../P130271
I hope this helps anyone else having similar issues. If you are already familiar with client/server connections, this is probably old news. To make a long story short, Progress treats "shared memory" connections in a way that will automatically perform "clean up". Whenever you are using "shared memory" connections then Progress will "clean up" all your connected ABL clients. If you are using "client/server" connections then you need to do your own clean up work after the database has been restarted on you.