PASOE session manager question (a basic connectivity issue,

Posted by dbeavon on 10-Apr-2020 16:14

I think I've asked this before but don't have a path forward yet.  This is related to a fairly basic scenario that comes up in PASOE with client/server database connections.

But the session manager seems unable to account for this issue and throws lots of unhandled errors.

Here is the session manager log (abl_train_uec.2020-04-09.log)

20:46:49.080/37943896 [thd-9] ERROR com.progress.appserv.Session - LocalSession(4D305FC34905EAF3BE5BB1594EA2E8408AA6D4D9A13A.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
20:46:49.082/37943898 [thd-9] ERROR com.progress.appserv.Session - LocalSession(4D305FC34905EAF3BE5BB1594EA2E8408AA6D4D9A13A.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
20:56:19.089/38513905 [thd-4] ERROR com.progress.appserv.Session - LocalSession(DFF282303244F872EC661C2E92C5144A3AA206EB15A0.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
20:56:19.090/38513906 [thd-4] ERROR com.progress.appserv.Session - LocalSession(DFF282303244F872EC661C2E92C5144A3AA206EB15A0.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:05:32.086/39066902 [thd-7] ERROR com.progress.appserv.Session - LocalSession(9D7C250A6BEA4878C7363C3A653E27B1712944990B5D.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:05:32.086/39066902 [thd-7] ERROR com.progress.appserv.Session - LocalSession(9D7C250A6BEA4878C7363C3A653E27B1712944990B5D.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:05:32.088/39066904 [thd-9] ERROR com.progress.appserv.Session - LocalSession(D8EF567A80C14C841BFD923968589014040FFDEB9304.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:05:32.088/39066904 [thd-9] ERROR com.progress.appserv.Session - LocalSession(D8EF567A80C14C841BFD923968589014040FFDEB9304.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:05:45.735/39080551 [thd-6] ERROR com.progress.appserv.Session - LocalSession(D7BE5A9DF0052AD0B63F6CA93FA7C85A31BE8A67F13C.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:05:45.735/39080551 [thd-6] ERROR com.progress.appserv.Session - LocalSession(D7BE5A9DF0052AD0B63F6CA93FA7C85A31BE8A67F13C.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:05:45.737/39080553 [thd-8] ERROR com.progress.appserv.Session - LocalSession(77B4BF400CDBCB02337E7CEE33E610A06648BD87969A.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:05:45.738/39080554 [thd-8] ERROR com.progress.appserv.Session - LocalSession(77B4BF400CDBCB02337E7CEE33E610A06648BD87969A.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:05:54.299/39089115 [thd-5] ERROR com.progress.appserv.Session - LocalSession(84009B5AE0C1F1D9207CB1D949EB12C86613BB57BCD2.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:05:54.299/39089115 [thd-5] ERROR com.progress.appserv.Session - LocalSession(84009B5AE0C1F1D9207CB1D949EB12C86613BB57BCD2.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:06:01.582/39096398 [thd-4] ERROR com.progress.appserv.Session - LocalSession(9117478F0284697396831747451415D3B0DB586BE730.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:06:01.583/39096399 [thd-4] ERROR com.progress.appserv.Session - LocalSession(9117478F0284697396831747451415D3B0DB586BE730.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:06:04.741/39099557 [thd-9] ERROR com.progress.appserv.Session - LocalSession(1DF37236375E28A5895302865814944EBFDA0F7982CB.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:06:04.741/39099557 [thd-9] ERROR com.progress.appserv.Session - LocalSession(1DF37236375E28A5895302865814944EBFDA0F7982CB.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:06:04.741/39099557 [thd-1] ERROR com.progress.appserv.Session - LocalSession(33B3C329971F8E6FA0357D4F61324F50B9C62D08451F.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:06:04.741/39099557 [thd-1] ERROR com.progress.appserv.Session - LocalSession(33B3C329971F8E6FA0357D4F61324F50B9C62D08451F.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)
21:06:04.743/39099559 [thd-5] ERROR com.progress.appserv.Session - LocalSession(7C16FBBE4C3FD470B3581E9E56483CA81FDEA7CA2FA5.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)
21:06:04.743/39099559 [thd-5] ERROR com.progress.appserv.Session - LocalSession(7C16FBBE4C3FD470B3581E9E56483CA81FDEA7CA2FA5.oepas1) : an error occurred while reading response message = java.io.EOFException:null. (18296)

21:06:04.743/39099559 [thd-6] ERROR com.progress.appserv.Session - LocalSession(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : error occurred while sending a message WRITEDATA = java.net.SocketException: Software caused connection abort: socket write error:Software caused connection abort: socket write error. (18299)
21:06:04.744/39099560 [thd-6] ERROR c.p.appserv.PoolMgt.Connection - TcpAgentConnectionPool(bl9eolKsStacAJ-cR3F0hQ,P54W0MoTT125gb3a5jvhyw) : error in closing connection = java.net.SocketException: Software caused connection abort: socket write error:Software caused connection abort: socket write error (18267)
21:06:04.744/39099560 [thd-6] ERROR com.progress.appserv.Session - LocalSession(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : Error sending message UBRQ_WRITEDATA = java.net.SocketException: Software caused connection abort: socket write error:Software caused connection abort: socket write error. (18293)
21:06:04.744/39099560 [thd-6] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NetworkException: Agent. (18324)
21:06:04.744/39099560 [thd-6] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : network exception occurred processing request : com.progress.appserv.broker.exception.BrokerException$NetworkException: Agent : NetworkError[java.net.SocketException: Software caused connection abort: socket write error : Error sending message for (83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) = Software caused connection abort: socket write error]:Agent. (18318)
21:06:04.745/39099561 [thd-8] ERROR c.p.appserv.adapters.apsv.Request - APSV(33B3C329971F8E6FA0357D4F61324F50B9C62D08451F.oepas1) : An error occurred processing the POST request :  Unexpected error : com.progress.appserv.broker.exception.BrokerException$InvalidSessionStateException: Session. (18320)
21:06:04.747/39099563 [thd-3] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!
21:06:04.748/39099564 [thd-3] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NoAvailableConnectionsException: Agent. (18324)
21:06:04.748/39099564 [thd-3] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : No connections available to process request.. (18318)
21:06:04.797/39099613 [thd-9] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!
21:06:04.797/39099613 [thd-9] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NoAvailableConnectionsException: Agent. (18324)
21:06:04.797/39099613 [thd-9] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : No connections available to process request.. (18318)
21:06:04.799/39099615 [thd-4] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!
21:06:04.799/39099615 [thd-4] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NoAvailableConnectionsException: Agent. (18324)
21:06:04.799/39099615 [thd-4] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : No connections available to process request.. (18318)
21:06:04.800/39099616 [thd-1] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!
21:06:04.800/39099616 [thd-1] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NoAvailableConnectionsException: Agent. (18324)
21:06:04.801/39099617 [thd-1] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : No connections available to process request.. (18318)
21:06:04.845/39099661 [thd-10] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!
21:06:04.846/39099662 [thd-10] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NoAvailableConnectionsException: Agent. (18324)
21:06:04.846/39099662 [thd-10] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : No connections available to process request.. (18318)
21:06:04.847/39099663 [thd-5] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!
21:06:04.847/39099663 [thd-5] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NoAvailableConnectionsException: Agent. (18324)
21:06:04.847/39099663 [thd-5] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : No connections available to process request.. (18318)
21:06:04.848/39099664 [thd-6] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!
21:06:04.848/39099664 [thd-6] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : An error occurred while writing message UBRQ_WRITEDATA = com.progress.appserv.broker.exception.BrokerException$NoAvailableConnectionsException: Agent. (18324)
21:06:04.848/39099664 [thd-6] ERROR c.p.appserv.adapters.apsv.Request - APSV(83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1) : No connections available to process request.. (18318)
21:06:04.888/39099704 [thd-8] ERROR com.progress.appserv.Session - processWriteData(vF3_2397R5a1pdkY2LC9QA,SESSION<83F0506FC5DC69BC41E72BA8F3B5922F69152FA91485.oepas1>) RQID<<REQ|9H8z25r2UEumE2zpa6PJ/w-03374118> : no connection found!

Notice all the scary-looking unhandled exceptions. Even worse than these exceptions are the related issues with tomcat/HTTP sessions.  None of them are closed after the failures.  The APSV client is notified of the failure but the tomcat/HTTP sessions are left hanging around.  The issue leaves a massive mess that needs to be cleaned up manually in the tomcat manager console.

These exceptions come up during high utilization.  If I look in the PASOE agent logs I find the problem:

[20/04/09@20:46:48.134-0400] P-003800 T-002124 1 AS-14 -- The server or the system has no more resources. Please contact Progress Technical Support. (748)
[20/04/09@20:46:48.134-0400] P-003800 T-002124 1 AS-14 MSAS Connection to networked db lumbertrack lost for AS-14.
[20/04/09@20:46:48.135-0400] P-003800 T-002124 1 AS-14 -- ** Server rejected login. (700)
[20/04/09@20:46:48.135-0400] P-003800 T-002124 1 AS-14 -- Error initializing the application server. (5479)
[20/04/09@20:46:48.135-0400] P-003800 T-002124 1 AS-14 MSAS Unable to initialize session!
[20/04/09@20:46:48.135-0400] P-003800 T-002124 1 AS-Aux-0 MSAS Could not get and/or initialize 'Stateless' session! Cannot process request.
[20/04/09@20:46:48.135-0400] P-003800 T-002124 1 AS-Aux-0 MSAS Error handling request! Status=-1003
[20/04/09@20:46:48.136-0400] P-003800 T-003704 1 AS-ResourceMgr MSAS Spawning New Worker Thread. Number: 6
[20/04/09@20:46:48.136-0400] P-003800 T-002124 1 AS-Aux-0 MSAS Worker Thread exiting. Number: 4, Status: -14
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-15 -- The server or the system has no more resources. Please contact Progress Technical Support. (748)
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-15 MSAS Connection to networked db lumbertrack lost for AS-15.
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-15 -- ** Server rejected login. (700)
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-15 -- Error initializing the application server. (5479)
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-15 MSAS Unable to initialize session!
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-Aux-0 MSAS Could not get and/or initialize 'Stateless' session! Cannot process request.
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-Aux-0 MSAS Error handling request! Status=-1003
[20/04/09@20:56:18.143-0400] P-003800 T-003704 1 AS-ResourceMgr MSAS Spawning New Worker Thread. Number: 4
[20/04/09@20:56:18.143-0400] P-003800 T-005836 1 AS-Aux-0 MSAS Worker Thread exiting. Number: 6, Status: -14

This is simply the error you get when the OE dbms stops accepting remote client/server connections.  (You have attempted to connect to a database with too many users connected to it. Retry the connection later, or increase -n on the server.)

Can someone tell me if this is expected behavior for PASOE?  It seems like it should be a bit more resilient.  I would think that a connectivity failure between the msagent's ABL session and the database shouldn't have such a severe resource consequences within the session manager (tomcat plugin).

I suspect I can open a support ticket with Progress easily enough but I thought I would check if I missed something obvious in my configuration first.  I can't imagine I'm the only one who has dealt with an ABL session that cannot connect to the database.

There is a defect that is in the KB which points to a similar message, and similar connectivity issues dealing with PASOE and "external systems".  It is vague, but I cannot imagine that by "external systems" they are referring to a connection with an OE database.  That is where I'm seeing my issues. 

https://knowledgebase.progress.com/articles/Article/Opening-a-socket-in-PASOE-crashes-the-agent

Any tips would be appreciated.  

All Replies

Posted by David Cleary on 10-Apr-2020 17:45

I answered your other question related to -n, but you are correct that we should be more resilient if the networked DB connection is lost, as opposed to just being out of resources. I will bring this use case up with the appropriate teams and get something in the backlog concerning this use case.

As a way to manually recover, if the connection is specified as an agent startup parameter, shutting down the agent, allowing a new agent to startup is a solution. However, all those APSV sessions will need to be regenerated by having the client reestablish the connection, i believe. Would need to try it myself to confirm.

If you are performing the DB connections in a sessionStartup procedure, you can perform a refreshAgent operation that will recycle all the sessions on an agent, reestablishing the database connection. However, I believe this will also require APSV clients to regenerate their ABL session. The other transports would never notice, I believe.

The issue with improving this for APSV has to do with the underlying protocol. It is the same protocol we created for Classic AppServer that assumes a persistent socket connection between client and server. We have a feature  in our backlog to create a truly stateless APSV transport, but it hasn't risen enough in priority for us to take on this rather large activity. If more customers request this to PM, it would go a long way toward us being able to implement it.

Dave

Posted by dbeavon on 10-Apr-2020 18:29

We use the "agentStartupParam" to specify the database connection.  That is set in openedge.properties.  For client/server connections it involves -H and -S parameters.

The main trouble is the session manager doesn't fail very gracefully, and keeps HTTP sessions alive even if the ABL session had failed to initialize.  Is this a bug in 11.7.x?  What is the expected behavior?  It is extremely difficult for typical sys admins to interpret these PASOE logs in this type of situation (and they have to understand PASOE at great depth).  Nobody is able to understand what is happening until we compare the session-manager logs, with msagent logs, with oe database logs.  Then someone with tomcat expertise has to open the tomcat "manager" webapp and clean up the mess.

Ideally there would be an approach to clear things up, or the HTTP session should close after failing to inititialize.

Posted by dbeavon on 10-Apr-2020 19:34

>> the HTTP session should close after failing to inititialize.

This type of situation not at all uncommon, and shouldn't be treated as the end of the world from a PASOE perspective.  Encountering an issue like the "-n" limit is something that I would NOT want to cause a critical or permanent failure in PASOE.   It may happen during high load, or in a preproduction environment where the "-n" is set a lot lower than it is in production. I don't mind the temporary failures - as long as I know they will go away again on their own.

Ideally the recovery would be automated (either by Progress or by us).  We already do something similar to your proposed "refreshAgent" - we trim all the inactive ABL sessions of all the msagents.  This happens every half hour.  However that approach isn't sufficient to fix this particular issue because the "session manager" has already corrupted itself.  The failed HTTP sessions, in this case, appear to be latching on to ABL sessions, and preventing anything from being trimmed via the oemanager API.  It is a really ugly predicament.

At this stage it is almost as if there needs to be yet another API to manage the failed HTTP sessions as well.  But there doesn't appear to be an easy way to determine which of them are the corrupted ones (that had failed to initialize) and target them for immediate expiration.  It would be far better if the *PASOE* (session manager) would immediately expire the sessions that had failed to initialize.  That seems like the obvious answer and I'd go so far as to call it a bug.  I can't imagine the design of PASOE involves allowing HTTP sessions to stick around after they had failed to initialize.

Posted by dbeavon on 13-Apr-2020 00:49

Please let me know if Progress would like to take a stab at fixing this issue. I'm happy to contact tech support and show them the repro, if that  is the next step. I'm sure this repro is familiar already, since it doesn't seem very complex.  The MSagent simply won't initialize a session because of a resource limit in the database.  I would hope Progress would take an initiative to help with this issue ... but it is a bit discouraging to think about how long the issue has been present, without any prioritization.

The logs for the session manager and MSagent are both showing incoherent error messages, so in the very least it would be nice if Progress would improve on that. Ideally they would just say that an ABL session failed to initialize because of a resource limit that is enforced in the remote DBMS. And after improving on the messages, then perhaps Progress might address the corrupted HTTP sessions as a follow-up work item too.

Barring any help from Progress to fix this issue, I suppose I will need to write more custom code that would follow around behind PASOE and clean up the mess. I'm wondering what strategy to use in order to identify and terminate/expire any HTTP sessions that were created but had failed to fully initialize (... because they were not able to coordinate with the msagent and create an ABL session.)

Perhpas I could make use of the time fields on the HTTP sessions : "Creation Time" and "Last Accessed Time" and "Used Time". (image below).

At the moment my plan is to compare  "Creation Time" and "Last Accessed Time".  If they are (1) close to the same moment of time, (2) over an hour in the past, and (3) the "Used Time" never exceeded three seconds.  Then I will assume it was a corrupted session and just kill it.

Another thought is that perhaps this type of thing is not a Progress-specific concern at all.  Perhaps I should be asking for suggestions on a general-purpose apache tomcat forum.  There might be some common strategies for clean up after any arbitrary webapps that are misbehaving and leaving sessions open.

Posted by dbeavon on 13-Apr-2020 14:05

I reviewed the HTTP session activity in our production environment.  I'm no longer comfortable with my plan to clean up failed sessions on my own (by identifying them using  "Creation Time" and "Last Accessed Time").  The challenge in doing that is primarily related to certain types of APSV clients (ones that are session-free, as indicated by SessionType=1).  These type of clients often have the appearance of being failed sessions, according to the classification logic that I was going to use, even though they are not .

I suppose another option is to parse the session manager logs myself and find the particular items that were associated with a scary message, IE.

21:06:04.741/39099557 [thd-1] ERROR com.progress.appserv.Session - LocalSession(33B3C329971F8E6FA0357D4F61324F50B9C62D08451F.oepas1) : error occurred while reading a message processAgentRsp() = java.io.EOFException:null. (18300)

This seems like sort of a hack as well.

Ideally PASOE would clean up the HTTP sessions that failed to initialize properly.  I'm assuming that the session manager is written in Java.  In the Java language, whenever a failure happens during an operation and you want to clean up any partial work that may have been done, there is a programming pattern for that (try/catch/finally).  I wonder if the PASOE developers have considered using an approach like that?  It would be a good way to avoid the problem with these orphaned HTTP sessions that are indirectly latching onto ABL/msagent resources.

Posted by dbeavon on 15-Apr-2020 00:23

>> It is the same protocol we created for Classic AppServer that assumes a persistent socket connection between client and server. We have a feature in our backlog to create a truly stateless APSV transport

I agree that ideally a "session-free" connection to PASOE should not rely on an extremely long-lasting HTTP session.  We encountered some of the related challenges very early in our migration to PASOE, and were forced to extend our default tomcat session timeouts from 30 mins to 3000 mins.

But if it is such a large undertaking to fix the "APSV" protocol, then a workaround could be used for now (in a way that doesn't involve changing the protocol itself).  Although it is a bit counter-intuitive, the openclient applications seem to have an internal session pool of "session-free" HTTP sessions.  The pool is fairly fragile at the moment but, if it were able to repair itself when HTTP sessions timed out, then it would alleviate the need to support such extremely long-lasting HTTP sessions.  That may allow us to go back to using 30 minute inactivity timeouts in tomcat again!  It would not be a truly stateless transport but, for the purposes of most "session-free" application clients, it would probably work just as well.  

In regards to the topic of the original problem with the tomcat session-manager, I opened a tech support case.  It describes how PASOE suffers with enduring problems after a very temporary period of database connectivity failures.  The case is 00548819.

This thread is closed