PASOE and removing tomcat session timeouts

Posted by dbeavon on 22-Jan-2018 08:40

If anyone has expertise on PASOE and its integration with tomcat, I have a fairly technical question.  Should I be cautious of any solution that tells me to *remove* the tomcat timeouts altogether (ie. no session timeouts)?  It seems like a dangerous workaround.  

When I've tested it, I discover that PASOE clients (APSV transport) will often have sessions left open in the list of tomcat sessions indefinitely. (This can be easily seen in the tomcat manager webapp).  This is especially true for "session-free" clients.  The tomcat sessions, once opened, will *never* be terminated from the server side of things, except when manual/administrative operations are performed (eg. restarting the tomcat instance).  They continue to remain in memory long after client application processes have been shut down (gracefully or otherwise).

We are running 11.7.2.  Here is an example of a kb that seems to indicate that we should remove tomcat session timeouts:

https://knowledgebase.progress.com/articles/Article/Changing-idle-session-timeout-option-has-no-effect-in-PASOE

ie. this could be done by changing the web xml configuration as follows:


<session-config>
<session-timeout>-1</session-timeout>
</session-config>

We've been having troubles with the default tomcat session timeout (30 mins), especially as it relates to "session-free" APSV clients.  But I think if the "solution" is to remove session timeouts altogether, then that cure may be worse than the original problem.  I wanted to get some input regarding whether it is appropriate to remove session timeouts.  I was surprised that was even mentioned in the OE KB.

 

Posted by ManfredBerthold on 22-Jan-2018 10:32

In my experience there is no need to set the <web.xml>session-timeout to -1 unless testing purposes (to be sure that the client timeouts are not coming from the PASOE config, f.e. evaluating network/firewall timeouts).

Testing this in a configuration with a mixed apsv and soap (both session-managed) Instance leads us to never ending memory consumption because some memory is never freed - until server goes out of resources.

And yes, a part of this problem was application specific, migrating from classic appserver, all in state-reset mode to session-managed PASOE. We are ending with separate Instances and different timeouts, a longer one for apsv

and shorter ones for soap with better results for memory consumption over the time.

All Replies

Posted by Brian K. Maher on 22-Jan-2018 08:46

Hi David,
 
If you are using session-free with .NET clients (which I believe you are doing), then I would find this to be expected because you are creating a connection pool in the client when you create a Connection object and pass it to the constructor of an AppObject.  By default, that pool is a sized at 1 but will grow as needed with no idle timeouts of connections in the pool.  Of course, you can customize this with the available properties of the Connection object.
 
Brian

Posted by ManfredBerthold on 22-Jan-2018 10:32

In my experience there is no need to set the <web.xml>session-timeout to -1 unless testing purposes (to be sure that the client timeouts are not coming from the PASOE config, f.e. evaluating network/firewall timeouts).

Testing this in a configuration with a mixed apsv and soap (both session-managed) Instance leads us to never ending memory consumption because some memory is never freed - until server goes out of resources.

And yes, a part of this problem was application specific, migrating from classic appserver, all in state-reset mode to session-managed PASOE. We are ending with separate Instances and different timeouts, a longer one for apsv

and shorter ones for soap with better results for memory consumption over the time.

Posted by dbeavon on 22-Jan-2018 16:13

Brian.  You are right that it is the session-free clients that I'm struggling with.  When used with PASOE, they crash after the tomcat session expiration (30 mins by default).  I haven't found a way to avoid the crashes from the .Net client side of things.  They always crash the moment that tomcat expires sessions (with a very vague and very generic exception *** below).

The workaround may be to extend tomcat session timeouts indefinitely, but that cure may be worse than the original problem.  I have an open case with Progress support.  The only workaround available to me at this time is to keep extending tomcat session timeouts.  

*** Here is the error from our "session-free" clients, after they are redirected to PASOE/APSV:

Progress.Open4GL.Exceptions.Open4GLException: Communication layer message: General Error: 9998. (7175)

Posted by dbeavon on 22-Jan-2018 16:28

Manfred, I don't seem to be having issues with session-managed connections.  They typically connect from scratch every time, allocating brand new resources and freeing them up when the connection is closed.

It is "session free" connections that seem more challenging because they are designed to hold on to resources for an extended period.  (I believe that a "pool" of available sessions in the client is linked to a corresponding set of tomcat sessions on the server).  

This advanced "session free" connection management/pooling seems to work less well in PASOE than in the "classic appserver".    It is perfectly understandable that the PASOE server would want to clean up its resources after a half an hour of inactivity.  The problem is that the .Net open client is not able to react properly when it tries to re-use an outdated session - from the pool - that the tomcat server has already purged from memory.  Ideally it would automatically allocate new server-side resources if needed.

Posted by ManfredBerthold on 23-Jan-2018 04:02

David, good to know that you don't have issues with session-managed connection.

Ok, but we already have, especialy when using soap webservices.

We see that after a while sessions are unresponsive and will never be reused, attached memory is lost (with 11.6.4, bit different also in 11.7.2).

Setting the idle timeouts properly helps, but is not a solution. TS cases are open at moment.

I'm wondering if you get rid of the connection problems when the idle timeouts are off without affecting memory consumption.

Posted by Brian K. Maher on 23-Jan-2018 06:10

David,
 
Can you post the code that creates the Connection object and sets all properties?
 
Brian

Posted by dbeavon on 23-Jan-2018 21:21

No problem.  I can provide lots of code.  I also sent a full repro to tech support already.  Here is a sample of the code that gets a "session free" connection.  This happens via the .Net openclient API:

        public static Progress.Open4GL.Proxy.Connection GetConnection()
        {

            // Create the connection
            Progress.Open4GL.Proxy.Connection ConnectionObj = new Progress.Open4GL.Proxy.Connection(
@"hddp://localhost:8815/myappserver/apsv", userId: "uuu", password: string.Empty, appServerInfo: "aaa,bbb"); // Change the session model. int MyStateFreeSessionFree = 1; ConnectionObj.SessionModel = MyStateFreeSessionFree; // Time in seconds for the session pool ConnectionObj.ConnectionLifetime = 120; // Return it return ConnectionObj; }

Once created, this connection gets reused frequently (and an internal session pool is associated with it, allowing the benefit of improved performance).  But after a period of inactivity, once the tomcat session has already expired, then any further use of it will result in a fatal exception.  Note that the tomcat session timeouts for PASOE are 30 mins by default.  That is when the exception occurs.  It is quite consistent.

Because of that very large PASOE session timeout of 30 mins, I was really hopeful that I could set a client-side "ConnectionLifetime" of two minutes.  You can see that in the code above.  The purpose of that is to trim sessions out of the .Net session pool after a period of time.  In general the "ConnectionLifetime" works as expected.  However, I think the way it generally works is by making one round-trip TOO MANY after the connection lifetime was supposed to have expired.  I imagine that is a bug that is unrelated, and has been around for many versions of OE that predated PASOE .  While "classic appserver" may have allowed for that final round-trip to be made, PASOE is less forgiving and won't allow any round-trips to be made after it's own 30 minute timeout.

Thus far my workaround is to extend the PASOE session timeout indefinitely.  Another option is to do my own maintenance of the session pool for the "session free" connection (as shown below).  This is something that I'd really rather avoid since (1) I'm messing with implementation details in the openclient that may change in future versions, and (2) there would be a need to do this before each attempt to use a "session free" connection, and (3) even with this session pool maintenance, I'm not guaranteed to avoid the problem since the tomcat server may STILL expire a session in the short amount of time between this code and the attempt to use the connection.

                // Reserve all the available sessions.
                int NumberSessionsAvailable = ActiveAppServer.StateFreeConnection.GetSessionPool().availableSessions();
                for (int LoopCount = 0; LoopCount < NumberSessionsAvailable; LoopCount++)
                { 
                    // Capture a session and reserve it temporarily
                    Session ReservedSessionObj = ActiveAppServer.StateFreeConnection.GetSessionPool().reserveSession(); 
                    SessionsReserved.Add(ReservedSessionObj);
                }

                // Immediately release sessions again.
                foreach (Session LoopSession in SessionsReserved)
                { ActiveAppServer.StateFreeConnection.GetSessionPool().releaseSession(LoopSession.SessionID); }

 

Anyway, I'm still waiting on someone to reproduce these issues on the Progress side.  I'm not sure why it has never come up before now.  I don't think we are doing anything too unusual.  My hope is that Progress will find a better workaround than the ones I've come up with on my own (or better yet, start fixing some bugs in the APSV transport with PASOE).

This thread is closed