We just started getting an odd error from our "classic" appserver broker. I'm not sure what changed. We upgraded to OE 11.6.3 about a year ago - on the broker side of things. The client side libraries are being upgraded as well (from 10.2.2.xxxx to 11.6.0.1408)
Here is the error in the classic broker logs:
[18/08/13@09:10:42.324-0400] P-005900 T-C-0095 2 UB Basic ConnectionID= 172.31.20.133::dev_statereset::3090::253bef44f57df62c:bc1568d:16531838e3d:-5305. (8096) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 2 UB Basic Client connected: 172.31.22.244. (8533) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB ----------- Unhandled exception caught in C-0095. (8419) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB ----------- java.lang.NullPointerException [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB -----------at com.progress.ubroker.util.ubMsg.tlvmap_get(ubMsg.java:2324) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB -----------at com.progress.ubroker.util.ubMsg.getTlvField(ubMsg.java:1585) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB -----------at com.progress.ubroker.broker.ubClientThread.negotiateAskCapabilities(ubClientThread.java:2877) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB -----------at com.progress.ubroker.broker.ubASclientThread.processConnect(ubASclientThread.java:270) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB -----------at com.progress.ubroker.broker.ubClientThread.processEvent(ubClientThread.java:1179) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB -----------at com.progress.ubroker.broker.ubClientThread.mainline(ubClientThread.java:439) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 1 UB -----------at com.progress.ubroker.broker.ubClientThread.run(ubClientThread.java:360) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 2 UB Basic Client disconnected : 172.31.22.244. (8534) [18/08/13@09:10:42.325-0400] P-005900 T-C-0095 3 UB Basic Ending clientPool thread: C-0095. (8077)
On the client side we see generic connection failures of three different types:
Progress.Open4GL.DynamicAPI.SessionPool+NoAvailableSessionsException: SessionPool : NoAvailableSessions[Could not connect to the AppServer: General Error: Connection reset by peer. (7203) ] Progress.Open4GL.DynamicAPI.SessionPool+NoAvailableSessionsException: SessionPool : NoAvailableSessions[CONNECT FAILURE: Unable to resolve hostname 172.31.20.133 (8821) ] Progress.Open4GL.DynamicAPI.SessionPool+NoAvailableSessionsException: SessionPool : NoAvailableSessions[Request failure: No Available Servers (7251) ]
There is a KB that references the null pointer exception from the "getTlvField" but it has a very cryptic explanation of what is going wrong.
It simply says there was a version mismatch of some kind. Our .net clients are not on an extremely old version of the progress client libraries. They are being migrated to Progress.o4glrt.dll version 11.6.0.1408. That should be compatible with 11.6.3.025 on the server. In any case, I think progress open clients are supposedly allowed to have a full version difference from the server (1 full version up or down). Another odd thing about the KB is that it doesn't really make a reference to .Net open clients, or explain how they would be affected. Finally, the KB seems to be documenting a client-side issue. But our error is showing up in the classic broker log on the server. It is really hard to interpret that KB.
Any tips would be appreciated.
After a lot of troubleshooting, I restarted the entire client machine (a windows machine) that was responsible for making up the majority of connections to the broker.
This seems to have fixed the problem since I'm not seeing any more of these NullPointerExceptions in the broker logs. And on the client side I'm not getting any more of the generic connection failures ("NoAvailableSessionsException").
My theory is that a temporary port in the *client* port range (ie. an ephemeral port on the client-side in windows) was conflicting with another windows application in some way. I always thought that it was the OS'es responsibility to provide unused port numbers; but perhaps something was going wrong. By restarting the entire machine, it must have resolved all port conflicts. I had already restarted the individual client applications numerous time without any success.
In case anyone ever sees that callstack in their broker log again, here is what Progress had told me about it. "The NullPointerException is about SocketInfo being NULL which is an indication of connection error."
Hopefully this makes sense. The client tried to connect, was given a new client-side port number for communication but that port was immediately and forcefully closed on the client-side because of conflicts. It would be nice to have a dedicated range of ports for OpenEdge to use on the client side (just as there are on the server). But according to the KB, there is no way to set aside a port range on the client side, so as to avoid conflicts. In general it is the operating system's responsibility to protect against conflicts. See...
After a lot of troubleshooting, I restarted the entire client machine (a windows machine) that was responsible for making up the majority of connections to the broker.
This seems to have fixed the problem since I'm not seeing any more of these NullPointerExceptions in the broker logs. And on the client side I'm not getting any more of the generic connection failures ("NoAvailableSessionsException").
My theory is that a temporary port in the *client* port range (ie. an ephemeral port on the client-side in windows) was conflicting with another windows application in some way. I always thought that it was the OS'es responsibility to provide unused port numbers; but perhaps something was going wrong. By restarting the entire machine, it must have resolved all port conflicts. I had already restarted the individual client applications numerous time without any success.
In case anyone ever sees that callstack in their broker log again, here is what Progress had told me about it. "The NullPointerException is about SocketInfo being NULL which is an indication of connection error."
Hopefully this makes sense. The client tried to connect, was given a new client-side port number for communication but that port was immediately and forcefully closed on the client-side because of conflicts. It would be nice to have a dedicated range of ports for OpenEdge to use on the client side (just as there are on the server). But according to the KB, there is no way to set aside a port range on the client side, so as to avoid conflicts. In general it is the operating system's responsibility to protect against conflicts. See...