Garbage collection for the sake of static temp-tables (in long-running _progres) - Forum - OpenEdge Development - Progress Community

Garbage collection for the sake of static temp-tables (in long-running _progres)

 Forum

Garbage collection for the sake of static temp-tables (in long-running _progres)

This question is not answered

I was wondering how to read this KB.

https://knowledgebase.progress.com/articles/Article/000037476

The only specific place where ABL garbage collection is forced to happen is on return from an AppServer call (deactivate procedure)

Is it saying that there is no ABL garbage collection except in appserver?  Or is it saying that appserver *forces* GC, but other types of clients (like _progres) will perform GC on occasion but it cannot be forced.

I have class that takes a reference to a static temp-table and BIND's to it:

 
USING Progress.Lang.*.

BLOCK-LEVEL ON ERROR UNDO, THROW.

Class dkb.ReproBug.SeededDocumentLogic: 
   
   
   {dkb/ReproBug/Data/LocalSalesTransData.i REFERENCE-ONLY PRIVATE}
   
   CONSTRUCTOR PUBLIC SeededDocumentLogic (  
      INPUT-OUTPUT DATASET FOR DS_LocalSalesTrans BIND):
         
      SUPER ().
      
   END CONSTRUCTOR.
 
   
END Class.

The include dkb/ReproBug/Data/LocalSalesTransData.i is like so:

/* ************************************************************************ */
/* Intial data                                                              */
/* ************************************************************************ */
DEFINE {2} TEMP-TABLE LL{3}_sls_trans NO-UNDO {1}

   FIELD Field1 AS CHARACTER 
   FIELD Field2 AS CHARACTER 
   FIELD Field3 AS CHARACTER 
   
   INDEX LL_sls_trans_01 IS PRIMARY UNIQUE Field1 Field2 Field3
   
   INDEX LL_sls_trans_02 Field2
   
   INDEX LL_sls_trans_03 Field3.
   
   
   
     
      

/* ************************************************************************ */
/* The dataset with the local/session data that we care about.              */
/* ************************************************************************ */
DEFINE {2} DATASET DS{3}_LocalSalesTrans  {1}
 
   FOR 
   
      LL{3}_sls_trans.
      

I use this class in logic loops like so, and don't always run DELETE OBJECT (eg. in the case of unexpected errors).

/* ************************************************************************ */
/* LL_sls_trans - used for composing the data                               */
/* ************************************************************************ */
{dkb/ReproBug/Data/LocalSalesTransData.i}


/* ************************************************************************ */
/* Operation vars                                                           */
/* ************************************************************************ */
DEFINE VARIABLE v_DocLogic AS dkb.ReproBug.SeededDocumentLogic NO-UNDO.

  
/* ************************************************************************ */
/* Do logic in a loop.                                                      */
/* ************************************************************************ */
DEFINE VARIABLE v_Loop AS INTEGER NO-UNDO.
DO v_Loop = 1 TO 5:
   

   /* ************************************************************************ */
   /* Output for this invoice                                                  */
   /* ************************************************************************ */
   EMPTY TEMP-TABLE LL_sls_trans.
         
   /* ********************************************************************* */
   /* SEEMS TO CREATE LEAKS */
   /* ********************************************************************* */
   v_DocLogic = NEW dkb.ReproBug.SeededDocumentLogic(INPUT-OUTPUT DATASET DS_LocalSalesTrans BIND).
   
   /* DO WORK HERE */
  
   /* Clean up Doc Logic   
   DELETE OBJECT v_DocLogic.*/
   
 
   
END.

Unfortunately I find that my static temp-tables "leak" as a result of the reference that is made by the OOABL class.  The error I eventually get is:

SYSTEM ERROR: Attempt to define too many indexes for area 6 database DBI11996a17396. (40) (14675)

I'm assuming the OOABL class instances are leaking too, but the problem doesn't expose itself as severely as the temp-tables that are leaked at the same time. I'm wondering what I need to do to avoid the leaking of static temp-tables.  Is there a way to ensure that GC will run at some point to release the "leaked" temp-tables?  Can I put a pause statement in the code or something, to encourage GC?  Any tips would be appreciated.

All Replies
  • You should log an issue with tech support if you want help with your forensic analysis!

  • I gave them one case with a dump, and then we closed it after a long period of inactivity.  I have tried using WinDbg and doing some small investigation myself.  It appears to be heap allocations in native memory.  Beyond that it is very hard to recognize the contents of that memory.  

    PASOE agents are the longest-lived OE processes we have ever worked with, so our memory issues are somewhat more serious than they were in any given _progres CHUI.  And the PASOE agent processes have high utilization and support concurrent sessions, which compounds any memory leak.

  • >> PASOE agents are the longest-lived OE processes we have ever worked with ...

    On that note I think there could be a solution to memory issues that is tailored to the design of PASOE as a whole.  For example, I think as a reasonable and fix, there should be an upper limit on the MSagent's memory usage.  If the _mproapsv process grows above some mind-blowing value like 5 GB, then the ABL application should gracefully abandon that original _mproapsv process and just start a new one.  The original _mproapsv process could just be flagged for deletion after it goes idle.

    Unless tech support can prioritize the analysis of a memory dump, my plan at this point is to wait for another large customer to start reporting leaks in PASOE (maybe QAD will start adopting this soon)?  And I can benefit from their efforts.  I've already worked on a few PASOE tech support issues myself.

    ... and I would spend more time on this too if I could do it productively.  But coming up with a repro is very challenging when the leak is "only" a few MB per hour, and is in native memory that is outside the boundaries of the AVM sessions.  If someone could tell me what is contained in the process memory dump, then it might give me the hint I need to create a repro.  It seems like it has to be a collaborative process, and I cannot build a repro on my own, as things stand right now.

    Another thing I'm hoping Progress might do is to eventually enable _mproapsv inactivity timeouts in the PASOE product by *default*.   (It is hard to trust the way we specify this resource timeout configuration in PASOE today, and I suspect that few people are actually using it to cycle their _mproapsv processes .)

  • On that note I think there could be a solution to memory issues that is tailored to the design of PASOE as a whole.  For example, I think as a reasonable and fix, there should be an upper limit on the MSagent's memory usage.  If the _mproapsv process grows above some mind-blowing value like 5 GB, then the ABL application should gracefully abandon that original _mproapsv process and just start a new one.

    This is the purpose of the REST and JMX APIs in PAS for OE. You can use an APM* tool that supports REST or JMX. We decided this approach was better since it supported all our platforms and is what is done throughout the industry. See knowledgebase.progress.com/.../How-to-call-into-the-OEManager-s-REST-API-for-insight-into-PASOE

    *Application performance management (APM) is a discipline that includes all the tools and activities involved in observing how software and hardware are performing. These tools present that performance information in a form managers and developers can use to make decisions.

    Also, if you are on PASOE 11.7.5 or 12.x, there is a memory leak analyzer tool that was demo'd on the CVP. As long as you are a CVP member you can access a video and the tool. See https://community.progress.com/community_groups/openedge_customer_validation_program/m/documents/3729

    -Shelley

  • Thanks Shelly.  I'll check out the video.  Most of articles I've seen about detecting leaks are tied to the ABL sessions (by way of oemanager REST queries or whatever).    

    In our case there aren't any ABL sessions in the msagent since they've already been trimmed.  So the results of any oemanager queries against ABL sessions always come back empty.  There needs to be a way to track down the memory that is used outside the bounds of our ABL sessions. That is memory that we cannot be held responsible for anymore as ABL programmers.