xref-xml issue with RUN PERSISTENT ON SERVER

Posted by dbeavon on 05-Sep-2018 09:29

As many Progress OE customers probably do, we have a home-grown code analysis tool that helps us find references to ABL programs.  It is used whenever a programming interface is changed and we need to find all the client references and update them for consistency.

Our home-grown tools rely on "xref-xml" which Progress ABL can generate during compile.  Unfortunately there are no Progress code analysis tools to make use of the outputs.  (see kb knowledgebase.progress.com/.../000044671

I'm having trouble with the xref-xml and I'm wondering if someone can confirm that this is a bug in the way the xref is generated.  

 

Supposing I have the following line of code, the xref reference is generated (see xml immediately following the ABL RUN statement):

RUN aaa/xreftest/Callee.p PERSISTENT SET v_Handle.


    <Reference Reference-type="RUN" Object-identifier="aaa/xreftest/Callee.p">
      <Source-guid>rQokBdTNnZdDFCBSKGZdIw</Source-guid>
      <File-num>1</File-num>
      <Ref-seq>10</Ref-seq>
      <Line-num>36</Line-num>
      <Object-context>PERSISTENT</Object-context>
      <Access-mode/>
      <Data-member-ref/>
      <Temp-ref/>
      <Detail/>
      <Is-static>false</Is-static>
      <Is-abstract>false</Is-abstract>
    </Reference>

Notice that the Object-identifier points to the name of the program being run, and the Object-context provides additional context to say that the program is run persistently ("PERSISTENT").  For databases that support XML and selective indexes on xpath expressions, it is fairly easy to load this into a database and search for the object identifier.  That will find all references to the "Object-identifier" in calling programs.

Things go wrong when we make this just slightly more interesting.  Below I run the same program on a server handle.

DEFINE VARIABLE v_Handle AS HANDLE NO-UNDO.

DEFINE VARIABLE v_ServerHandle AS HANDLE NO-UNDO.

CREATE SERVER v_ServerHandle.
v_ServerHandle:CONNECT (
   "-URL MyUrl -sessionModel Session-free",
   "user",
   "pass",
   "extra") NO-ERROR.


RUN aaa/xreftest/Callee.p PERSISTENT SET v_Handle ON SERVER v_ServerHandle.



    <Reference Reference-type="RUN" Object-identifier="aaa/xreftest/Callee.p PERSISTENT">
      <Source-guid>rQokBdTNnZdDFC9ShDjrnQ</Source-guid>
      <File-num>1</File-num>
      <Ref-seq>10</Ref-seq>
      <Line-num>14</Line-num>
      <Object-context>REMOTE</Object-context>
      <Access-mode/>
      <Data-member-ref/>
      <Temp-ref/>
      <Detail/>
      <Is-static>false</Is-static>
      <Is-abstract>false</Is-abstract>
    </Reference>

 

See that the purpose of the Object-context is now changed and it contains the word "REMOTE" but not "PERSISTENT".  It no longer cares about the fact that the program was run persistently.  Worse than that, however, is the fact that the "PERSISTENT" modifier is now added into the Object-identifier attribute thereby screwing up our selective indexes and related xpath queries.

Is there anyone who can comment on whether this is a bug?  I suspect the Object-identifier should remain the *same* regardless of whether the program is run normally, as PERSISTENT, or as PERSISTENT ON SERVER.  I suspect those modifiers shouldn't be affecting the Object-identifier but should be annotated somewhere else (eg in the Object-context).

Any feedback would be appreciated.  I would guess that any third-party developer tools would be facing the same challenge with this malformed xref-xml.  If I can get some confirmation that this behavior is probably a bug then I will open a ticket with Progress.  We are running OE 11.7.3 in development/PDSOE.

We are not in the business of writing ABL developer tools, and would rather Progress create both the xref-xml *and* the tools to analyze it.  But until that happens we need more meaningful xref-xml output for analysis.

Posted by dbeavon on 27-Sep-2018 08:27

I opened a support ticket and they agreed that the object identifier ("aaa/xreftest/Callee.p PERSISTENT") shouldn't include the "PERSISTENT" and that it was inconsistent behavior.  The object identifier for "RUN" should just be the name of the program that is being called.

It sounds like this will be fixed in some future version of OE.

Does anyone know of third-party tools that consume xref-xml? 

...It seems odd that I should be the first to encounter and report these problems, considering we are probably creating some of the least sophisticated custom tooling that is based around xref-xml.  I'd love to have a point of contact when dealing with xref-xml issues.  Maybe we'd evaluate third-party tooling so we don't have to write and maintain our own.

Posted by Jeff Ledbetter on 28-Sep-2018 12:02

Hi. Yes, that is our own repo (we use Roundtable to manage Roundtable development). :-)

"I would guess that the older text output omits some detail about modern ABL (eg some critical stuff on the OO side of things)"

As far we I know, the same data is in the flat-file and the XML output. We track classes, sub-classes, inheritance, properties, annotations, etc. All the current stuff.

"Will you ever consider giving xml-xref another shot?"

We've discussed it but it's a lower priority issue. We did give feedback several versions back, but have not revisited.

There are quirks in the XML text file. We've taken effort to work around them. :-)

All Replies

Posted by dbeavon on 27-Sep-2018 08:27

I opened a support ticket and they agreed that the object identifier ("aaa/xreftest/Callee.p PERSISTENT") shouldn't include the "PERSISTENT" and that it was inconsistent behavior.  The object identifier for "RUN" should just be the name of the program that is being called.

It sounds like this will be fixed in some future version of OE.

Does anyone know of third-party tools that consume xref-xml? 

...It seems odd that I should be the first to encounter and report these problems, considering we are probably creating some of the least sophisticated custom tooling that is based around xref-xml.  I'd love to have a point of contact when dealing with xref-xml issues.  Maybe we'd evaluate third-party tooling so we don't have to write and maintain our own.

Posted by Stefan Drissen on 27-Sep-2018 15:39

I think that with the introduction of dynamic everything in Progress 9, xref became a bit pointless.

Posted by Tim Kuehn on 27-Sep-2018 16:58

Xref can be a huge aid for finding out what's referenced where. For people looking for tools - attempts to develop tools hasn't gone very far in this space with ProTop being a notable exception.

Posted by Jeff Ledbetter on 28-Sep-2018 06:14

When xref-xml first became available, we switched all of our xref processing engine to use it but eventually we went back to parsing the xref text file output. It has a few quirks but at least we know what they are. And, we weren't happy with the performance.

"..attempts to develop tools hasn't gone very far in this space .."

I'd like to think that we've gone quite far with it. :-)

Posted by dbeavon on 28-Sep-2018 09:10

My feeling is that average line-of-business customers shouldn't be building their own general-purpose tooling around xref-xml.

I believe there are a few third-party vendors in the OE ecosystem and I was hoping to flush one of them out in the open.  Ideally Progress would build some of their own tooling for this, and incorporate it directly into PDSOE where it would be most useful.

My primary goal is to avoid being the only one running into xref-xml bugs.  If there were some bigger players reporting these bugs to Progress then I wouldn't have to.

Posted by jankeir on 28-Sep-2018 09:19

If you want off the shelve things, there are some that use either xref or xref-xml or both, like:

github.com/.../xref

github.com/.../pct

There are a couple more if you search around a bit.

Posted by Jeff Ledbetter on 28-Sep-2018 09:26

"..incorporate it directly into PDSOE where it would be most useful."

Yeah, something cool like this:

Posted by jankeir on 28-Sep-2018 09:29
Posted by Tim Kuehn on 28-Sep-2018 10:30

I'd like to think that we've gone quite far with it. :-)

Indeed - you guys have been around so long you're like part of the woodwork. :) 

Posted by Tim Kuehn on 28-Sep-2018 10:46

For someone looking for a parser, here's one I wrote back in 2008-ish.

community.progress.com/.../1614.xref-parser

Posted by dbeavon on 28-Sep-2018 11:55

Jeff, thanks for the info.  Is this screenshot from your TSMS?  

Will you ever consider giving xml-xref another shot?  Or if you don't intend to, have you given any feedback to Progress to help improve the problems with their xml output?

I would guess that the older text output omits some detail about modern ABL (eg some critical stuff on the OO side of things).  That's why I started down the path of using xml-xref.   As far as performance goes, I just stuff it all in a SQL Server database and use xpath queries against it.  (SQL has special tricks for performance-tuning on xpath  queries - one trick is based on something called a "selective xml index").  

If you aren't using the xml-xref, that explains why you aren't facing these problems in OE's xml-xref output format.

Posted by Jeff Ledbetter on 28-Sep-2018 12:02

Hi. Yes, that is our own repo (we use Roundtable to manage Roundtable development). :-)

"I would guess that the older text output omits some detail about modern ABL (eg some critical stuff on the OO side of things)"

As far we I know, the same data is in the flat-file and the XML output. We track classes, sub-classes, inheritance, properties, annotations, etc. All the current stuff.

"Will you ever consider giving xml-xref another shot?"

We've discussed it but it's a lower priority issue. We did give feedback several versions back, but have not revisited.

There are quirks in the XML text file. We've taken effort to work around them. :-)

This thread is closed