Starting out in OERA - Forum - OpenEdge Architecture - Progress Community
 Forum

Starting out in OERA

  • Hello All,

    Starting out in OERA and I've got a couple of questions. I have to admit I'm abit over taken with the complexity and the amount of code that needs to be written and can't help wonder if it's going to be a case of maybe 1% who even understand what it is.

    About fetchWhere( ) in the BE and it's DAO, where what ?

    Is the where clause against the physical schema buffers and fields ? If it is against the physcial schema what's the point in the layers separation ? If the BE or even worse it's API would need to know and reference the physical schema and tie it or the rest of the application to the physical source.

    If it's against the logical view where is the translation done ? Another thing is that the where clause would, possibly, need to be refactored or broken down into multiple where clauses if the data source is made up of multiple buffers. Breaking the where clause may not be very hard to do if it was just predicates/conditions and AND operators but is abit harder to do with real world where clauses with parethesis and OR operators.

    One more. Because of how the dataset is filled maybe similar to an outer-join and if there was a filter on a data-source down the line it would not limit the records created by the data-sources that were higher up, similar to an inner-join. What I'm trying to ask can you only have a filter, in this method, on only the main or first data-source.

    Many thanks.

  • That issue I raised here http://www.psdn.com/library/thread.jspa?threadID=3576&start=45&tstart=0. John Sadd's reply at the bottom of that page:

    "...

    In addition, the materials out on PSDN, among other things, show the mapping process between a filtering request as expressed on the client and what it gets mapped into when it gets back to where the physical data is. If the example shows only a fieldname change, this at least is a placeholder for larger changes that the back-end data access logic would need to be prepared to deal with in a real application.

    ..."

    I guess that mapping is a just a field name replacement....

  • Since OERA is a Reference Architecture and not an implementation, the answer to all of your questions is "that depends how you implement it".

    That said, I'm going to assume you're asking specifically in regard to the OERI (OpenEdge Reference Implementation) - AutoEdge.

    About fetchWhere( ) in the BE and it's DAO, where what ?

    The where clause is passed to the BE in a context temp-table that accompanies the call.

    Is the where clause against the physical schema buffers and fields ?

    Without digging back through all the AutoEdge code, I think that AutoEdge didn't make the distinction here. That said, I'm working on a project where we're doing the mapping in a Data Access Object super procedure. To this end we've used the DATA-SOURCE-COMPLETE-MAP attribute.

    Another thing is that the where clause would, possibly, need to be refactored or

    broken down into multiple where clauses if the data source is made up of multiple buffers.

    In our case, we don't have to do too much refactoring as we've been able to do filtering on fields that are returned by the Data Source queries - not values that are calculated in after row fill or after table fill events. Of course, we could use these places, removing any results that don't fit the given filtering criteria, but that could really kill performance.

    Breaking the where clause may not be very hard to do if it was just predicates/conditions and AND operators but is abit harder to do with real world where clauses with parethesis and OR operators.

    Very true. That's why we've restricted our where clauses to only allow AND's. That's a luxury that not every project would have

    One of the things that would actually really help us here though would be an ABL Query Optimiser. If we're not filtering on indexed fields in the first table of the query, we would have to reorganise the whole query to make it into something that performs. Another luxury of this project (to date. Let's hope that doesn't change too soon!)

    What I'm trying to ask can you only have a filter, in this method, on only the main or first data-source.

    As always, it depends on your implementation. In our current project, we can filter on any level of the ProDataSet, but for example batching only makes sense (for us) at the top level. We could of course implement that too if need be, but we haven't found a need yet.

    As for outer join like stuff, it probably depends on how the tables in your ProDataSets are related and whether the relations are active at the time of the fill. With respect to AutoEdge, perhaps someone who has more recently looked at the code could comment on what was done.

  • I guess that mapping is a just a field name replacement....

    It might not be quite so easy, for example if you moved a field into another table. That said though, as long as you can add the new table to your Data Source's query, it's not that hard at all.

    Hard might be something like: you used to store the order's total value in the order header record, but you decided to reduce redundancy, removed it from the header and now calculate it each time. Now you try to filter the orders to only show those with a total value of more than $X.

    Of course, this is not a new problem - we've always had to deal with this one in some way.

  • Thanks guys.

    That did alot to reassure me. Progress had a development team, 4 architects and a good sized amount of time and they came up with something that's what, still academic ? What do they expect us to do, the consultants, private developers, small development teams to do ?

    It may be the frustration talking but here we are AGAIN following the next big thing, the hype and it's almost everything we hear about. And 5 years from now there may only be a few places that started out OERA projects even less people who actually understand it and we'll be reading end of life and migration articles from OERA because it's too rigid, complicated or whatever.

    I don't know if it'll catch on. For one it's faaarrrrr from simple (and documentaton have never been a strong side), for that reason alone I don't think it's going to be anything like a common practice and for example we'll be seeing regular discussions on OERA @PEG. It just doesn't seem to me simplicty and practicality get the attention they deserve (if it's not simple it simply won't be ... or how ever the saying goes). And the thing is that apparently it's still not completely thought through.

    In regards to a query optmizer I wouldn't hold my breath. We're still asking for basic functionality in queries, specifically CAN-FIND and BREAK BY. It's like some other database vendor saying let's takeout EXIST and GROUP BY and see how they coupe now. My feeling is that Progress just doesn't know enough about queries. ABL is very good and maybe even elegant at transaction processing but if you ever have a query that's abit more complicated then simple data validation it's just not good enough. If it's the language features and even the database, for example, messaging they're just not good enough.

    Kind of funny since every performance tuning book I've read starts something along the lines of ... from my experience the biggest performance factor in applications is the database access and by far out of that are the queries.

    On a side note we have been working on a 4GL/ABL (V9 and above) query optimizer. Mapping from one view to another will not be in the first release but the where clause is entered for the entire query, although, it can also be set specifically for buffers among other features like a dynamic execution plan, special scans etc. We will be looking for beta sites in 2 to 3 month and I'm very interested to talk with anyone who'd like to tryout the product and participate.

    My first reaction is to use parameters instead of a where clause similar to a report that can also be freeform conditions that can be mapped to physical fields and maybe parameters that are abit more complicated then just a simple condition.

    Maybe even not to bother with DATA-SOURCE objects just use one query that can be optimized according to the criteria and CAN-FINDS to fill the dataset. For one you could have a filiter on the entire dataset and not just the first data source, which to me seems far too rigid and far too unrealistic. For example, think of having a dataset with an order and orderline temp-tables, can you think of not allowing for filtering the orderline for example by item ? I just hope DATA-SOURCE won't turn out to be a language wart.

    I really appreciate the help guys. Still not sure about OERA.

  • To rephrase what I'm trying to say perhaps more clearly, I'm worried we're going towards ADM3.

    And it's not that ADM2 didn't have uses, I recently used ADM2 on a small project. But at the end of the day it was too complicated to be used by most people (and documentation was not good). And it was too rigid and didn't pass the real world test.

    If I would ask for one thing from the higher ups is if you want people to be able to use it make it simple. And we'd like easy to understand, preferably short, documentation.

  • I think the first thing you need to do here is to be clear that OERA <> AE. AE is an exercise that was intended to illustrate one version of OERA, but it is certainly not definitive and, if you have been following these forums, there is considerable dispute about how complete and correct it really is in terms of best practices. I think people need to be very aware of these issues and consequently cautious about trying to use AE as a model for a production system. And this is why we have been having the discussion about a requirements document, a solution components document, and some sample code to help provide a reference which is closer to a production ready model.

    Which said, AE is a non-trivial accomplishment that illustrates a lot of interesting and important principles. It clearly is good enough that it has provoked a lot of detailed discussion and I think it could lead to a lot more. When something misses the point badly enough, there isn't much to say but that it missed the point. I don't think that is true of AE at all.

    I think that some of what you are wrestling with is the same kind of issue we seem to have in the discussion of OO OERA. I.e., there are those of us who want to encapsulate data and behavior into an entity object and to have all of the application except the DA layer relate to the contract of that entity object, not to the database at all. The other view is that ABL has lots of nice structures for what amount to local representations of relational data and we should continue to take advantage of those, even though it means encapsulating logic separate from data and having to have shared definitions of the data structure. As you have seen, this latter approach tends to result in seeing to distribute awareness of the stored data structure through the application ... technically, one can include name changes and denormalization and such, but the simple thing is to make the two structures very, very similar. Of course, there is nothing wrong with having your database schema highly congruent with your object structure ... it is desirable in fact. But, I think you need careful discipline to make sure that you keep the separation. AE doesn't do that very clearly and it is one of the things I dislike about it. But, that doesn't mean that we can't define something better and that you can do something better in your own implementations. I think that once you get sorted out how it should work, you will find that it is easier, not harder. It just takes some work to get there.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

      • deleted

    Message was edited by:

    Alon Blich

  • I wouldn't say that no one knows how to implement it. OERA is really no different than the architectural approach I was introduced to with Forté in 1995 and it wasn't new then. What it might be fair to say is that there are not yet any published ABL examples illustrating implementing OERA best practices. AE is a start, but more as a foil for discussion than a finished work.

    While I realize that tackling OO and OERA at the same time might seem like a lot, I think there are some very good reasons to go that route. These include:

    • Most patterns that one can read about and learn from are strongly tied to OO concepts. Not that they can't be applied in .p code, but they often need "translation".

    • Modeling tools can be a big help in sorting out OERA issues and designs and essentially all modern modeling tools are inherently OO. Yes, there is an EA set of models for the .p AutoEdge, but again it is not working with the natural mode of the tool.

    • There is a lot of synergy between OO design principles and OERA design principles. Thinking about your components in an OO way is a good running head start to thinking about them in an OERA way and vice-versa. This may not be quite as obvious from the DOH/ROH examples.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • Thanks Thomas,

    I'm looking at a future project that I'll be starting in a couple of month, relatively small Saas project.

    And I'm looking at using OERA, though, it probably won't be OO.

    So I'll probably have lots of other questions. Thanks.

  • Consulting and mentoring are available ....

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • Hi Alon

    As the person who's prime responsibility is the OERA, and one of those involved in AutoEdge, let me try answer some of your points.

    That did a lot to reassure me. Progress had a

    development team, 4 architects and a good sized

    amount of time and they came up with something that's

    what, still academic ? What do they expect us to do,

    the consultants, private developers, small

    development teams to do ?

    I'm not sure where this figure of 4 architects comes from, I don't believe we've ever stated how many architects, developers etc where involved, but I can tell you it wasn't as big as people seem to be imagining. At any one time there was 1 lead architect, plus some development resource, so please don't have impression you need an army of technical resource to do OERA. Which leads onto your next point..

    It may be the frustration talking but here we are

    AGAIN following the next big thing, the hype and it's

    almost everything we hear about. And 5 years from now

    there may only be a few places that started out OERA

    projects even less people who actually understand it

    and we'll be reading end of life and migration

    articles from OERA because it's too rigid,

    complicated or whatever.

    Firstly, we need to distinguish between OERA and implementations. The OERA is a Reference Architecture, it is a set of guidelines on how we (Progress) believe applications should be architected. It has a bunch of definitions and explanations of what each component is, why we think they're valuable, and things to consider when designing these components. None of this is supposed to be rigid in any way, shape or form, other than providing what we feel is a valid Reference Architecture.

    The next level down are implementations of these concepts. As Jamie has already commented, there is no one right way to implement these concepts. It's all about what is right for your given situation. Be that your application, your current technical resource, your future direction, etc. To that end, what we've done, and will continue to do, is provide ideas and examples on how some of these components could be implemented. For example we have the original work done by John Sadd (http://www.psdn.com/library/kbcategory.jspa?categoryID=289) which are small samples that are procedural based. We have AutoEdge (http://www.psdn.com/library/kbcategory.jspa?categoryID=298) which takes some of the concepts and code from that initial work by John and expands it into more of an application example within a given described business case and a great amount of documentation. Just this week we have posted some new class based work (http://www.psdn.com/library/kbcategory.jspa?categoryID=1212). The point of this isn't so say, hey aren't we great look at all the content we've done, but to say, look there are different ways you can approach this, and here are some ideas and thoughts for you to take and adapt to your situation. Again, I would strongly argue that none of this is rigid. Nowhere are we telling you that you have and must do it this way! Oh, and no we're not building ADM3 !!!! (although some may argue we should)

    I don't know if it'll catch on. For one it's

    faaarrrrr from simple (and documentaton have never

    been a strong side), for that reason alone I don't

    think it's going to be anything like a common

    practice and for example we'll be seeing regular

    discussions on OERA @PEG. It just doesn't seem to me

    simplicty and practicality get the attention they

    deserve (if it's not simple it simply won't be ... or

    how ever the saying goes). And the thing is that

    apparently it's still not completely thought

    through.

    One of my postings in another Thread (http://www.psdn.com/library/thread.jspa?threadID=3232&tstart=0) talks about what's coming, and as I said in that posting, we realize that the material posted so far could be considered advanced or appear complex. So in the very near future (the next couple of weeks all things being equal) we will be posting the first phase of a new set of material entitled 'Architecture Made Simple', where the aim is very much to simplify and help introduce ideas and code in terms and coding concepts that most people are using today when they create ABL applications. So hopefully in a couple of weeks this will add even more reassurance.

    I really appreciate the help guys. Still not sure

    about OERA.

    It's my, plus a group of other's job to make sure that we help as much as possible with making the concepts and ideas around OERA as simple and as consumable as possible. By using the forum, getting feedback, listening, answering and producing material I hope we achieve that. If there are specific questions then please continue to use the forum. If there are comments or questions of the material, please continue to use the forum. As others have said, we're starting to have some deep and meaningful discussions which although can get a little heavy and loud sometimes, are hopefully valuable, but at the same time we have to be mindful of the simple stuff as well

    Mike

  • As others have said, we're starting to have some deep and meaningful discussions which although can get a little heavy and loud sometimes, are hopefully valuable, but at the same time we have to be mindful of the simple stuff as well

    Mike

    If nothing else, opening the channels of communications, and keeping them open, is one of the huge potential benefit from this effort.

  • ...

    If nothing else, opening the channels of

    communications, and keeping them open, is one of the

    huge potential benefit from this effort.

    And communication is a two way thing, so please keep it coming. As I've said before, please encourage others to join in. It's only by having a strong community with a good communication channel that we can not only validate what we're striving to achieve, but also make sure that we're delivering on what is needed, not what we think is needed.

  • Mike, I certainly think that it is a great idea to be thinking along the lines of what you can do to create OERA for Dummies, but I think it is also worth noting that there is really nothing about OERA that is more complex than the world it is trying to respond to. The truth of the matter is that the development target has simply become more complex as the years have gone by and the old, simple, monolithic architectures that many of us grew up on simply don't cut the mustard any more. It isn't that SOA and ESB are virtuous for the enhanced functionality they provide, although that is true to, it is also gotten to be that they are essential. In this context, OERA is itself already helping to simplify the complex requirements of modern applications because it gives us a strategy, an approach for managing these complex requirements by partitioning and encapsulating the application into manageable units. Once one gets the idea, actually producing these components becomes pretty straightforward and complex requirements become straightforward to achieve. There is just some ramp up in coming to grips with the issues.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com