OERA,, SOA, OO, Entity, and View - Forum - OpenEdge Architecture - Progress Community
 Forum

OERA,, SOA, OO, Entity, and View

  • Murray posted an article in the Modeling Form which contains a reference to an article called Razorbills: What and How of Service Consumption ( http://www.architecturejournal.net/2005/issue4/aj4razor_pf.aspx )

    and that article in turn contains a reference to an article called Data on the Outside vs. Data on the Inside ( http://msdn2.microsoft.com/en-us/library/ms954587.aspx ). There are some interesting ideas here for our consideration of how data access should be packaged.

    In Razorbills, there is presented the notion of Entities vs. Views. There is a strong relationship here between Entity and what one would usually think of as a class in the service which owns that data. The point is made that the "view" of this data needed by various consuming services doesn't need to be the entirety of the data in the entity, nor is it limited to the data which is within the entity, i.e., it may require data resulting from a join with other entities. This creates the interesting notion that, for example, the AR service, which owns Customer, might have a Customer entity or class, but that when the Order service needs Customer data, the view that it requires is something other than this complete entity or class. Thus, the AR Service might have a message destined for the Order Service which allows the Order Service to build an OrderCustomer class corresponding to this view. I think this simplifies some of the issues of class design which have been raised earlier.

    In Outside/Inside there is an interesting discussion about the different types of data which exist in a system leading to a perception that when coming from a world of monolithic applications, one has a rather extreme view of data consistency because all updates occur within atomic transactions, but this is simply unrealistic ... and unnecessary ... in a distributed system. There are some interesting categorizations about when data must be immutable, when it can be versioned, and when it has validity within a defined realm of space and time. One of the points where I think this leads is partitioning of data. For example, in most businesses, most of the time, the Order service can work with information about the items available to order as of, for example, the start of the business day, meaning that it is usual to never that one would add a part to the catalog or modify the characteristics of a part and expect to them process orders on that changed information during the same business day. Yes, I know we can all think of exceptions and we can probably all think of ways to handle the exceptions, so let's just go with the observation for the moment. But, some of the information about items in the part catalog that is managed by the Inventory Service is much more dynamic, e.g., the quantity available to allocate to orders. Publishing that information once a day and relying on it would be very dangerous. But, if that quantity is impacted by a message from the Order Service which is attempting to make an allocation and the Inventory Service then reports that the request was successful or not, then the Order and Inventory services have a reliable way to handle this dynamic information. The returned information might include the remaining quantity available, but the Order service would only take this information as advisory, not as authoritative.

    More later perhaps, but I thought I would throw this out for comment.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • One of the Seattle BizzTalk-meetings (SOA) I attended back in 2000 or so was one of Pat Helland about "the fortress model". It's about not trusting incoming data at every level:

    "...

    The Helland Model (from Pat Helland, of Microsoft) describes separation based on trust boundaries. He sees an enterprise architecture as based on a collection of "fiefdoms". A fiefdom is a trust boundary and a functionality boundary with its own private data storage. Pat also thinks of the presentation tier as an "emissary" representing browser based clients to the fiefdom.

    ..."

    I borrowed the summary from http://www.objectwatch.com/newsletters/issue_36.htm

    from

    Another thing he writes about is "Metropolis": "...Pat Helland relates the evolution of technology architectures to the evolution of city planning and shows us how far we have yet to go...". See http://www.pathelland.com/ and http://msdn2.microsoft.com/en-us/library/aa480026.aspx.

  • In Razorbills, there is presented the notion of

    Entities vs. Views. There is a strong relationship

    here between Entity and what one would usually think

    of as a class in the service which owns that data.

    The point is made that the "view" of this data

    needed by various consuming services doesn't need to

    be the entirety of the data in the entity, nor is it

    limited to the data which is within the entity,

    i.e., it may require data resulting from a join with

    other entities. This creates the interesting notion

    that, for example, the AR service, which owns

    Customer, might have a Customer entity or class, but

    that when the Order service needs Customer data, the

    view that it requires is something other than this

    complete entity or class. Thus, the AR Service

    might have a message destined for the Order Service

    which allows the Order Service to build an

    OrderCustomer class corresponding to this view. I

    think this simplifies some of the issues of class

    design which have been raised earlier.

    Yep - I've used this sort of thing before (Modelling tools like to use this I've found). Where you can run into trouble if your not careful is creating excessive reads. Lets say we do create an OrderCustomer object. Now lets say we query all order's for day and return the OrderCustomer objects. Now, lets also assume we had 1000 orders and one customer did 990 of them. So, if you don't architect carefully, you end up with a solution where you go:

    for each order where :....

    no-lock:

    find first customer of order no-lock ...

    create OrderCustomer object.

    assign fields / call methods.

    end.

    Of course what we end up with reading the same customer 990 times. I've seen this quite a bit in Java OO systems where you make the solution so generic that it loses some of its smarts.

    So - as we go down this OO path, I think we are going to need better ways of "caching" objects in memory and linking them directly back to the DB. Maybe we could flag an Object as "like table" and have the DB keep it automatically upto date just like a normal row ??? E.g. have the DB handle all the writing / locking. Then we could simply look up a record (e.g. find customer of object) and put a pointer to the "real" in memory buffer into the ABL runtime. This could them be copied into a complete memory image when its passed outside the "shared memory" ABL pool (e.g. via an appserver).

    Anyway - back of track - getting this "business" level objects is great for the business people. Of course there are downsides when you start creating and shipping around lots of objects as the speed of that is a lot less than a straight query from the ABL for SQL.

    Others questions to ask are

    • how many different types of objects to I want? Do I want a customer, order and ordercustomer, ordercustomerlines, etc or will I leave them on their own?

    • When do I return ordercustomer and just order?

    • When I create a new order, do I want the end user to send OrderCustomer or just order? If its just "order" how do I find the customer or does "order" contain the unique customer key? If so, how do they get it?

    • What happens if my OrderCustomer is out of date? How do I maintain state?

    • How do I refactor a change through all the objects / WebServices and ensure they all still line up?

    • Can I "version" my objects and WebServices?

    A lot of these are covered in the article but I never really ""understood"" them until I tried it for myself ...

    Anyway - its all fun and games and its especially good to keep stretching ourselves. Thats why I love these forums

    Murray

  • This is

    where I think the second article has some interesting notions. If

    you think about it, even in a monolithic application, if the

    customer information is read no-lock for inclusion in the order,

    there is a certain danger of it being out of date by the time the

    order is completed. By and large, that doesn't actually matter

    except for information which the Order needs to update, such as

    CustomerTotalValueOnOrder and that is going to occur in a

    transaction post which occurs as a part of the completion process

    and will take the form of "add $1000 to TotalValueOnOrder" not

    "Make TotalValueOnOrder = $5000". So, the only kind of updates to

    Customer that one needs to worry about are things like changes to

    tax authority or an immediate change in shipping address ... but

    those are things where one just cant worry about the few seconds of

    time in which the order is being processed because the chances of

    those update occurring in that interval are vanishingly small. Even

    in a monolithic system, it is much more likely that the update to

    Customer will happen either before or after the processing of the

    Order. As well as being deadly serious.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • While I hear your wish, I think it is misplaced. If

    the database is local, we already get this service

    performed for us by the database buffer. The real

    issue is what happens when the authoritative database

    is not local.

    This has nothing to do with the database or the -B parameter. Its about managing objects, mapping objects to database tables and defining a unit of work. Once you go the route of using classes to represent your entities and when every entity represents a persistent object, than you have to track the objects during the request, the same way as the ABL tracks buffers (subsequent finds are tested against a local buffer context). I'm talking about a Customer class instance being mapped to a customer database row. Why wouldn't the ABL help us with this?

    But one step back, which we did earlier, is the question: should we instantiate an ABL entity (class instance) per domain object or should we pass (wrapped) temp-tables... So should we do it the Java/.Net way or the ABL way

  • By and large, that doesn't

    actually matter except for information which the

    Order needs to update, such as

    CustomerTotalValueOnOrder and that is going to occur

    in a transaction post which occurs as a part of the

    completion process and will take the form of "add

    $1000 to TotalValueOnOrder" not "Make

    TotalValueOnOrder = $5000".

    What kind of value would that CustomerTotalValueOnOrder be and in which currency?

  • For me, temp-table

    certainly have their place, but if one is writing OOABL, then a

    domain object is a class instance. One could elect to have a one

    row temp-table inside that class instance, but I am not currently

    thinking of that as best practice.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • A monetary value in whatever base currency

    the application is using as a reference. Are you asking why one

    would want such a number?

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • Well, for starters, because the mapping is quite

    likely not to be one object to one row.

    No, and I didn't claim so either

    Not to

    mention, of course, that service A might ask service

    B for a Customer view, which is sent via a Sonic XML

    message to a remote machine where service A

    instantiates a Customer object.

    You forgot to mention that the message was transmitted to a satellite, forwarded to ISS and echoed back to earth. This is all middleware. Object persistence is an internal issue. Messages are transmitted in external format and translated back to internal requests.

    You expect this to

    be linked automatically back to a database managed by

    another service on another machine?

    Huh? I expect the other service to receive an external request, accept the request and transform the request to internal calls or deny it. When accepted it will delegate the actual work:

    - massaging the message

    - instantiating domain objects

    - invoking operations on them

    - assembling the return message

    But why make things so complex? Your referenced link makes sense by stating that tightly coupled services should probably have been designed as one service.

    For me, temp-table certainly have their place, but if

    one is writing OOABL, then a domain object is a class

    instance. One could elect to have a one row

    temp-table inside that class instance, but I am not

    currently thinking of that as best practice.

    He, he, that's an understatement. It's a bad design! It's like creating a table for every row you want to store in the database...

  • What kind of value would that

    CustomerTotalValueOnOrder be and in which currency?

    A monetary value in whatever base currency the

    application is using as a reference.

    Are you asking why one would want such a number?

    Quite frankly: yes, since it's like adding apples and peers and present that figure as "good". Why would you want to sum the order values and store that figure in the customer? And when you want to shoot that "add $1000 to TotalValueOnOrder" to the other side of the world via Sonic, how will you maintain integrity?

  • That is

    somewhat my reaction as well, but the counter argument is that

    temp-tables provide some very useful services like the to/from XML

    methods and the before-table functionality for determining what has

    changed. In fact, I had myself thinking recently about how many of

    the languages I have worked in have had some interesting way of

    dealing with problems that is not found in a lot of other

    languages, but which one sometimes wants to imitate. And, in

    specific, how COBOL was interesting for its ability to define the

    same data area in multiple ways. Suppose, for example, that one

    could define properties for a class, but that one could also define

    a buffer which contained those properties and the buffer had XML

    methods and before-table functionality. Wouldn't that be neat? We

    must have

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • And what

    is your basis for deciding that a customer has sufficient credit

    available to ship an order. To me, this is typically something like

    Credit Available = Credit Limit - (Total Open Invoices + Total

    Orders Ready to Ship) If the current order which is being proposed

    for authorization has a value less than or equal to the Credit

    Available, it is OK to authorize it. Then, its value needs to be

    added to Total Orders Ready to Ship before evaluating the next

    order. Otherwise, someone with a $1000 credit limit could place 100

    $990 orders in one day and get them all to ship. In companies where

    backorders are common, like publishing, it is also common to want

    to have visibility for total orders, even if they aren't ready to

    ship and are not involved in credit authorization. Do you want to

    query the database every time this is asked? E.g., take my former

    customer SYBEX who would get back orders for a future publication

    stacked up for every Barnes and Noble store in North America (plus

    freight forward orders for others) ... not infrequently multiple

    orders per store when they decided to increase the initial

    stocking, and then bang, the book is in press and all those orders

    are slated for release in the same day ... thousands of them.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com

  • Well, I'm talking about a Customer class instance

    being mapped to a customer database row. sounded

    a lot like it. I might add that some of the

    properties of the object might not even be stored

    values.

    That's the whole point of "transparant persistence": the mapping schema determines which attributes will be persisted in which tables. It's similar to temp-tables and real database tables: the ABL has provided an immense powerfull mechanism by treating temp-tables as database tables. They could do the same thing for classes. Other languages need some complex Object Query Languages, but the ABL can solve it in the runtime. Have a look at DLinq in .Net (http://msdn2.microsoft.com/en-us/library/aa697427(VS.80).aspx), which looks very much like the ABL with its buffers, but it uses objects instead.

    E.g., one could elect not to store a total

    for the order, but compute one while building the

    object.

    Sure, no problem!

    I'm not sure of your point here. Mine is that tying

    the object to the database is unreasonable and

    inappropriate due to the likelihood of them not being

    on the same machine.

    No, there is a runtime mechanism that knows how to load a class from the database. But in a tiered and distributed environment, you still have to cut the direct link between the two tiers: the database mapped object will be transmitted as XML/TempTable/ProDataSet to the client, so the data can be bound to the user interface. The client might be a .Net client that can't consume ABL-classs instances. We shouldn't make the same mistake as EJB, where domain objects are accessed remotely, so every property accessor means a network roundtrip. No, you do a request, remotely something happens to domain objects and persistence and the result is marshalled back as a message.

    Admittedly, if the ABL had an envelope which included

    multiple of what are now independent sessions, then

    one could imagine a pub-sub connection in which a

    service would both request the data for an object and

    simultaneously register for a change notice.

    There is no need to inform the whole world about updates. The paper you referenced is about disconnected, versioned data. Its about the problems that are introduced by that in the new and loosely coupled world.

    Given

    that the source of the data was the authoritative

    source, then it could know that there was a change to

    a subscribed record and publish a need to refresh

    message or just publish the changes.

    This all assumes that the client is able to respond to these published messages. What if I'm a mobile device and I just want to display my lates order. I don't want to be connected to some server all the time....

    That is somewhat my reaction as well, but the counter

    argument is that temp-tables provide some very useful

    services like the to/from XML methods and the

    before-table functionality for determining what has

    changed.

    These are very simple to implement using reflection, runtime inspection of objects. Hopefully this feature will be in the next OOABL release. But even without that, you can generate some code that will cover this issue, don't you think? Mapping data is relatively easy with all the tools available.

    And the before-table isn't that usefull in all cases as well. You have to treat it in a special way and in that way it's good. When you need something else, you're in troubles. In the early days of the ProDataSet we kind of fought the before-table, since it didn't want to do what seemed logical.

    In fact, I had myself thinking recently about how

    many of the languages I have worked in have had some

    interesting way of dealing with problems that is not

    found in a lot of other languages, but which one

    sometimes wants to imitate. And, in specific, how

    COBOL was interesting for its ability to define the

    same data area in multiple ways.

    Data sections are great! I have asked for such a feature in ABL as well. Give us a declarative way of defining data sources. But I think it was you who replied that defining a prodataset was simple enough.

    Suppose, for

    example, that one could define properties for a

    class, but that one could also define a buffer which

    contained those properties and the buffer had XML

    methods and before-table functionality. Wouldn't

    that be neat?

    Wouldn't it be nice if you could define class properties and declaratively map them to table columns? And when you would do a "FIND cust" the runtime would recognize this as:

    - find "customer" buffer

    - create "Customer" class instance

    - bind the buffer to the "Customer"

    - decouple automatic transactions in this concept

    - "SAVE cust" would actually commit the changes to the database

    This is what we suggested earlier in this thread....

  • Do you want to query the database every time this is

    asked?

    Do you want to update the database everytime something happens to the order? Besides that, certain amounts are related to a date (due payments). And what will happen when you re-base the currency? Do you refresh all totals as well?

  • I

    certainly wanted this when I was first trying to create a

    temp-table of Customer objects with a ProDataSet, but once I

    figured out how to do that, it wasn't bad and it was certainly far

    more flexible than I can imagine a declarative structure being.

    I.e., it is the kind of thing that, by the time you make all the

    declarations, it takes up about as much effort and space as it does

    to make the assignments, especially with the capabilities in 10.1B

    where one can have multiple constructors, including one that sets

    all properties at once. This seems to me to be a different issue

    than I was trying to raise in this thread, however.

    Consulting in Model-Based Development, Transformation, and Object-Oriented Best Practice  http://www.cintegrity.com