Batching and sorting sample - Forum - OpenEdge Development - Progress Community
 Forum

Batching and sorting sample

  • I'm attaching a preliminary version of a batching and sorting sample that is scheduled to be included in the samples for FCS.

    It can be imported into any project in OEA or just extracted into any directory. You must have this target directory in propath (default for OEA projects) and a connection to the sports2000 database to run the sample. (The assembly file has more references than needed for this sample, as it has all references currently used in the samples scheduled for FCS.)

    The sample is using the same approach to support batching as the one that Mike Fechner posted, but it shows how to use the basic batching support in ProDataSets instead of the SDO and it also shows how to support sorting both with and without batching.

    Sorting:

    I think the most important conclusions so far is that it is almost mandatory to override the UltraGrid's default sorting also without batching. Using the ABL query for sorting is many times faster than the UltraGrids native sort.

    On the other hand, the UltraGrid makes the task of implementing sorting exceptionally easy. I practically had to do no work to be able to implement multi-column support. (I have looked into how this could be achieved in the adm2 with the browser and the UltraGrid wins big time)

    Batching:

    This sample illustrates the default batching support, which is an "appending forward batching" mode. This means that it will NOT give fast access to the last batch if you browse 10 million records. Batching is really for n-tier architectures (even if you run in a connected session) and I'm not convinced that it is the solution for someone who ONLY want to replace the browse grid as a direct database browser, unless they also are moving towards a layered architecture. It should still give acceptable initial performance in such a scenario though.

    It would certainly be interesting to test this approach in such an environment and find out where the limits and issues are. (I have not even tested it with other tables than the attached).

    I assume it is possible to implement two way batching similar to what you get in adm2 with the RebuildOnRepos attribute, but this is adds a lot of complexity, both on the client and the server.

    The service is simple and it should be very easy to replace it with your own accessing your own data, by copying the OrderLineService.cls.

    Feedback is welcome. Any criticism might be taken seriously and cause improvements in FCS, but I'd state upfront that there is NO attempt to do any form of error handling and this is not likely to change for FCS.

    Batching.zip

  • I guess I should give a short summary of where the actual batching code is as the attached data service and access code includes both basic and advanced stuff intended for all FCS samples, and less important for the batching.

    The actual running sample is Batching/BatchingForm.cls. You can run it directly from OEA, but there is also a runner.p as in the other samples allowing you to run this in any progress session or directly at the start with -p. The difference from the other samples is that the parent directory of the Batching directory is the start and propath directory.

    Presentation Layer

    The BatchingForm has an UltraGrid and some controls that allow you to test batching sizes and sort options. The controls are somewhat irrelevant for the batching topic, except that BatchSize is set in the service on the ValueChanged event of one of the controls.

    The actual code that manages the batching and sorting is implemented in the UltraGrid's AfterRowChange and the BindingSource's OffEnd events and the GetOrderLineDataSource method that starts and fetches data in an OrderLineService.

    The BindingSource's Batching property is set from a HasLast method in the Service after any request.

    Logic Layer/Adapter (temp-table layer) - DataServices directory.

    The OrderLineSource used in the sample inherits a DataBatchingService, which implements IDataBatchingService with properties DatasetHandle, BatchSize and HasLast and methods FetchData(), FetchNextBatch() and ResortQuery().

    FetchData retrieves the first batch from the Data Access Object while FetchNextBatch calls the Data Access to do the actual bathing. ResortQuery deals with local sort and calls FetchData with the new query string if it is batching and need to do the resort in the data access.

    The methods use two private members to keep track of the current sort expression and the next batch rowid.

    Data Access - DataSources directory

    The DynamicDAO is a dynamic class that handles the data access. It implements IDataAcess which has three fetch methods. We use the same FetchBatch method for all data access.

    Attachment

    I'm attaching a new edited and slightly reorganized readme.out file with a bit more details about the actual code. I've also added a Performance and sorting and/or batching section, that points out that sorting on large tables without index support can be slow and that this might be an issue on each batch request. These are issues that most ABL developers are aware of, but might forget when moving away from the database.

    Minor edits by:

    Havard Danielsen

    3288.mEdtScr.w.zip

    2766.mEdtScr2.w.zip

    7701.mEdtScrLaunch.p.zip

    1680.test1Form.cls.zip

    1641.test1Form.resx.zip

  • Hi Havard,

    I tried your sample and the DataService class did not work in the beta. I have attached a fix.

    Nice sample! The next challenge is to include filtering... we are working on that now.

    -peter

    6332.mEdtScr.w.zip

    8231.mEdtScr2.w.zip

    7801.mEdtScrLaunch.p.zip

    7446.test1Form.cls.zip

    4213.test1Form.resx.zip

  • Ouch this turns out to be a can of worms... AFAICS there is no such thing as a BeforeFilter event. The documentation talks about a 'Filter Evaluation Trigger' but how do you intercept it?

    Clarification: the ultraGrid comes with a nice filtering feature. You can enable filtering on the headers or even have an automatic filter row (see attachment). The same rules that apply to sorting apply to filtering (in combination with batching): it has to be done on the server. But how do you do that?

  • Havard,

    I get a code parser error when attempting to open.

    "The language did not provide a code parser for this file. Please make sure that this file type supports a designer.

    Could not load BatchingForm.cls".

    If I use the ABL Editor I can see the code just fine.

  • Then you probably did not place the provided assemblies.xml in your project root directory.

  • Then you probably did not place the provided assemblies.xml in your project root directory.

  • Another victim of the big pause... Any progress on the filtering part?

  • Okay, this was the issue. I had to replace the assemblies that was already there.

    The dataservice.cls does not seem to like the following:

    dataColumn = dataTable:Columns:Add(fieldHandle:Name,NetType(fieldHandle:Data-type))

    Any ideas.

  • I forgot the details:

    Paramter 2 for method ADD is not type compatible with its definition (12905)

  • I posted the fix in this thread yesterday.

  • I saw that and downloaded the cls with the same results. I will try again.

  • Must be the guy sitting in the chair!! I cut the code from the forum this time and pasted and it works.

    Thanks!!

  • Thanks for the beta fix, I forgot that you guys don't have the luxury of a fresh install each morning...

    The BeforeRowFilterChange and AfterRowFilterChange fire when you AllowFiltering on a grid column. I assume they also work with the Fixed RowFilter feature, but i have not tested this.

    I haven't tried to make these events work with server requests. (They are somewhat geared towards local filtering). The event argument object passed to the event has a Cancel option, that I assume you can use to take complete control. It is fully possible that this can be achieved without canceling the default filter though.

    Grid filter code should ideally take into account the RowFilterMode and RowFilterAction options on the grid's Override. Some of the modes shows all data with filtered data with a different appearance and does not fit with server filtering (probably easier to implement though.) The frmFilterRow in the WinGrid/ExplorerSamples demonstrates the RowFilterMode with many other filter options).

    The default filter has options like 'all', 'blank' an 'non-blank' which one would need to deal with. The frmUltraDataSource2 sample shows how to remove and add values to the default column filter options. You might want to support some of these options of course, but it probably does not make sense to let the grid add its own values to this list if you are batching or if the column filter is to apply the criteria on the server.

  • Havard,

    Where do you find these events? I don't see them among the events and cannot find them in the documentation either.