This rule modeling case study examines how the Corticon Business Rules Management System provides a way to automate this yet still provide the flexibility for the business experts to create and manage the rules.
Should this loan application be approved, rejected or referred for further consideration?
To make this decision we will need some rules.
Here are two of the sample rules provided by CIMB (there are more which we will introduce during the modeling exercise)
We will walk through the entire process of modeling these rules, including the discovery of errors, ambiguities and incompleteness using Corticon Studio.
We’ll find that even these two apparently simple rules contain problems which Corticon will help us discover and resolve
By referring to the rule statements we can deduce the existence of objects such as these:
Loan – the entity that is being considered for approval
We can also deduce that a loan will have some attributes such as
Status is what the rules are being asked to determine
This will be modeled in the Corticon vocabulary:
Figure 1 Client Business Object
This helps to make sure the data model makes sense
This can be done in the Corticon Testing tool which is built into Corticon.
We can also set up some expected results to verify that our rules are producing the answers we expect.
We’ll see during the demo how Corticon will automatically compare the actual and expected results and flag any variations once the rules have filled in the output column
Figure 2 Sample Test Data
There are no hard and fast rules about how to divide up the rules into groups, but a good way to start is to group the rules according to the main attribute that they are determining.
The book “The Decision Model” has some good guidelines on how to organize rules.
In this example we only have one sheet
These are the rule statements that would be on the rule sheet. The rule statements are always the starting point for rule modeling.
Figure 3 Rule Statements
Now we are ready to model the rules.
Within this phase of rule modeling there are a number of steps
Here’s what the rules might look like
Figure 4 Rules Modeled As Specified
Note that it’s also possible to toggle this view to a natural language view:
Figure 5 Natural Language
At this point it appears that we are done. Our rules seem to correspond to what was in the original specification. But are they correct?
If we were developing these rules in a programming language or even a different rule engine we’d probably need to run some tests to see if the rules are correct.
In the Corticon world we don’t believe that’s the most effective way of finding errors and so Corticon has built into Studio some tools for checking the rule model
Figure 6 Analysis Buttons
However, for the purposes of this case study, let’s do things that way other rule engines would have to do it – with test data.
We already have some test data and expected results so we can run our rules:
This is what we will get:
Figure 7 A Test Case
Note the second test case is highlighted in red because the actual result did not match the expected result.
But why is that?
If we select loan in the middle column, you will see that two rule statements are highlighted below – these are the ones that apply to loan.
Figure 8 Conflicting Rules Found
Notice that the first did get the correct answer. However a second rule also fired and overwrote the correct answer with an incorrect answer.
This is what we call a rule conflict or ambiguity.
Now we discovered this with test data – and for this simple example there aren’t very many possible test cases. But in the real world there may be thousands or millions of possible test cases. The chances of finding problems that way are very slim and often don’t happen until the rules are in production and an angry customer complains.
Corticon has a much better way.
Let’s return to the rule sheet and see how Corticon can find this problem without the need for test data.
If we run the conflict checker we’ll see this
Figure 9 Ambiguity Detected Automatically By Corticon
Corticon has determined that the conditions are not sufficient to distinguish these two rules and since they have different actions it considers them a potential ambiguity.
So, how do we resolve it?
Well you might say “just put them in the other order and everything will be fine”.
Yes, in this simple case that’s true but when there are dozens of rules it may not be obvious what the correct order is. And in any case it’s the rule engine’s job to figure out the order.
In order to resolve this we need to clarify the intent of the rules.
As specified it appears that rule 2 was intended to deal owner occupied loans
This can be modeled as follows:
Figure 10 Resolving the Ambiguity
And now WITHOUT HAVING TO RUN ANY TEST CASES Corticon can confirm that this is ambiguity free.
If we choose to rerun the test case we will find that we now get the expected answer for loan 2
Figure 11. Test Case Now Gets the Correct Answers
But notice that we still have a discrepancy. Loan 3 did not get a status assigned. That means somehow our rules are incomplete.
We can run the completeness checker and Corticon will identify any missing rules:
Figure 12 Completeness Checker
Figure 13 Rule for LTV 80% or more
Rules rarely stay the same for very long.
So we might imagine that this first rulesheet represents our default approval rules that apply to everyone.
Suppose one of our clients (say client XYZ) want to use slightly different rules for all of their loan approval.
We have a number of ways we can implement this:
The first step is to create a fresh rulesheet for client XYZ. This is done by copy/pasting the existing rulesheet.
Then we need to do two things
Here’s what that might look like:
Of course we should check this for completeness and consistency:
And Corticon will discover that we are missing some rules:
Now that we have two rulesheets (one generic and one for XYZ) we can configure them using the ruleflow:
In the first ruleflow, the global rules go first and the XYZ rules can override them.
In the second case the XYZ rules go first but can be constrained or overridden by the global rules.
Only XYZ users would be able to change the XYZ rules whereas corporate users would be able to change the Global rules (and the XYZ rules if needed)
We can specify effective dates for a rule flow:
This dialog specifies that the rule flow will become effective on 8/14/2012 and expire on 8/31/2012.
If a rule flow with a higher version number is deployed with an overlapping effective date then that new one will supersede this one.
Most rule engines can model the simple rules that we’ve looked at so far.
But they may not be able to handle the analysis and testing quite so well.
What happens when it gets more complex?
It can get more complex in two ways:
In this section we’ll look at how Corticon’s unique features make it easy for a business person to model rules that have to deal with complex data structures. In many other rule engines this is a task that may have to be done by a programmer.
Loans are usually associated with customers and loans will typically have one or more payments
This leads to the following vocabulary:
Figure 14 Vocabulary for Complex Data Structure
Here’s a sample of some data that conforms to this structure
Figure 15 Sample Data for Complex Structure
Suppose we want to sum the principal payments to determine the outstanding balance
The key to this is Corticon’s concept of SCOPE.
When we do the summing of amounts we want to make sure that it’s the sum of the transactions that belong to the accounts that belong to the client.
To indicate this hierarchy we drag the entities into the scope section
Figure 16 Client, Account, Transaction Hierarchy
We can also create aliases to the business objects. This will allow us to write more concise rules:
To compute the amount total we can now do something very similar to what you would do in Excel – we use the SUM operator:
Figure 17 Summing Amounts
Here’s a small test case
Figure 18 Test Data for Complex Data Structure
Here are some of the other operators that are available for dealing with collections
Figure 19 Collection Operators
Once the rule model is complete it can be deployed to the Corticon execution engine.
Corticon offers two modes of execution:
For the Web Services option the Corticon Web Console is used.
Only authorized users can login to the console.
You can deploy a decision service using the Deployment Console
Figure 20 Deployment Console
This will will show up in the server console
Figure 21 Logging on to the Server Console
Figure 22 Server Console Options
The first option allows you to see the currently deployed decision services.
S1, S2, S3 and S4 are different stages in the development of our Contra Loss rules
Figure 23 Deployed Decision Services
We can see the details of the decision service
Figure 24 Decision Service Settings
We can view the rules as they are in the server using a web interface:
This shows the natural language view of the rules in the web browser
Figure 25 Natural Language View of the Deployed Rules
We can even edit the rules using this web interface
First we create a new version from the existing version
Figure 26 Creating a New Version in the Web Console
Now we can edit the rules or the rule statements (access to this is configurable)
Now the rule values become editable
Figure 27 Web Based Rule Editor
This view allows a business user to change the values tested in the rules but not to add new rules
Figure 28 Analysis Tools in the Web Console
We can set up monitoring of key attributes in the decision service
Figure 29 Setting Up Monitored Attributes
You can also run test cases in the web interface
First export the test case from Studio as an XML file (or create it some other way)
Figure 30 Exporting a Test Case as XML
Specify the location of the test data
Figure 31 Setting Up a Test in the Web Console
The results will come back as XML
Figure 32 Test Results from Web Console
We can now set up a test in Studio that invoke the deployed decision service
Figure 33 Using Studio to Execute Deployed Decision Services
By clicking on the number of executions you can view the performance statistics for that decision service:
Figure 34 Execution Count
Details of the monitored attributes can be viewed:
Figure 35 Monitored Attributes
Corticon Collaborator is the tool that provides this functionality. Authorized users check in and checkout their rule models, vocabularies and test cases and a complete history of changes and versions is maintained. Here is an example of what a user might see:
Figure 36 Rule Repository Folders
Within a folder the user will see:
Figure 37 Repository Functions
The menu shows some of the functions that can be performed.
When a rule model needs to be moved into production, a workflow can be started in Collaborator which moves the assets through a series of approval steps such as notifying approvers by email, coordinating their responses and keeping an audit trail of the approvals
Here’s what a rule reviewer might see in her email inbox:
Figure 38 Email Notification of Rule Models to be Approved
And here is what might appear in the content of the email
Figure 39 Sample Email Notification
By following the link the rule approver can gain access to the rule asset: This may require that she logon
Figure 40 Access to the Repository is Controlled
After Jenny performs her review task, Tom (the rule author) will be notified by email:
Figure 41 Email Notification of Approval
This information is also maintained in the Collaborator audit trails.
Tom can monitor the progress of the approval process as follows:
Figure 42 Tracking Approval Workflows
You can see that there are several overdue tasks.
Tom can view the details of any of these tasks:
Figure 43 Approval Workflow Details
We can see in this example that Jenny (the QA person) approved it, *** (whose participation was optional) was skipped but Harry (the administrator) is holding things up
Corticon supports the deployment of rules and rule sets to various environments (i.e. development, staging, production).
Back in Collaborator we saw the folder that the rule author had access to. If we now login as the administrator we see more folders (which the rule author does not have access to). The administrator can see the development folder and also the UAT and production folders
Figure 44 Migrating Rule Assets to Production
In response to the rule author’s workflow, the administrator will migrate the rule assets from Development into UAT. Collaborator will keep a log of this activity. Once in UAT more testing and approval cycles will probably take place until eventually the rules get migrated into production.
(note: from this point on a slightly different rule model is used in the example, but the principles are the same)
Once the decision service is deployed on the production server it can be invoked by any SOAP Client or by any BPM software that can make a web services call or from application code such as Java or .NET
These same tests can be executed from any SOAP client such as SoapUI .
In order to do this you must first generate the WSDL for the decision service.
This is done from the deployment console which is accessible in Studio:
Figure 45 Deployment Console in Studio
A generic SOAP client such as SOAP UI can be used to set up and run tests against the deployed decision service:
Figure 46 A Typical SOAP Client
The SOAP Request would look like this
Figure 47 Corticon SOAP Request
The results are resulted similarly to what you see in the Corticon tester:
Figure 48 Corticon SOAP Response
In fact you can export this soap message from the test cases in Studio:
Figure 49 Studio Test Data
By selecting from the Test menu
Figure 50 Studio Export SOAP Menu
Corticon also partners with many of the BPM vendors.
Here’s an example of a business process that invokes a decision service.
Most BPMs provide for the import of WSDL and automatically generate the necessary connection to external web services. All that is required to invoke the decision service is to map the process variables to the input variables of the decision service.
Figure 51 Decision Service Invoked from a Business Process (Savvion)
Process attributes are mapped as inputs to Corticon
Figure 52 Mapping Corticon Inputs
Corticon results (status) is mapped to process attributes:
Figure 53 Mapping Corticon Outputs
Processes are started in the BPM Portal
Figure 54 Starting the Business Process
Inputs are entered on the web form
Figure 55 Web Data Entry Form
These are then passed to the Corticon rule engine
The user’s task list can be viewed:
Figure 56 User's Task List
The results are then displayed
Figure 57 Suspension Status Determined
Statistics Recorded in the Corticon Web Console
Figure 58 Monitoring the Decision Results
Decision Services may also be invoked from programming languages such as Java.
In this case, in addition to the web services interface, you can also make a direct in-process call. To do this the Corticon Server classes are compiled into your java program. Then you can invoke the server class “execute” method and pass the data either as XML or as native java objects.
Here’s an example of the core java code you would write to invoke a decision service:
Figure 60 In-Process Execution with Java Payload
The actual code would contain additional statements to populate the FEMAdata object , to catch any errors and to process the results coming back from the rule engine.
Here’s generally how our programmer will make an in-process call to the rules using an XML payload:
Figure 61 In-Process Execution with XML Payload
Figure 62 Invocation from Visual Basic
Figure 63 Invocation with XML Payload
Web Services Call from Java
Figure 64 Web Services Call from Java
These two sheets take care of generating all possible test cases:
Figure 65 Specifying the Test Values
Figure 66 Creating Instances of the Client
That really is all it takes!
You could get more sophisticated by writing actual rules to decide what base values to create but listing them seems the simplest way.
They could also be read in from a database. But having them in a sheet makes it easy to switch on or off various values for testing.
Just add this rule flow and you can run the test date against the suspension rules:
60 test cases generated.
Why is Corticon so much faster than Traditional Rule Engines?
In a traditional (RETE) rule engine a significant portion of the execution time is allocated to figuring out which rule to execute in which sequence – and if you process 1 million transactions the engine has to do this figuring out a million times.
Corticon does the figuring out ONCE at design time and so execution performance is considerably better than RETE based engines for most business rule applications.
Additionally Corticon scales linearly as the number of rules and the amount of data increases. RETE engines eventually reach a point where performance drops off dramatically as the data payload increases. RETE does scale well with increasing number of rules however.