The Business Problem

This rule modeling case study examines how the Corticon Business Rules Management System provides a way to automate this yet still provide the flexibility for the business experts to create and manage the rules.



Basic Rule Modeling

Identify the Business Decision(s) to be made

Should this loan application be approved, rejected or referred for further consideration?

Collect and Review Rules needed for each decision

To make this decision we will need some rules.

Here are two of the sample rules provided by CIMB (there are more which we will introduce during the modeling exercise)

  1. We do not currently do non owner occupied loans
  2. Loans under 80% LTV may be automatically approved

  

We will walk through the entire process of modeling these rules, including the discovery of errors, ambiguities and incompleteness using Corticon Studio.

We’ll find that even these two apparently simple rules contain problems which Corticon will help us discover and resolve

Identify Business Objects (Entities)

By referring to the rule statements we can deduce the existence of objects such as these:

Loan  – the entity that is being considered for approval

We can also deduce that a loan will have some attributes such as

  • amount
  • appraisal value
  • loan to value ratio
  • ownerOccupied (true or false)
  • status

Status is what the rules are being asked to determine

Create a Corticon Vocabulary

This will be modeled in the Corticon vocabulary:

Figure 1 Client Business Object

Create Sample Data

This helps to make sure the data model makes sense

This can be done in the Corticon Testing tool which is built into Corticon.

We can also set up some expected results to verify that our rules are producing the answers we expect.

We’ll see during the demo how Corticon will automatically compare the actual and expected results and flag any variations once the rules have filled in the output column

  

Figure 2 Sample Test Data

 

Decide how the rules need to be grouped into Rule sheets

There are no hard and fast rules about how to divide up the rules into groups, but a good way to start is to group the rules according to the main attribute that they are determining.

The book “The Decision Model” has some good guidelines on how to organize rules.

 

In this example we only have one sheet

 

These are the rule statements that would be on the rule sheet. The rule statements are always the starting point for rule modeling.

Figure 3 Rule Statements

 

Model Rules #1 and #2

Now we are ready to model the rules.

Within this phase of rule modeling there are a number of steps

 

For each rule statement create a rule column that connects the conditions to the actions

Here’s what the rules might look like

Figure 4 Rules Modeled As Specified

 

Write Natural Language Statements if Desired

Note that it’s also possible to toggle this view to a natural language view:

Figure 5 Natural Language

 

At this point it appears that we are done. Our rules seem to correspond to what was in the original specification. But are they correct?

If we were developing these rules in a programming language or even a different rule engine we’d probably need to run some tests to see if the rules are correct.

In the Corticon world we don’t believe that’s the most effective way of finding errors and so Corticon has built into Studio some tools for checking the rule model

Figure 6 Analysis Buttons

However, for the purposes of this case study, let’s do things that way other rule engines would have to do it – with test data.

Run Test Cases

We already have some test data and expected results so we can run our rules:

This is what we will get:

Figure 7 A Test Case

Note the second test case is highlighted in red because the actual result did not match the expected result.

But why is that?

If we select loan[2] in the middle column, you will see that two rule statements are highlighted below – these are the ones that apply to loan[2].

Figure 8 Conflicting Rules Found

Notice that the first did get the correct answer. However a second rule also fired and overwrote the correct answer with an incorrect answer.

This is what we call a rule conflict or ambiguity.

Now we discovered this with test data – and for this simple example there aren’t very many possible test cases. But in the real world there may be thousands or millions of possible test cases. The chances of finding problems that way are very slim and often don’t happen until the rules are in production and an angry customer complains.

Check for Rule Ambiguity

Corticon has a much better way.

Let’s return to the rule sheet and see how Corticon can find this problem without the need for test data.

If we run the conflict checker we’ll see this

Figure 9 Ambiguity Detected Automatically By Corticon

Corticon has determined that the conditions are not sufficient to distinguish these two rules and since they have different actions it considers them a potential ambiguity.

So, how do we resolve it?

Well you might say “just put them in the other order and everything will be fine”.

Yes, in this simple case that’s true but when there are dozens of rules it may not be obvious what the correct order is. And in any case it’s the rule engine’s job to figure out the order.

In order to resolve this we need to clarify the intent of the rules.

As specified it appears that rule 2 was intended to deal owner occupied loans

This can be modeled as follows:

Figure 10 Resolving the Ambiguity

And now WITHOUT HAVING TO RUN ANY TEST CASES Corticon can confirm that this is ambiguity free.

If we choose to rerun the test case we will find that we now get the expected answer for loan 2

Figure 11. Test Case Now Gets the Correct Answers

Check for Rule Completeness

But notice that we still have a discrepancy. Loan 3 did not get a status assigned. That means somehow our rules are incomplete.

We can run the completeness checker and Corticon will identify any missing rules:

Figure 12 Completeness Checker

Figure 13 Rule for LTV 80% or more

Rule Changes

Rules rarely stay the same for very long.

So we might imagine that this first rulesheet represents our default approval rules that apply to everyone.

Suppose one of our clients (say client XYZ) want to use slightly different rules for all of their loan approval.

We have a number of ways we can implement this:

  1. We can provide an alternative rulesheet that applies just to client XYZ
  2. We can allow client XYZ’s rules to override our default rules
  3. We can constrain client XYZs rules to fit within our default rules

 

The first step is to create a fresh rulesheet for client XYZ. This is done by copy/pasting the existing rulesheet.

Then we need to do two things

  1. Indicate that this new sheet applies only to client XYZ
  2. Make the changes that are specific to XYZ

Here’s what that might look like:

Of course we should check this for completeness and consistency:

And Corticon will discover that we are missing some rules:

 

The Ruleflow

Now that we have two rulesheets (one generic and one for XYZ) we can configure them using the ruleflow:

  or 

 

In the first ruleflow, the global rules go first and the XYZ rules can override them.

In the second case the XYZ rules go first but can be constrained or overridden by the global rules.

Only XYZ users would be able to change the XYZ rules whereas corporate users would be able to change the Global rules (and the XYZ rules if needed)

Effective Dating of Decision Services

We can specify effective dates for a rule flow:

This dialog specifies that the rule flow will become effective on 8/14/2012 and expire on 8/31/2012.

If a rule flow with a higher version number is deployed with an overlapping effective date then that new one will supersede this one.




Advanced Rule Modeling

Most rule engines can model the simple rules that we’ve looked at so far.

But they may not be able to handle the analysis and testing quite so well.

What happens when it gets more complex?

It can get more complex in two ways:

  1. More attributes, more conditions and more combinations of conditions
  2. More complex data structures and relationships between them

In this section we’ll look at how Corticon’s unique features make it easy for a business person to model rules that have to deal with complex data structures. In many other rule engines this is a task that may have to be done by a programmer.

 

Loans are usually associated with customers and loans will typically have one or more payments

Vocabulary for Complex Data Structures

This leads to the following vocabulary:

Figure 14 Vocabulary for Complex Data Structure

 

Sample Complex Hierarchical Data

Here’s a sample of some data that conforms to this structure

Figure 15 Sample Data for Complex Structure

Suppose we want to sum the principal payments to determine the outstanding balance

Identifying the Context of a Rulesheet using Scope

The key to this is Corticon’s concept of SCOPE.

When we do the summing of amounts we want to make sure that it’s the sum of the transactions that belong to the accounts that belong to the client.

To indicate this hierarchy we drag the entities into the scope section

Figure 16 Client, Account, Transaction Hierarchy

We can also create aliases to the business objects. This will allow us to write more concise rules:

 

To compute the amount total we can now do something very similar to what you would do in Excel – we use the SUM operator:

 

Figure 17 Summing Amounts

 

Here’s a small test case

Figure 18 Test Data for Complex Data Structure

Operators for Collections

Here are some of the other operators that are available for dealing with collections

Figure 19 Collection Operators


 

Deploy Decision Services

Once the rule model is complete it can be deployed to the Corticon execution engine.

Corticon offers two modes of execution:

  1. Web Services
  2. In-process execution

 

For the Web Services option the Corticon Web Console is used.

Only authorized users can login to the console.

 

The Deployment Console

You can deploy a decision service using the Deployment Console

Figure 20 Deployment Console

This will will show up in the server console

 

The Web Console

Figure 21 Logging on to the Server Console

Figure 22 Server Console Options

Viewing Deployed Decision Services

The first option allows you to see the currently deployed decision services.

S1, S2, S3 and S4 are different stages in the development of our Contra Loss rules

Figure 23 Deployed Decision Services

 

Decision Service Details

We can see the details of the decision service

Figure 24 Decision Service Settings

 

 

Web Interface for Viewing the Rules

We can view the rules as they are in the server using a web interface:

 

This shows  the natural language view of the rules in the web browser

Figure 25 Natural Language View of the Deployed Rules

 

Editing the Rules in the Web Interface

We can even edit the rules using this web interface

First we create a new version from the existing version

Figure 26 Creating a New Version in the Web Console

Now we can edit the rules or the rule statements (access to this is configurable)

Now the rule values become editable

Figure 27 Web Based Rule Editor

This view allows a business user to change the values tested in the rules but not to add new rules

The web editor includes the analysis tools

Figure 28 Analysis Tools in the Web Console

 

Monitoring Key Decision Service Attributes

 We can set up monitoring of key attributes in the decision service

Figure 29 Setting Up Monitored Attributes

 

Running Tests from the Web Interface

You can also run test cases in the web interface

First export the test case from Studio as an XML file (or create it some other way)

Figure 30 Exporting a Test Case as XML

 

Specify the location of the test data

Figure 31 Setting Up  a Test in the Web Console

The results will come back as XML

Figure 32 Test Results from Web Console

   

Invoking Deployed Decision Services from Corticon Studio

We can now set up a test in Studio that invoke the deployed decision service

 

Figure 33 Using Studio to Execute Deployed Decision Services

 

By clicking on the number of executions you can view the performance statistics for that decision service:

Figure 34 Execution Count

 

Details of the monitored attributes can be viewed:

Figure 35 Monitored Attributes

 

 

 

Managing Rules

Corticon Collaborator is the tool that provides this functionality.  Authorized users check in and checkout their rule models, vocabularies and test cases and a complete history of changes and versions is maintained.  Here is an example of what a user might see:

Figure 36 Rule Repository Folders

Within a folder the user will see:

Figure 37 Repository Functions

The menu shows some of the functions that can be performed.   

When a rule model needs to be moved into production, a workflow can be started in Collaborator which moves the assets through a series of approval steps such as notifying approvers by email, coordinating their responses and keeping an audit trail of the approvals

Here’s what a rule reviewer might see in her email inbox:

Figure 38 Email Notification of Rule Models to be Approved

And here is what might appear in the content of the email

Figure 39 Sample Email Notification

By following the link the rule approver can gain access to the rule asset: This may require that she logon

Figure 40 Access to the Repository is Controlled

After Jenny performs her review task, Tom (the rule author) will be notified by email:

Figure 41 Email Notification of Approval

This information is also maintained in the Collaborator audit trails.

Tom can monitor the progress of the approval process as follows:

Figure 42 Tracking Approval Workflows

You can see that there are several overdue tasks.

Tom can view the details of any of these tasks:

Figure 43 Approval Workflow Details

We can see in this example that Jenny (the QA person) approved it, *** (whose participation was optional) was skipped but Harry (the administrator) is holding things up

Corticon supports the deployment of rules and rule sets to various environments (i.e. development, staging, production).

 

Back in Collaborator we saw the folder that the rule author had access to. If we now login as the administrator we see more folders (which the rule author does not have access to). The administrator can see the development folder and also the UAT and production folders

Figure 44 Migrating Rule Assets to Production

In response to the rule author’s workflow, the administrator will migrate the rule assets from Development into UAT. Collaborator will keep a log of this activity. Once in UAT more testing and approval cycles will probably take place until eventually the rules get migrated into production.




 

 

Invoke the production rules via the application code (or BPM)

(note: from this point on a slightly different rule model is used in the example, but the principles are the same)

Once the decision service is deployed on the production server it can be invoked by any SOAP Client or by any BPM software that can make a web services call or from application code such as Java or .NET

Using Other SOAP Clients

These same tests can be executed from any SOAP client such as SoapUI .

In order to do this you must first generate the WSDL for the decision service.

This is done from the deployment console which is accessible in Studio:

Figure 45 Deployment Console in Studio

 

A generic SOAP client such as SOAP UI can be used to set up and run tests against the deployed decision service:

Figure 46 A Typical SOAP Client

The SOAP Request would look like this

Figure 47 Corticon SOAP Request

The results are resulted similarly to what you see in the Corticon tester:

Figure 48 Corticon SOAP Response

In fact you can export this soap message from the test cases in Studio:

Figure 49 Studio Test Data

By selecting from the Test menu

Figure 50 Studio Export SOAP Menu

 

BPM Clients

Corticon also partners with many of the BPM vendors.

Here’s an example of a business process that invokes a decision service.

Most BPMs provide for the import of WSDL and automatically generate the necessary connection to external web services. All that is required to invoke the decision service is to map the process variables to the input variables of the decision service.

Savvion Integration

Figure 51 Decision Service Invoked from a Business Process (Savvion)

 

Process attributes are mapped as inputs to Corticon

Figure 52 Mapping Corticon Inputs

Corticon results (status) is mapped to process attributes:

Figure 53 Mapping Corticon Outputs

Processes are started in the BPM Portal

Figure 54 Starting the Business Process

Inputs are entered on the web form

Figure 55 Web Data Entry Form

These are then passed to the Corticon rule engine

The user’s task list can be viewed:

Figure 56 User's Task List

The results are then displayed

Figure 57 Suspension Status Determined





Savvion Web Application

Statistics Recorded in the Corticon Web Console

Figure 58 Monitoring the Decision Results

 

 

 

Java Clients

Decision Services may also be invoked from programming languages such as Java.

In this case, in addition to the web services interface, you can also make a direct in-process call. To do this the Corticon Server classes are compiled into your java program. Then you can invoke the server class “execute” method and pass the data either as XML or as native java objects.

Here’s an example of the core java code you would write to invoke a decision service:

In-Process Java with Java Objects

Figure 60 In-Process Execution with Java  Payload

The actual code would contain additional statements to populate the FEMAdata object , to catch any errors and to process the results coming back from the rule engine.

Java with XML Payload

Here’s generally how our programmer will make an in-process call to the rules  using an XML payload:

Figure 61 In-Process Execution with XML Payload

Visual Basic Invocation

Figure 62 Invocation from Visual Basic

In-process call with XML

Figure 63 Invocation with XML Payload

Web Services Call from Java

Figure 64 Web Services Call from Java




 

How to Generate Test Data

These two sheets take care of generating all possible test cases:

Figure 65 Specifying the Test Values

Figure 66 Creating Instances of the Client

 

That really is all it takes!

You could get more sophisticated by writing actual rules to decide what base values to create but listing them seems the simplest way.

They could also be read in from a database. But having them in a sheet makes it easy to switch on or off various values for testing.

 

Just add this rule flow and you can run the test date against the suspension rules:

to get

60 test cases generated.

 

 

 

 

Performance

 

 

 

 

 

Why is Corticon so much faster than Traditional Rule Engines?

In a traditional (RETE) rule engine a significant portion of the execution time is allocated to figuring out which rule to execute in which sequence – and if you process 1 million transactions the engine has to do this figuring out a million times.

Corticon does the figuring out ONCE at design time and so execution performance is considerably better than RETE based engines for most business rule applications.

 

Additionally Corticon scales linearly as the number of rules and the amount of data increases. RETE engines eventually reach a point where performance drops off dramatically as the data payload increases. RETE does scale well with increasing number of rules however.