Decisioning and Explainability Using Red Hat Decision Manager

Overview

The abundance of computational power and data has made Intelligent Decisioning a reality today. An interesting aspect of this is to provide context-aware actions like providing hyper-personalized customer experience and smart service management. With data comes responsibility. Enabling explainability metrics around these decisions is essential to both the customers as well as businesses. With data regulation rules, customers have the right to understand why certain decisions are made. The same benefits the organizations as well, as it provides a clearer picture of how the decision models are performing allowing them to be improved based on market needs.

In a previous article, I described why having real-time business insights is such a critical piece of a business process system. I also described what it takes to enable integration between Red Hat Process Automation Manager and Elasticsearch. In this article, let us see how we can push Decision metrics to Elasticsearch.

The DMN (Decision Model Notation) standard provides a graphical notation for defining Decision flows. Based on open standards, DMN provides a way for business users to easily create and extend the decision artifacts without having to depend on IT. Having traceability in the decision execution metrics is key for identifying trends and enforcing process improvements.

As we discussed in the previous article, Elasticsearch provides full-text search capabilities that can be used to create a business-friendly dashboard using Kibana.

Customer Eligibility Usecase

Let us now look at a sample use case for our demonstration. We have a Customer Eligibility Decision which determines the eligibility of a customer for a proposed service. Decision Manager allows you to pull in PMML models to be executed as a part of the decision process. In this example, we can see that Risk Checks are performed on a Machine Learning model (Risk Model).

As is evident from the Decision model, there are multiple factors that determine eligibility. Also, there are multiple levels at which the eligibility criteria is determined. Having auditing at a granular level, helps us understand the taxonomy of the business case. 

DMN Listener for Elastic Integration

Red Hat Decision Manager provides a way to register a DMN listener in order to be notified of several events during DMN model evaluations. The CustomDMNListener will be used to push Decision execution metrics to Elasticsearch. 

A sample implementation of the listener can be found here.

Let us look at the key highlights of the listener.

First, we use the HttpClientBuilder to authenticate and connect to the elastic cluster. Note that for the purpose of this exercise, we have bypassed the certificate authentication. In production, it is necessary to authenticate using a valid certificate.

xxxxxxxxxx

protected static CloseableHttpClient buildClient() throws Exception{

        HttpClientBuilder builder = HttpClients.custom();

        if (elasticSearchUser != null && elasticSearchPassword != null) {

            SSLContextBuilder builder1 = new SSLContextBuilder();

            builder1.loadTrustMaterial(null, new TrustStrategy() {

                @Override

                public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException {

                    return true;

                }

            });

            CredentialsProvider provider = new BasicCredentialsProvider();

            UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(elasticSearchUser, elasticSearchPassword);

            provider.setCredentials(AuthScope.ANY, credentials);

            SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(builder1.build(),  new NoopHostnameVerifier());

            builder.setDefaultCredentialsProvider(provider);

            builder.setSSLSocketFactory(sslConnectionSocketFactory).build();

        }

        return builder.build();

}

Next, we define a simple Audit object which we will use to push metrics to Elastic.

x

public class AuditObject {

    private String decisionName;

    private Date decisionExecutedAt;

    private Object inputs;

    private String status;

    private Object result;

   ....

We then override the afterEvaluateDecisionEvent. This is where we populate the Audit Object and finally perform a POST to the Elastic URL.

x

@Override

public void afterEvaluateDecision(AfterEvaluateDecisionEvent e) {

DecisionNode decisionNode = e.getDecision();

String decisionNodeName = decisionNode.getName();

DMNDecisionResult result = e.getResult().getDecisionResultByName(decisionNodeName);

//populate the Audit Object

AuditObject auditObject = new AuditObject();

auditObject.setDecisionName(decisionNodeName);

auditObject.setDecisionExecutedAt(new Date());

auditObject.setStatus(result.getEvaluationStatus().name());

Map<String, Object> mapInputs = e.getResult().getContext().getAll();

Map<String, Object> auditMap = new HashMap<>();

auditMap.put(str, String.valueOf(mapInputs.get(str)));

auditObject.setInputs(auditMap);

auditObject.setResult(String.valueOf(result.getResult()));

String json = new Gson().toJson(auditObject);

//POST to Elastic REST URL

HttpPost httpPut = new HttpPost(elasticSearchUrl + "/cust/cust");

httpPut.setEntity(new StringEntity(json, "UTF-8"));

httpPut.setHeader("Content-Type", "application/json");

httpPut.setHeader("Accept", "application/json");

. . .

}

Business User Visualization

Now that the data has been pushed to Elastic, let us now visualize the data using visual dashboards so that the business user can understand the trends of the Rule execution.

Let us first look at a minimal dashboard visualization of the Rule Metrics using Kibana. The visualization below shows a monitoring view, categorizing the rules by name, time, and result.

Most of the time, business-specific dashboards are important to understand the explainability of the decisions. As we described in our example, there are several factors that determine Customer Eligibility. Reporting on these factors will help better understand how the decisions are being made. Here is a sample visualization.

We are now able to gather insights into the various factors that go in, in determining Customer Eligibility. Not only are we able to visualize overall trends, but we can also drill down into specific customer data. Let us look at the breakdown for a customer.

We can clearly see how the eligibility decision was made based on all the variables involved in the equation.

Summary

This approach also allows us to measure the effectiveness of the Decision and AI models used in conjunction with profile-based decisions. By providing an ability for the business users to make context-aware decisions provides for both auditability as well as the ability to measure the effectiveness of decision models.