Expert-based AI is a strong and validated way to bring AI models to the workplace. Good methods have been developed whereby banking experts label cases, after which the algorithm learns the relationship between the input variables and the desired outcome. Practice shows that the models are often better able to make the right decision than a human individual. The method and technique can be used for credit acceptance, portfolio monitoring or customer qualification, among other things. Advantages compared to classical approaches are that there is no problem with a lack of historical data. In addition, the employees understand the model results, which increases the degree of grip and acceptance.
Mismatch with traditional approaches
Forward-thinking commercial service providers make gains by (partially) automating decisions that require scarce, specialist capacity. This includes decisions like the acceptance of insurance and loans, credit risk review and renewal of credits. This traditionally relationship-driven domain generally struggles with a lagging investment in technology[1], lower data availability, higher complexity and a dominant position of human experts.
Typically, business analysts and data scientists see only two ways to develop an automated decision model: a) the creation of business rules by experts or b) the application of Machine Learning (AI) to historical data. However, both methods suffer from the aforementioned struggles in the Business domain. In particular, they suffer from the higher product and process complexity and the lower maturity level of data management.
With business rules, more complex patterns can only be captured by exponential growth of the number of rules. Moreover, defining the details per rule and maintaining the whole thing is labor-intensive.
An AI model based on historical data does not perform well when the historical data is limited or of poor quality. In addition, these types of models are slow to adapt to new circumstances or new policies because they can only learn from historical results.
An alternative approach is possible
A method that does not have these disadvantages and that we successfully apply in practice is Expert-based Learning. In doing so, we develop a decision model by training an AI algorithm based on expert input. In this method, experts label the desired outcome of so-called training cases, after which the algorithm learns the relationship between the input variables and the desired outcome from these examples.
This approach does not suffer from any lack of historical data (quality) and results in models that are easily understood and accepted by the specialists who work with them. Moreover, the models are easy to control in practice and can be adapted very quickly to changing market conditions or policies. We illustrate this with one of our practical cases.
Client case
Our client is a market leader in corporate lending. Many manual processes led to significant efficiency losses and have therefore prevented optimal risk management, scalability, and focus on new business. The first process that IG&H has automated together with the client is the annual credit risk review process. In this review process, the client's risk is assessed once per year. This process was entirely manual and very specialized. Now, an AI model reviews 75% of the cases automatically, which results in freeing up a significant number of FTE, as well as an improvement in quality and scalability.
As a starting point, we developed a rule-based review engine together with the experts. This engine was able to identify many high-risk items. However, it also had a lot of by-catch of less risky items (false positives). This would limit the workload reduction. Moreover, maintaining the rules so that they would continue to reflect current risk opinion was labor-intensive. That is why an AI application was chosen, and partly due to the lack of historical data, we used the Expert-Based Learning approach here.
Together with credit risk experts, we developed a decision model in the following four steps:
1) Selection of relevant attributes for the prediction of need for manual review
2) Compiling a dataset with several hundred client cases and the relevant attributes
3) Assignment of outcome labels to the training cases by the experts
4) Training AI model on the compiled training dataset and integrating it into a decision model
The model is found to be able to assess new cases very accurately for the need of a manual revision and has far fewer false positives than the rule-based engine. The model assesses the risk even better than a human expert. The realized workload reduction of the model is 75% of all required reviews.
When the world was taken by surprise with the COVID-19 crisis, the model had to reflect the revised risk opinion as soon as possible. Thanks to the Expert-based Learning method, this was easily and quickly achieved.
What makes the difference?
The development and adoption of the Expert-based Learning method has its challenges. IG&H has developed various methods that make a difference in this process. This ensures the approach is used quickly and successfully. Four of our best practices are:
1) Minimizing the workload for the experts
Limit the number of cases in the training dataset, making labeling the cases efficient for the experts. Due to the lower volume, the consistency of the labeling is important, the algorithm should not be confused when learning the patterns. We work with a fast iterative method to achieve consistency and consensus between different experts with an efficient 4-eye principle.
2) Being aware of human capacity
Optimize the amount of information that goes into the model. Make sure that as much relevant information as possible is offered, but in such a way that the experts can oversee the information when they label the training data. To select the right data attributes, we use a combination of data analysis and interview techniques.
3) Realizing trust and insight
Make sure that the model and the results remain explainable for the customer, auditor, and specialists. We ensure that an AI model can be explained in three ways; firstly with insight into the general operation of the model; secondly with an explanation per outcome/case; and thirdly with interactive dashboarding for monitoring and analysis of input and output of the model.
4) Encouraging adoption throughout the process
The motivation of both the experts who train the model and the specialists who work with the results is an essential success factor. Their close involvement in training and gamification during the process greatly encourages the adoption of the resulting model.
Realizing potential
Data Science and AI make the application of expertise scalable, consistent, and efficient. This allows business specialists to focus on the customer and on situations where their capabilities really matter. The B2B financial services domain in particular can benefit from an approach that requires fewer data and provides models that are embraced by the experts and that are also quickly adaptable to changing circumstances. The necessary interplay between business and data science is our expertise.
Curious about how our experience and technology are of value to you?
We are happy to talk to you.
Get in touch:
Simone Mensink
Director Banking
[1] https://www.consultancy.nl/nieuws/30011/data-science-biedt-kansen-aan-de-zakelijke-verzekeringsma