Commercial financial service providers can significantly improve the efficiency and results of their processes. With the appropriate deployment of Data Science and AI techniques, IG&H proves that AI will also play a crucial role in the future of B2B financial services. To do this, we use a method that combines the best of both worlds: we let AI models learn from experts and experts learn from models.
Mismatch with traditional approaches
Forward-thinking commercial service providers gain by (partially) automating decisions that require scarce, specialist capacity. This includes decisions like the acceptance of insurance and loans, credit risk review, and renewal of credits. But in general, this traditionally relationship-driven domain struggles with a lag in investment in technology1, lower data availability, higher complexity and a dominant position of human experts.
Typically, business analysts and data scientists see only two ways to develop an automated decision model; a) the creation of business rules by experts, or b) the application of Machine Learning (AI) to historical data. However, both methods suffer from the aforementioned struggles in the Business domain. In particular from the higher product and process complexity and the lower maturity of data management.
With business rules, more complex patterns can only be captured by exponential growth of the number of rules. Moreover, defining the details per rule and maintaining the whole thing can be a lot of work.
An AI model based on historical data does not always perform well when the historical data is relatively limited or of poor quality. In addition, these types of models are slow to adapt to new circumstances or new policies because they can only learn from historical results.
An alternative approach is possible
A method that does not have these disadvantages and that we successfully apply in practice is Expert-based Learning. In doing so, we develop a decision model by training an AI algorithm based on expert input. In this method, experts label the desired outcome of so-called training cases, after which the algorithm learns the relationship between the input variables and the desired outcome from these examples.
This approach does not suffer from any lack of historical data (quality) and results in models that are easily understood and accepted by the specialists who will work with them. Moreover, the resulting models are easy to control in practice and can be adapted very quickly to changing market conditions or policies. We illustrate all this below with one of our practical cases.
Our client is a market leader in corporate lending. Many manual processes led to significant efficiency losses and therefore prevented optimal risk management, scalability, and focus on new business. The first process that IG&H has automated together with the client is the annual credit risk review process. In this review process, the client's risk is assessed once per year. This process was entirely manual and very specialized. Now, an AI model reviews 75% of the cases automatically, which results in freeing up a significant number of FTE, as well as an improvement in quality and scalability.
As a starting point, we developed a rule-based review engine together with the experts. This engine was able to identify many high-risk items. However, it also had a lot of by-catch of less risky items (false positives). This would limit the workload reduction. Moreover, maintaining the rules so that they would continue to reflect current risk opinion was labor intensive. That is why an AI application was chosen, and partly due to the lack of historical data, we used the Expert-Based Learning approach here.
Together with credit risk experts, we developed a decision model in the following steps:
1. Selection of relevant attributes for the prediction of need for manual review
2. Compiling a dataset with several hundred client cases and the relevant attributes
3. Assignment of outcome labels to the training cases by the experts
4. Training AI model on the compiled training dataset and integrating it into a decision model
The resulting model is found to be able to assess new cases very accurately for the need of a manual revision and has far fewer false positives than the rule-based engine. The model assesses the risk even better than an individual expert. The realized workload reduction of the model is 75% of all required reviews.
When everyone was surprised by the onset of the corona crisis, the model had to reflect the revised risk opinion as quickly as possible. Thanks to the Expert-based Learning method, we were able to achieve this easily and quickly. We immediately created a new training set and retrained the AI model. As a result, the review decision model was adapted to the new risk policy within a few weeks, to everyone's satisfaction.
Meanwhile, the AI model has been validated and approved by a model validation party and implemented in the IT architecture of the customer. Following the successful implementation of the credit risk review model, we took the next step and developed an AI model for credit extensions using the same methodology. This second model is already running successfully in a pilot.
What makes the difference
The development and adoption of the expert-based learning method has its challenges. We have gradually developed various methods with which we can make a difference and we can deploy this approach quickly and successfully. Here are four of our best practices.
Limit the workload for the experts
Limit the number of cases in the training dataset so that labeling the cases does not take up too much of the experts' time. Because of this lower volume, the consistency of labeling is incredibly important so that the algorithm is not confused when learning the patterns behind these sparse examples. We work with a fast iterative method to achieve consistency and consensus between different experts with an efficient 4-eye principle.
Be aware of human capacity
Optimize the amount of information that goes into the model. Make sure that as much relevant information as possible is offered, but in such a way that the experts can oversee the information when they label the training data. To select the right data attributes, we use a combination of data analysis and interview techniques.
Realize trust and insight
Make sure that the model and the results remain explainable for the customer, auditor, and specialists. We ensure that an AI model can be explained in three ways; firstly with insight into the general operation of the model; secondly with an explanation per outcome/case; and thirdly with interactive dashboarding for monitoring and analysis of input and output of the model.
Encourage adoption throughout the process
The motivation of the experts who train the model and the specialists who will work with the results is an essential success factor. Their close involvement in training and gamification during the process greatly encourages the adoption of the resulting model.
Data Science and AI make the application of expertise much more scalable, consistent, and efficient. This allows business specialists to focus more on the customer and on situations where their capabilities really matter. The B2B financial services domain in particular can benefit greatly from an approach that requires fewer data and provides models that are embraced by the experts and that are also quickly adaptable to changing circumstances. The necessary interplay between business and data science is our expertise.
Sr manager Banking