Data is the lifeblood of companies today. Not only does day-to-day functioning rely on a constant feed of data about every aspect of operations, but it’s also becoming increasingly clear that with enough data and the right analysis, previously intractable problems can be solved and processes improved. It should come as no surprise that data science is currently ranked #2 on Glassdoor’s 2021 list of best jobs in the US (and has been #1 for 4 of the past 6 years).
But as the 2018 Facebook/Cambridge Analytical political scandal made clear to the world, modern methods for gathering and analyzing large amounts of data can also raise ethical issues. After that scandal broke, the whole world started forming an opinion about how data may and may not be used, kicking off what might be called the age of data ethics. Couple this with existing and emerging legislation that aims to limit how much customer data can be collected and for what purpose, and the bottom line is: if your company uses customer data to make customer-facing decisions, the ethical and legal issues involved in making those decisions must be considered.
Experts in ethical thinking
In our last blog, we described the kinds of issues that can arise as the result of negligent data usage. We also touched on the numerous frameworks that your data scientists can use as primers to begin thinking about the potential issues that may be posed by their work. But as data increasingly becomes a driver of decision making across an organization, awareness of data ethics needs to expand beyond your data science team. Avoiding unwitting traps means considering the ethics of every data use case as an integral part of your organization’s processes.
IG&H helps prevent companies from falling into ethical data traps. The ethics of data usage is very much a part of our DNA. Every IG&H consultant is trained on data security and ethics to some degree. We also recently established an Inclusivity Diversity Equity and Awareness (IDEA) group for people within IG&H whose passion is to think about how ethics and fairness impact business decision-making. When working on a data project, the IDEA group helps ensure that our data scientists are aware of the latest trends regarding the ethics of leveraging customer data.
Ethical data usage frameworks
Often companies think addressing ethical issues will be unfavorable for the business, or somehow “break” what they do. But in our experience, this is never the case. IG&H specializes in helping companies reach their goals with data in an ethical way.
To that end, we have created an Ethical Risk Quick Scan for helping clients quickly assess whether their data use case may cause an ethical risk. It is difficult to evaluate risk and define required measures for use cases for which solutions have not yet been designed, let alone built. Nevertheless, the ethical risk is a crucial criterion in prioritizing use cases and choosing an approach. Therefore, we developed a Quick Scan to be used precisely during this early stage of use case selection and requirements gathering. It gives us a feel for the level and areas of risk involved early on in a project, and therefore in which domains extra attention is necessary.
The Quick Scan looks like this:
We have designed the Quick Scan to provide a visual representation of potential ethical trouble spots. The analysis is done by filling in the dots as they correspond to the details of the case.
Here is a sample of the thinking involved in determining the extent of the check questions: Imagine a taxi service that digitally monitors many aspects of the rides that are assigned and carried out by their taxi drivers. They want to develop a model that will score driver performance based on this data and automatically adjust driver paychecks accordingly.
Let's assess this use case with the framework:
Vulnerable people impacted? The people impacted typically have below-average financial means and some may live paycheck-to-paycheck.
Number of people impacted large? This is an internal application that does not impact many people (only the company’s taxi driver employees).
‘Matters of life’ affected? The model’s decisions will affect matters of life since they will impact financial wellbeing and job security.
Influencing personal behavior? The model’s decisions may influence the behavior of the taxi drivers in the sense that they might stimulate taxi drivers to work longer hours, or to accept rides to or from locations they are not comfortable with.
No, or slow, feedback loop? The effect of the model’s decisions will be visible quickly (every week or month), so there will be a fast feedback loop.
No human in the loop? The use case calls for autonomous performance scoring and paycheck adjustment, which means there is no human in the loop.
Bias in the data? Performance is typically a subjective concept, so the data the model will be trained on may be biased.
Personal Data used? Finally, the data will involve fine-grained personal location data, which is considered personal data.
Here’s how the Quick Scan would be filled in based on this particular use case:
For each of the areas that have red dots filled in, actions to mitigate potential issues must be considered or taken. Additionally, it is recommended to pay extra attention to areas that have the ‘Medium Risk’ dot filled in. The table below provides a sample of possible mitigations for the biggest trouble spots in this case.
The Quick Scan focuses attention on measures and mitigations that can be taken before a project is even underway. Once a project is in full swing, and as the actual solution and data requirements become increasingly clear, the team can take its evaluation to the next level with the Ethical Risk Evaluation Framework. This further examines at-risk dimensions in terms of the commonly accepted ethical guidelines of Safety, Fairness, Transparency, and Privacy. IG&H distilled these guidelines from a meta study done on 36 prominent AI principles documents.
Ensuring an ethical transformation
During a transformation, IG&H's solution teams work alongside our sector experts to ensure the seamless interplay between data, analytics, technology, and business skills. At the same time, IG&H's solution teams ensure that data usage is optimized to effectively realize business objectives without running afoul of ethical data usage guidelines and regulations.
In our next article we intend to shed light on the types of actions we can take based on the Quick Scan outcomes using our Ethical Risk Evaluation Framework to ensure Safety, Fairness, Transparency, and Privacy. Using these guidelines, we can eliminate and/or reduce identified risks and make the use of AI not just feasible, but also viable and desirable.
Download the Ethical Risk Quick Scan Framework and assess your own data project
Contact
Tom Jongen
Floor Komen