Background: The need for a potentially wrong and unjust algorithmic decision to be explained.
Governments around the world are relying increasingly on decision making powered by artificial intelligence. Sooner or later, they will make numerous decisions that are important to citizens, based on machine-learning (ML) algorithmic treatment (hereafter referred to as “algorithmic decisions”).
An issue arises because algorithmic decisions can be potentially misleading and biased and can wrongfully infringe on the rights and welfare of those who are affected by these decisions (hereafter referred to as “affectees”). This issue points to the importance of safeguarding affectees’ right to an explanation for an algorithmic decision and offering explanations that render a decision transparent, accountable, and contestable.
In recent years, there have been moves to promote the right to an explanation through the passage of laws and regulations, most notably European Union’s General Data Protection Regulation. Such moves have made explainable artificial intelligence (XAI), which is designed to make an AI system’s decisions and behaviors intelligible to humans by offering explanations, ever more important.
Our Research Question: Does the type of explanation matter to people’s attitudes towards an algorithmic decision?
From a legal perspective, providing explanations for algorithmic decisions is important, so affectees can challenge a decision if necessary. In our paper, we argue, from societal and human-centric perspectives, that it is important for members of society to perceive that public authorities are making fair, accurate, and trustworthy algorithmic decisions in light of the aforesaid issues. Explanations might be able to help promote positive attitudes towards the decisions.
We investigated whether the status and the type of explanation matters to the fairness, accuracy, and trustworthiness of an algorithmic decision, as perceived by the affectees. The XAI literature points to the fact that explanations for algorithmic decisions can differ in type with respect to the kind of information they present; they can be
- input-based, explaining how much an input variable influences the output,
- group-based, showing the outcome distributions across groups in the AI’s training data,
- case-based, presenting the cases in the AI’s training data that are most similar to the inquirer’s case, or
- counterfactual, pointing out what would have had to be different to yield a desirable outcome.
Inspired in part by psychological studies examining factors that have an impact on a person’s attitudes towards an outcome they receive, we posited that the perceived fairness, accuracy, or trustworthiness of an adverse algorithmic administrative decision depends on the type of explanation, because explanations differ in scope; some are global, explaining how an overall model works, while others are local and instance-specific, explaining the outcome in the case in question. Explanations also differ with respect to the points of comparison for the assessment of the distributive justice they provide, and with respect to the impression they give about the extent to which the AI system is subject to monitoring and correction.
Our Pre-Registered Studies: Experiments using scenarios involving decisions regarding a grant application and a tax inspection.
To test our hypotheses, we conducted two studies in December 2022, each of which involved an online survey experiment, pre-registered via the Open Science Framework. In both studies, the subjects were officers in high positions at stock companies registered in Japan, and they were presented with a hypothetical scenario consisting of an algorithmic decision made by a public authority: a ministry’s decision to reject a grant application from their company (Study 1) and a tax authority’s decision to select their company for an on-site tax inspection (Study 2).
In both studies, we randomly assigned one of the experimental conditions (E1 to E5) to the subjects: they were told that no explanation was given (E1) or they were given a generic description of the type of explanation offered — namely, input-based (E2), group-based (E3), case-based (E4), or counterfactual (E5). We then asked all the subjects to assess whether the decision was fair, accurate, and trustworthy, and investigated how their attitudes differed across experimental groups.
Findings and Implications: It’s not just about offering any explanation – types matter.
Our studies revealed that offering the subjects some types of explanations had a positive impact on their attitude towards a decision, to various extents, except in the case of fairness in the tax inspection study. However, the detailed results were not robust across studies and decision domains. One reason for the discrepancies might have had to do with the nature of the decisions involved and differences in the types of explanations the subjects were seeking in order to develop their attitude towards the decision.
It is most important for algorithmic decisions to be actually fair, accurate, and trustworthy, but even if they are, whether members of the public perceive the decisions as such is a different issue. While more studies are needed to build strong evidence, our studies suggest that public authorities should consider providing some sort of explanation for an algorithmic decision to promote affectees’ perceptions of its fairness, accuracy, and trustworthiness. In doing so, they are reminded of the fact that some types of explanations might work better than others and that the effects might differ across decision domains.
Research Team
Naomi Aoki (Graduate School of Public Policy)
Tomohiko Tatsumi (Graduate Schools for Law and Politics / Faculty of Law)
Go Naruse(Graduate Schools for Law and Politics / Faculty of Law)
Maeda Kentaro (Graduate School of Public Policy / Faculty of Law)
Paper Information
Aoki, N., Tatsumi, T., Naruse, Go., & Maeda, K. (2024). Explainable AI for government: Does the type of explanation matter to the accuracy, fairness, and trustworthiness of an algorithmic decision as perceived by those who are affected? Government Information Quarterly, 41(4), 101965.
https://doi.org/10.1016/j.giq.2024.101965