Toward Explainable AI (Part 2): Bridging Theory and Practice—The Two Major Categories of Explainable AI Techniques

Series reminder: This series explores how explainability in AI helps build trust, ensure accountability, and align with real-world needs, from foundational principles to practical use cases.

Previously, in Part I: Why AI Needs to Be Explainable: Understanding the risks of opaque AI.

In this Part: We explore the two main explainability approaches: interpretable models and post-hoc techniques like LIME or SHAP, each with distinct strengths.

3. Overview of Approaches

Contrastive and Counterfactual Explanations

Contrastive explanation plays a key role in interpreting decisions made by artificial intelligence, as it helps answer the critical question: “Why this decision and not another?” Rather than simply offering a static justification, this approach relies on counterfactual scenarios. It involves exploring what would have happened if certain conditions had been different. In other words, it aims to answer questions like: “What would have happened if this parameter had been changed?”

This method is especially relevant in areas where algorithmic decisions have a direct impact on individuals, such as credit approval. For instance, when a customer is denied a loan, an explainable AI system could indicate that a slightly higher income or a different credit history would have led to a favorable decision. This shows that contrastive explanation is not just about justifying a decision, it also provides actionable insights that the user can consider to adjust their behavior or information.

By identifying the decisive factors that influence an outcome, an approach like this helps reduce the perception of arbitrariness and, in turn, strengthens user trust. It is also highly useful for detecting bias, as it allows for checking whether certain population groups are systematically disadvantaged by analyzing how similar variations in input parameters affect the decisions made.

Functional Approaches and Transparency

Some AI approaches focus on providing an intuitive understanding of how the system works, without exposing every step of its algorithmic reasoning. In this case, rather than explaining the internal mechanisms of the model in detail, these methods aim to make the inputs and outputs more interpretable. This approach is especially useful for complex AI models, such as deep neural networks, whose internal processes are often difficult to interpret. Instead of trying to make every computation understandable, the focus is placed on how input variables influence final decisions, by identifying the key factors that shaped the prediction.

For example, in a loan recommendation system, an AI model might indicate that criteria such as monthly income or credit history played a key role in the decision, without necessarily detailing all the internal weightings of the model.

This type of explainability is especially useful for non-expert users, who primarily need to understand a model’s output in order to take action, without having to grasp its mathematical inner workings. It helps build trust in AI and supports its adoption in sectors such as finance, healthcare, and human resources, where automated decisions must be justifiable in a simple and accessible way.

Techniques for Interpreting AI Decisions

Tools like SHAP (SHapley Additive Explanations) and LIME (Local Interpretable Model-Agnostic Explanations) are now widely used to analyze the impact of different variables on a model’s final decision. They make it possible to identify the key factors driving the predictions of a complex model.

We will return to their application in more detail in the second part of this document, through real-world case studies in medicine and finance.

Other approaches complement these tools depending on specific needs :

• Contrastive and counterfactual explanation: These methods make it possible to explain a decision by comparing it to plausible alternatives, answering the question “Why this decision instead of another?".
• Functional approaches focused on transparency: These aim to make a model’s internal workings more accessible without relying on post-hoc tools, primarily by better structuring the algorithm’s inputs and outputs.
• Intrinsically Interpretable Models: Unlike post-hoc methods, these models (explicit decision trees, linear regressions, etc.) are designed to be understandable from the ground up.
• Ontology-Based and Knowledge Graph Methods : By structuring information as logical networks, these methods make it easier to understand the relationships between variables and the decisions made by the AI system.

Wrap-up and what’s next: Different methods serve different needs, but how are they used in the real world?

Next, in part III, we’ll dive into concrete applications of explainable AI in medicine, finance, and the public sector.

Glossary

Algorithmic Bias: Systematic and unfair discrimination in AI outcomes caused by prejudices embedded in training data, model design, or deployment processes, which can lead to disparate impacts on certain population groups. Detecting and mitigating algorithmic bias is a key objective of explainable AI.

Bias Detection (via XAI): Use of explainability methods to identify biases or disproportionate effects in algorithmic decisions.

Contrastive and Counterfactual Explanations: Explanations that compare the decision made to what could have happened by changing certain variables (e.g., “Why this outcome instead of another?”).

Decision Plot: A graphical representation tracing the successive impact of variables on an individual prediction.

Evaluation Metrics for Explainability: Criteria used to assess the quality of an explanation (fidelity, robustness, consistency, etc.).

Feature Importance (Variable Contribution): Measurement or attribution of the relative impact of each variable on the model’s final decision.

Force Chart (Force Plot): An interactive visualization illustrating positive or negative forces exerted by each variable on a prediction.

Fidelity: A measure of how faithfully the explanation reflects the true logic of the model.

Global Explanation: An overview of the model’s behavior across the entire dataset.

Human Interpretability: The quality of an explanation to be understood and useful to a human, non-expert user.

Intrinsically Interpretable Models: Models whose very structure allows direct understanding (e.g., decision trees, linear regressions).

LIME (Local Interpretable Model-agnostic Explanations): A local explanation method that generates simple approximations around a given prediction to reveal the influential factors.

Local Explanation: A detailed explanation regarding a single prediction or individual case.

Model Transparency: The quality of a model in making its decision-making processes accessible and understandable.

Post-hoc Explainability: Explainability techniques applied after model training, without altering its internal functioning.

Robustness (Stability): The ability of an explainability method to provide consistent explanations despite small variations in input data.

SHAP (SHapley Additive exPlanations): An approach based on game theory that assigns each variable a quantitative contribution to the prediction, providing both global and local explanations.

Summary Chart (Summary Plot): A visualization ranking variables according to their average influence on predictions.

Waterfall Chart: A static visualization showing step by step how each variable contributes to the final prediction.

References

1. LIME: [link]

2. Chest X-Ray Images (Pneumonia): [link]

3. German Credit Dataset: [link]

4. Complete notebook for the LIME case study: [link] 

5. Complete notebook for the SHAP case study: [link] 

6. Leslie, D., Rincón, C., Briggs, M., Perini, A., Jayadeva, S., Borda, A., et al. (2024). AI Explainability in Practice, The Alan Turing Institute, AI Ethics and Governance in Practice Programme. Participant & Facilitator Workbook. https://www.turing.ac.uk/sites/default/files/2024-06/aieg-ati-7-explainabilityv1.2.pdf

7. AI explainability in practice. (2024). The Alan Turing Institute. https://www.turing.ac.uk/sites/default/files/2024-06/aieg-ati-7-explainabilityv1.2.pdf

8. What is AI (artificial intelligence)? (2024, April 3). McKinsey & Company. https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-ai

9. Kite-Powell, J. (2022, July 28). Explainable AI is trending and here’s why. Forbes. https://www.forbes.com/sites/jenniferhicks/2022/07/28/explainable-ai-is--trending-and-heres-why

10. Excellence and Trust in AI — Brochure. (n.d.). Shaping Europe’s Digital Future. https://digital-strategy.ec.europa.eu/en/library/excellence-and-trust-ai-brochure

11. Heikkilä, M. (2024, March 6). Nobody knows how AI works. MIT Technology Review. https://www.technologyreview.com/2024/03/05/1089449/nobody-knows-how-ai-works/

12. Heaven, W. D. (2025, July 30). Large language models can do jaw-dropping things. But nobody knows exactly why. MIT Technology Review. https://www.technologyreview.com/2024/03/04/1089403/large-language-models-amazing-but-nobody-knows-why/

13. Francis, J. (2024, November 15). Council Post: Why 85% of your AI models may fail. Forbes. https://www.forbes.com/councils/forbestechcouncil/2024/11/15/why-85-of-your-ai-models-may-fail

14. Bao, A., & Zeng, Y. (2024). Understanding the dilemma of explainable artificial intelligence: a proposal for a ritual dialog framework. Humanities and Social Sciences Communications, 11(1). https://doi.org/10.1057/s41599-024-02759-2

15. Gallese, C. (2023). The AI Act proposal: a new right to technical interpretability? arXiv (Cornell University). https://doi.org/10.48550/arxiv.2303.17558

16. GALDON CLAVELL, G. (n.d.). AI Auditing - Checklist for AI auditing. In SUPPORT POOL OF EXPERTS PROGRAMME. https://www.edpb.europa.eu/system/files/2024-06/ai-auditing_checklist-for-ai-auditing-scores_edpb-spe-programme_en.pdf

17. White, B., & Guha, K. (2025, June 14). Component based Quantum Machine Learning explainability. arXiv.org. https://arxiv.org/abs/2506.12378