Toward Explainable AI (Part 9): Bridging Theory and Practice—Conclusion: Explainability Under Real-World Conditions

Series reminder: This series explores how explainability in AI helps build trust, ensure accountability, and align with real-world needs, from foundational principles to practical use cases.

Previously, in Part VIII: SHAP: Powerful, But Can We Trust It? Strengths and vulnerabilities of SHAP explanations.

In this Part: We compare LIME and SHAP across multiple criteria, highlight their complementarity, and draw practical lessons for real-world implementation.

Summary: Explainability Put to the Test in Real-World Conditions

Through these two use cases, pneumonia detection with LIME and credit risk assessment with SHAP, we have highlighted how explainability can partially lift the veil on the functioning of algorithmic models often perceived as opaque.

In the medical field, LIME allowed visualization of the areas of an X-ray that most influenced a deep learning model’s decision. This ability to localize the source of a prediction helps build trust among healthcare professionals, where even the slightest error can have critical consequences.

In the financial sector, SHAP transformed a high-performing but hard-to-interpret model into a tool whose decisions become understandable to business analysts and even to the clients themselves. Detailed variable analysis makes it possible to explain a credit denial or tailor sales messaging.

Each method has its strengths and limitations. LIME offers an intuitive, local, and visual interpretation, but can sometimes be unstable. SHAP provides solid mathematical foundations and both global and individual granularity, albeit with a higher computational cost. Neither approach is a universal solution, but their combination allows for cross-analysis, increasing the robustness of interpretations and adapting explainability to business needs.

Explainability cannot be limited to mere explanation. 

The explanations must be reliable, useful, and understandable by their recipients. This requires rigorous evaluation of the tools themselves, including stability testing, bias detection, consistency across approaches, and so on. The goal is to avoid false transparency.

Explainability cannot be limited to mere explanation. The explanations must be reliable, useful, and understandable by their recipients. This requires rigorous evaluation of the tools themselves, including stability testing, bias detection, consistency across approaches, and so on. The goal is to avoid false transparency.

To advance this goal, further development of methods and metrics for assessing explanation quality in operational contexts is required. Concrete examples of fidelity and robustness testing applied to real-world use cases are needed, such as measuring the fidelity of LIME’s superpixels or the stability of SHAP values against small, non-Gaussian perturbations. 

User studies with domain experts (e.g., physicians or financial analysts) are necessary to assess explanation usefulness and clarity. It means that technical metrics need to be combined with human feedback. While important for practical explainability, these topics fall outside this document’s scope, which centers on foundational concepts and applications.

Finally, this applied exploration reminds us that the goal of explainable AI is not so much to understand everything as to make better decisions. In a context where models become increasingly complex, such as large language models and multimodal architectures, it is clear that designing systems capable not only of predicting but also of explaining, justifying, and auditing their decisions is essential.

Only under these conditions can AI truly serve humans: powerful, yes, but also understandable, fair, and trustworthy.

Wrap-up and what’s next: No single method fits all needs. Choosing the right explainability tool depends on context, constraints, and goals.

Next, in part X, we’ll look beyond the tools to question what it really takes to build responsible AI.

Links to the previous articles published in this series:

• Toward Explainable AI (Part I): Bridging Theory and Practice—Why AI Needs to Be Explainable
• Toward Explainable AI (Part 2): Bridging Theory and Practice—The Two Major Categories of Explainable AI Techniques
• Toward Explainable AI (Part 3): Bridging Theory and Practice—When Explaining AI Is No Longer a Choice
• Toward Explainable AI (Part 4): Bridging Theory and Practice—Beyond Explainability, What Else Is Needed
• Toward Explainable AI (Part 5): Bridging Theory and Practice—A Hands-On Introduction to LIME
• Toward Explainable AI (Part 6): Bridging Theory and Practice—What LIME Shows – and What It Leaves Out
• Toward Explainable AI (Part 7): Bridging Theory and Practice—SHAP: Bringing Clarity to Financial Decision-Making
• Toward Explainable AI (Part 8): Bridging Theory and Practice—SHAP: Powerful, But Can We Trust It?

Glossary

Algorithmic Bias: Systematic and unfair discrimination in AI outcomes caused by prejudices embedded in training data, model design, or deployment processes, which can lead to disparate impacts on certain population groups. Detecting and mitigating algorithmic bias is a key objective of explainable AI.

Bias Detection (via XAI): Use of explainability methods to identify biases or disproportionate effects in algorithmic decisions.

Contrastive and Counterfactual Explanations: Explanations that compare the decision made to what could have happened by changing certain variables (e.g., “Why this outcome instead of another?”).

Decision Plot: A graphical representation tracing the successive impact of variables on an individual prediction.

Evaluation Metrics for Explainability: Criteria used to assess the quality of an explanation (fidelity, robustness, consistency, etc.).

Feature Importance (Variable Contribution): Measurement or attribution of the relative impact of each variable on the model’s final decision.

Force Chart (Force Plot): An interactive visualization illustrating positive or negative forces exerted by each variable on a prediction.

Fidelity: A measure of how faithfully the explanation reflects the true logic of the model.

Global Explanation: An overview of the model’s behavior across the entire dataset.

Human Interpretability: The quality of an explanation to be understood and useful to a human, non-expert user.

Intrinsically Interpretable Models: Models whose very structure allows direct understanding (e.g., decision trees, linear regressions).

LIME (Local Interpretable Model-agnostic Explanations): A local explanation method that generates simple approximations around a given prediction to reveal the influential factors.

Local Explanation: A detailed explanation regarding a single prediction or individual case.

Model Transparency: The quality of a model in making its decision-making processes accessible and understandable.

Post-hoc Explainability: Explainability techniques applied after model training, without altering its internal functioning.

Robustness (Stability): The ability of an explainability method to provide consistent explanations despite small variations in input data.

SHAP (SHapley Additive exPlanations): An approach based on game theory that assigns each variable a quantitative contribution to the prediction, providing both global and local explanations.

Summary Chart (Summary Plot): A visualization ranking variables according to their average influence on predictions.

Waterfall Chart: A static visualization showing step by step how each variable contributes to the final prediction.

Links

1. LIME: [link]

2. Chest X-Ray Images (Pneumonia): [link]

3. German Credit Dataset: [link]

4. Complete notebook for the LIME case study: [link] 

5. Complete notebook for the SHAP case study: [link] 

6. Leslie, D., Rincón, C., Briggs, M., Perini, A., Jayadeva, S., Borda, A., et al. (2024). AI Explainability in Practice, The Alan Turing Institute, AI Ethics and Governance in Practice Programme. Participant & Facilitator Workbook: [link]

7. What is AI (Artificial Intelligence)? McKinsey & Company, April 2024: [link]

8. Jennifer Kite-Powell. Explainable AI Is Trending And Here’s Why. Forbes, 2022. Disponible sur: [link]

9. European Commission. Excellence and Trust in AI, European Commission, 2020: [link]

10. Melissa Heikkilä (2024). Nobody really knows how AI works — and that’s a problem. MIT Technology Review: [link]

11. Heaven, W. D. (2023) - Large language models can do jaw-dropping things. But nobody knows exactly why. MIT Technology Review: [link]

12. Forbes - Jameel Francis, “Why 85% of Your AI Models May Fail”, Forbes, 15 Nov. 2024: [link]

13. A. Bao & Y. Zeng, “Understanding the dilemma of explainable artificial intelligence,” Nature -  Humanities and Social Sciences Communication (2024): [link]

14. C. Gallese, The AI Act proposal: a new right to technical interpretability?, arXiv preprint, arXiv:2303.17558 [cs.CY], 2023: [link]

15. European Data Protection Board. (2024). Checklist for AI Auditing Scores – EDPB Support Pool of Experts Programme. June 2024: [link]

16. White, B., & Guha, K. (2025). Component-Based Quantum Machine Learning Explainability. arXiv preprint arXiv:2506.12378: [link]