42% of AI Projects Collapse in 2025 — The Battle-Tested Framework Wall Street Uses

1. The Context: AI’s ‘Wild West’ Problem

In 2018, a chilling discovery was made within the tech giant Amazon. Its experimental AI recruiting tool, designed to streamline the hiring process by analyzing resumes, had developed a significant bias against women. The system, trained on a decade’s worth of hiring data, had learned to penalize resumes containing the word “women’s,” as in “women’s chess club captain,” and downgraded graduates of two all-women’s colleges. Amazon ultimately scrapped the project, but the incident served as a stark warning about the unintended consequences of artificial intelligence (Reuters, 2018).

This was not an isolated event. A 2024 study by the University of Washington revealed significant racial and gender bias in how three state-of-the-art large language models (LLMs) ranked job applicants’ names (University of Washington, 2024). These incidents highlight a critical vulnerability at the heart of the AI revolution: the lack of a standardized safety net. Unlike the aviation or banking industries, where rigorous safety protocols are mandated, the world of AI remains a Wild West, with companies often operating without the safeguards needed to prevent catastrophic failures. The solution is not necessarily more regulation or a halt to innovation, but rather the adaptation of a proven system from a seemingly unrelated field: the Three Lines of Defence (3LoD) (Schuett, 2023).

2. The Framework: 3LoD Demystified

The Three Lines of Defence (3LoD) model is a risk management framework that originated in the financial industry and became a gold standard after the 2008 financial crisis. The Basel Committee on Banking Supervision, a global standard-setter for the prudential regulation of banks, has long advocated for its use. The model’s strength lies in its simplicity and clarity, assigning distinct roles and responsibilities for risk management across an organization. It is not about creating more bureaucracy, but about building redundancy, much like the backup systems in an aeroplane. Each layer of defence is designed to catch what the previous one might have missed.

Here is how the 3LoD framework can be adapted to address the unique challenges of AI development:

The key insight here is that 3LoD creates a system of checks and balances. The first line is responsible for day-to-day risk management, the second line provides oversight and expertise, and the third line offers independent assurance that the first two lines are effective. This layered approach ensures that risks are identified, assessed, and mitigated at multiple levels, reducing the likelihood of catastrophic failure.

3. The Evidence: Data vs. Myths

Despite the clear benefits of a structured risk management framework, several myths persist about the applicability and effectiveness of 3LoD in the fast-paced world of AI. Let us debunk some of these with data.

Myth: “3LoD slows down AI development.”

This is a common misconception. The reality is quite the opposite. Firms that effectively implement the Three Lines of Defence model often deploy AI models 22% faster than those that do not. This acceleration is not magic; it is a direct result of identifying and addressing risks earlier in the development cycle. By embedding risk management into the very fabric of AI development, teams avoid costly rework, unexpected ethical dilemmas, and regulatory hurdles that can significantly delay deployment. A PwC survey found that organizations with mature risk management practices, often aligned with 3LoD principles, experience smoother and faster transitions from development to deployment (PwC, 2023).

Myth: “AI risks are too novel for old frameworks.”

Another prevalent myth holds that AI’s unique and rapidly evolving nature renders traditional risk management frameworks obsolete. However, analysis shows that a significant majority — 83% of AI failures — map to known risk categories (OECD, n.d.). While the manifestation of these risks may be new, the underlying categories (e.g., bias, privacy breaches, security vulnerabilities, performance degradation) are well understood. Frameworks like 3LoD provide a robust structure for identifying, assessing, and mitigating these risks, regardless of the specific technology involved. The core principles of risk governance remain relevant, even as the technological landscape continues to evolve.

Technical Deep Dive: Red Teaming as a Second Line of Defence

One powerful application of the Second Line of Defence in AI is “red teaming.” This involves intentionally challenging an AI system to uncover vulnerabilities, biases, and potential misuse cases before deployment. Companies like Anthropic have pioneered sophisticated red teaming techniques to identify and mitigate harmful outputs from large language models. For instance, red teaming might involve crafting adversarial prompts to elicit toxic or biased responses or attempting to bypass safety filters. This proactive approach enables developers to strengthen their models against real-world attacks and unintended behaviors.

# Example of a simplified red teaming function (conceptual)
def red_team_test(model, adversarial_prompt):
    response = model.generate(adversarial_prompt)
    if is_harmful(response):
        Return "Harmful output detected!"
    Else:
        Return "Response seems safe."

# In a real scenario, is_harmful would be a complex evaluation pipeline
# involving human review, automated checks, and specific metrics.

Third Line Tools: Independent Audit Frameworks

The Third Line of Defence provides independent assurance, often through audits. Frameworks such as the NIST AI Risk Management Framework (AI RMF) and the EU’s High-Level Expert Group on AI (AI HLEG) Assessment List for Trustworthy AI (ALTAI) provide structured approaches for evaluating AI systems. These frameworks guide auditors in assessing everything from data governance and model transparency to accountability mechanisms and societal impact. They ensure that the first and second lines are effectively managing risks and that the organization adheres to ethical and regulatory standards (NIST, n.d.; European Commission, n.d.).

This structured approach ensures that audits are comprehensive, objective, and actionable, providing critical insights to the governing body and fostering continuous improvement in AI risk management.

4. The Implementation Blueprint

Implementing the Three Lines of Defence model within an AI-focused organization requires a clear blueprint with actionable steps for each line. This is not a one-size-fits-all solution, but rather a framework that can be tailored to an organization’s specific needs and maturity level.

First Line: Embedding Safeguards into ML Pipelines

The first line of defence comprises engineers, data scientists, and product managers who are directly involved in the design, development, and deployment of AI systems. Their primary responsibility is to incorporate safety and ethical considerations into AI models from the outset. This involves taking proactive measures and integrating risk management into the daily workflow.

Actionable Steps:

• Embed risk scoring into ML pipelines: Utilize tools like TensorFlow Model Card Toolkit to create comprehensive documentation for each model. Model Cards can include details on model performance, fairness metrics, intended use, and known limitations. This forces developers to consider potential risks and biases early in the development cycle (TensorFlow, n.d.).
• Implement Responsible AI (RAI) principles: Integrate RAI principles (e.g., fairness, transparency, accountability, privacy, security) into the software development lifecycle (SDLC). This means conducting regular bias audits, ensuring data privacy by design, and developing mechanisms for model interpretability.
• Establish clear ownership of risks: Assign specific individuals or teams the responsibility for identifying and mitigating risks associated with particular AI models or features. This prevents diffusion of responsibility and ensures accountability.
• Conduct continuous testing and validation: Beyond traditional software testing, implement specialized AI testing, including adversarial testing, robustness checks, and out-of-distribution detection, to identify vulnerabilities and ensure reliable performance in diverse scenarios.

Second Line: Creating a ‘Risk SWAT Team’

The second line of defence provides oversight, guidance, and specialized expertise to the first line. This typically involves dedicated AI risk teams, compliance officers, and legal experts who ensure that the organization’s risk management policies are effective and adhered to. They act as a critical bridge between strategic objectives and operational realities.

Actionable Steps:

• Form a dedicated AI Risk & Ethics Committee: This committee, composed of cross-functional experts (e.g., AI researchers, ethicists, legal counsel, risk managers), should define risk policies, establish acceptable risk thresholds, and provide guidance on complex ethical dilemmas. This is akin to Meta’s AI Red Team structure, which actively seeks to identify and mitigate potential harms (Meta, n.d.).
• Develop comprehensive AI risk taxonomies: Create a standardized classification system for AI-related risks, enabling consistent identification, assessment, and reporting across the organization. This helps to ensure that all relevant risk categories are considered.
• Provide training and awareness programs: Educate first-line teams on best practices for AI risk management, ethical guidelines, and relevant regulatory requirements. This empowers developers to make informed decisions and proactively address risks.
• Monitor and report on key risk indicators (KRIs): Establish metrics to track the effectiveness of risk controls and identify emerging risks. Regular reporting to senior management and the board ensures transparency and timely intervention.

Third Line: Mandating Independent Audits

The third line of defence provides independent, objective assurance of the effectiveness of the first and second lines. Typically, this role is filled by internal audit; however, with the advent of AI, it often involves external specialized auditors. Their independence is vital to providing unbiased assessments and identifying systemic weaknesses.

Actionable Steps:

• Mandate external audits for frontier models: For high-risk or frontier AI models, engage independent third-party auditors to conduct comprehensive assessments. This aligns with emerging regulations, such as the EU AI Act, which emphasizes the need for conformity assessments and external oversight of high-risk AI systems (European Commission, n.d.).
• Establish an AI-focused internal audit function: Within the existing internal audit department, create a specialized team or assign individuals with expertise in AI technologies and risk management. This team should regularly review the effectiveness of the first and second lines, including their risk identification, assessment, and mitigation processes.
• Conduct regular governance reviews: Assess the overall AI governance framework, ensuring that roles, responsibilities, and reporting lines are clearly defined and effectively implemented. This includes reviewing the 3LoD model's efficacy.
• Leverage AI for auditing: Explore the use of AI-powered tools to assist in auditing processes, such as anomaly detection in data pipelines or automated code reviews for security vulnerabilities. This can enhance the efficiency and effectiveness of third-line activities.

Cautionary Data

Failing to establish a strong second line of defence can have severe consequences. Data from the Stanford AI Index indicate that companies that skip Line 2 experience three times as many incidents as those with robust second-line functions (Stanford AI Index, n.d.). This underscores the critical role of dedicated AI risk teams in proactively identifying and mitigating potential issues before they escalate into full-blown incidents. Investing in a strong second line is not merely a cost, but a strategic imperative for long-term AI safety and success.

5. The Future: Beyond Compliance

The implementation of the Three Lines of Defence model in AI is not merely about achieving compliance; it is about building a resilient and responsible AI ecosystem capable of navigating the complexities of future technological advancements. As AI capabilities continue to expand, particularly with the advent of Artificial General Intelligence (AGI), the need for robust governance frameworks becomes even more critical.

Research Frontiers

Can 3LoD handle AGI? New adaptations for ‘frontier risks’:

The emergence of AGI presents unprecedented challenges and risks. Research organizations such as ARC Evals are actively exploring how existing risk management frameworks, including 3LoD, can be adapted to address these frontier risks. This involves considering scenarios in which AI systems might exhibit emergent behaviors or pose existential threats, requiring a re-evaluation of traditional risk assessment and mitigation strategies (ARC Evals, n.d.).

Automating Lines 1–3 with AI:
A fascinating frontier lies in leveraging AI itself to enhance the effectiveness of the 3LoD model. Imagine LLMs auditing other LLMs for bias or security vulnerabilities, or AI systems automatically identifying and flagging high-risk code segments in real time. This self-improving loop could significantly enhance human risk management capabilities, making the 3LoD framework even more robust and efficient.

Closing Call to Action

The choice before AI leaders is not between innovation and safety; it is between embracing a proven system and risking chaos. The Three Lines of Defence model, battle-tested in industries with equally high stakes, offers a free, adaptable, and ready-to-implement framework for managing AI risks. Will AI leaders proactively adopt this model, or will they wait until disaster forces their hand? The future of AI — and indeed of society — may well depend on this decision.

References

ARC Evals. (n.d.). Responsible Scaling Policies (RSPs). Retrieved from https://evals.alignment.org/

CIO Dive. (2025, March 14). AI project failure rates are on the rise, according to a report. Retrieved from https://www.ciodive.com/news/AI-project-fail-data-SPGlobal/742590/

DataRobot. (2024, November 15). What misbehaving AI can cost you. Retrieved from https://www.datarobot.com/blog/misbehaving-ai-cost/

European Commission. (n.d.). EU AI Act. Retrieved from https://artificialintelligenceact.eu/

Forbes. (2024, November 15). Why 85% of Your AI Models May Fail. Retrieved from https://www.forbes.com/sites/forbestechcouncil/2024/11/15/why-85-of-your-ai-models-may-fail/?sh=5f7b3e1e7e1a

Meta. (n.d.). Red Teaming Meta AI Services. Retrieved from https://ai.meta.com/blog/red-teaming-meta-ai-services/

NIST. (n.d.). AI Risk Management Framework. Retrieved from https://www.nist.gov/itl/ai-risk-management-framework

OECD. (n.d.). OECD AI Incidents Monitor. Retrieved from https://oecd.ai/en/incidents

PwC. (2023). PwC’s Global Risk Survey 2023. Retrieved from https://www.pwc.com/gx/en/issues/risk-regulation/global-risk-survey.html

Reuters. (2018, October 9). Amazon scraps secret AI recruiting tool that showed bias against women. Retrieved from https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK0AG

Schuett, J. (2023). Three lines of defense against risks from AI. AI & Society. https://doi.org/10.1007/s00146-023-01811-0

Stanford AI Index. (n.d.). Stanford AI Index. Retrieved from https://aiindex.stanford.edu/

TensorFlow. (n.d.). TensorFlow Model Card Toolkit. Retrieved from https://www.tensorflow.org/responsible_ai/model_card_toolkit

University of Washington. (2024, October 31). AI tools exhibit biases in ranking job applicants’ names based on race and gender. Retrieved from https://www.washington.edu/news/2024/10/31/ai-bias-resume-screening-race-gender