Why AI Projects Fail and How They Can Succeed – RAND

Posted by timmreardon on 08/15/2024
Posted in: Uncategorized.

The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed

Avoiding the Anti-Patterns of AI

James Ryseff, Brandon De Bruhl, Sydne J. Newberry

RESEARCHPublished Aug 13, 2024

DOWNLOAD PDF:

Click to access RAND_RRA2680-1.pdf

To investigate why artificial intelligence and machine learning (AI/ML) projects fail, the authors interviewed 65 data scientists and engineers with at least five years of experience in building AI/ML models in industry or academia. The authors identified five leading root causes for the failure of AI projects and synthesized the experts’ experiences to develop recommendations to make AI projects more likely to succeed in industry settings and in academia.

By some estimates, more than 80 percent of AI projects fail — twice the rate of failure for information technology projects that do not involve AI. Thus, understanding how to translate AI’s enormous potential into concrete results remains an urgent challenge. The findings and recommendations of this report should be of interest to the U.S. Department of Defense, which has been actively looking for ways to use AI, along with other leaders in government and the private sector who are considering using AI/ML. The lessons from earlier efforts to build and apply AI/ML will help others avoid the same pitfalls.

Key Findings

Five leading root causes of the failure of AI projects were identified

  • First, industry stakeholders often misunderstand — or miscommunicate — what problem needs to be solved using AI.
  • Second, many AI projects fail because the organization lacks the necessary data to adequately train an effective AI model.
  • Third, in some cases, AI projects fail because the organization focuses more on using the latest and greatest technology than on solving real problems for their intended users.
  • Fourth, organizations might not have adequate infrastructure to manage their data and deploy completed AI models, which increases the likelihood of project failure.
  • Finally, in some cases, AI projects fail because the technology is applied to problems that are too difficult for AI to solve.

Recommendations

  • Industry leaders should ensure that technical staff understand the project purpose and domain context: Misunderstandings and miscommunications about the intent and purpose of the project are the most common reasons for AI project failure.
  • Industry leaders should choose enduring problems: AI projects require time and patience to complete. Before they begin any AI project, leaders should be prepared to commit each product team to solving a specific problem for at least a year.
  • Industry leaders should focus on the problem, not the technology: Successful projects are laser-focused on the problem to be solved, not the technology used to solve it.
  • Industry leaders should invest in infrastructure: Up-front investments in infrastructure to support data governance and model deployment can reduce the time required to complete AI projects and can increase the volume of high-quality data available to train effective AI models.
  • Industry leaders should understand AI’s limitations: When considering a potential AI project, leaders need to include technical experts to assess the project’s feasibility.
  • Academia leaders should overcome data-collection barriers through partnerships with government: Partnerships between academia and government agencies could give researchers access to data of the provenance needed for academic research.
  • Academia leaders should expand doctoral programs in data science for practitioners: Computer science and data science program leaders should learn from disciplines, such as international relations, in which practitioner doctoral programs often exist side by side at universities to provide pathways for researchers to apply their findings to urgent problems.

Article link: https://www.rand.org/pubs/research_reports/RRA2680-1.html