Executive Summary

From the marketplace to the sports arena, humans thrive on competition where they are often rewarded for their creativity and excellence. In the narrower field of technology innovation, competitions can advance the state of the art and reward breakthroughs in areas that have confounded researchers or slowed progress. Government, too, can play a role. It can provide powerful incentives for the private sector by organizing competitions that spur technological progress. If organized correctly, competitions can foster collaboration and innovation around unique national security challenges.

But choosing how to organize and host competitions involves far more than just offering monetary incentives. Competition organizers should consider the problem or issue that any given competition centers around, its reward system, and the number of participants or teams. They should also consider the best way to share the competition results, and the frequency of competitions. Over the last decade progress in artificial intelligence has occurred rapidly. AI-powered systems have already demonstrated great potential for solving some of the most difficult scientific challenges of the day. These include areas such as determining protein structure, helping drivers operate vehicles safely in urban environments, and identifying diseases. Given its transformational potential, governments should incentivize the development of AI applications for national security. U.S. federal agencies have begun to use competitions to do just this, but their use remains limited.

This policy brief examines 814 competitions conducted from 2010 to 2020 across the U.S. federal government as found in the Challenge.gov archive. During this time period, the total prize purse reached roughly $243 million. In comparison, total federal research and development (R&D) outlays for the period were over $1.3 trillion. While the prize purse does not represent all of the expenses involved in facilitating a competition—such as associated personnel costs or assembling the necessary infrastructure—the total costs still represent only a small fraction of yearly federal outlays and remain an underutilized tool to promote innovation. A review of federal competitions suggests that there are three factors that contribute to the potential for a greater chance of satisfying organizer goals or useful implementation of a successfully designed product.

These factors both increase the incentives for participants and increase the likelihood that promising entries are operationalized within the federal sector. First, prizes should reflect the time, effort, and resources involved for participants. Though larger prizes are more likely to attract high quality participants, it is not entirely necessary. Second, organizers should design competitions to ensure promising entries have a path towards rapid procurement or scaling versus the traditional acquisition and procurement process. Finally, competitions can benefit from access to or the creation of professional networks among participants, which ultimately can improve public-private partnerships.

While the private sector also uses competitions and benchmarks to incentivize progress in AI, they have not addressed the full range of national security requirements. The federal government should consider additional competitions centered on the challenges of AI safety and security, AI trust and explainability, and AI applications for cybersecurity. Private sector work is occurring in each of these fields and can provide examples for how agencies may structure their own challenges. However, federal competitions have the potential to uplift the research and focus on the specific national security aspects of each of these topics. With that, this brief makes three findings on the use of federal competitions to promote innovation in artificial intelligence:

• Agencies have yet to fully leverage AI competitions for national security requirements: Agencies could increase their use of prize competitions without significantly impacting their R&D budgets. Competitions have unique benefits that are less easily replicated through traditional R&D processes. Disregarding additional expenditures, the sponsoring agency or department typically pays only for success, shifts risk of failure to participants, and is therefore able to establish ambitious goals. This form of open innovation attracts a broader range of participants by lessening bureaucratic impediments to working with the government. There are challenges happening within the private sector that the federal government could emulate.

• Competitions provide a means to test operational effectiveness prior to potential prototyping, scaling, or procurement: Running a competition prior to procurement allows the sponsoring agency to gain insight into the available tools and technologies and to test their operational performance by welcoming a diverse range of participants with various technological approaches. Because competitions transfer most of the risk of failure to participants, the agency could pursue more ambitious mission-oriented goals without risking great financial loss. A competition could also serve as an alternative means of acquisition which can speed the process for the government and allow opportunities for more private sector entities to participate.

• The National AI R&D Strategic Plan and other federal strategies provide a roadmap for the topics of future competitions: Competitions can drive AI innovation for the government if they are designed around federal innovation strategies and priorities. Key topics include:
  • Safety, security, and trust in AI/machine learning (ML)- enabled systems: With increased emphasis on accelerating the adoption and integration of AI technologies into federal infrastructure and operations, the need for understanding AI-enabled system decision-making, trusting those decisions, and guaranteeing its continued performance is a critical area of strategic R&D focus.
  • Deepening public-private partnerships: Deepening and strengthening public-private partnerships in AI R&D remains a key priority for most federal agencies and departments, because much of the talent, expertise, and current breakthroughs in ML are occurring in the private sector and academia. Competitions are an additional avenue for collaboration, partnership, and cooperation with these critical sectors. Furthermore, sponsors of federal competitions would benefit from working directly with technology incubators within their departments like the U.S. Department of Defense’s Defense Innovation Unit or the U.S. Department of Health and Human Services’ IDEA Lab to streamline the procurement of potential solutions.