Mission planning and execution in the present and future multi-domain operation environment (MDO) employing manned and unmanned force elements demand that the human decision makers are very well supported to be able to handle the complexity and dynamics of the battlespace and make decisions faster and better than the adversary.

In mission planning different types of operational capabilities need to be carefully coordinated in time and space to achieve mission goals and counter expected threats. Labour intensive manual planning is infeasible within the constraints of available time and resources. The general objective is to develop advanced automated support tools for the generation and evaluation of courses of action (COAs) in an MDO context. The toolset is expected to support wargaming of the candidate COAs to ensure that commanders and staff can assess the plan and options in detail before final decision making.

Specific objective

This call aims to explore technologies, concepts, products, processes and services towards a common simulation framework for wargames/combat simulations with the potential to facilitate reinforcement learning for mission planning and execution support.

Re-planning and decision-making during mission execution are likely to be challenged in the interconnected, manned-unmanned, automated and high-speed battlespace. In the future, the clear distinction between mission planning and execution is expected to be challenged by exploiting battlespace information and predictive capabilities. Proper support is needed to speed up the OODA-loop to outpace the adversary in the planning phase as well as in the execution phase.

The development and use of a computer-based decision support system that leverages AI, machine learning, wargames/combat simulations and digital twins of the battlespace has the potential to change the military planning and decision-making concept of operations (CONOPS).

Reinforcement Learning (RL) in Artificial Intelligence (AI) has shown a huge potential for solving planning problems in civilian applications. However, despite its headline success in video games, strategy games and other planning domains over the last few years, RL is not making similar progresses in the realm of wargames/combat simulations for military operations planning. Videogames leave a lot of margin when it comes to critical (life or death) simulation. Nevertheless, if access to classified data from the field is not possible, videogames data may be used for a proof of concept.

Simulation frameworks tailored to particular domains have played a major role in facilitating reinforcement learning in those domains, as witnessed by the impact of e.g., OpenAI Gym and the Arcade Learning Environment (ALE).

A common simulation framework for wargames/combat simulations has the potential of similarly facilitating reinforcement learning–support in mission planning and execution.

As it is related to EUDIS, this topic aims to support, in addition to the research activities, the creation of an innovation test hub in the field of simulation and training. To achieve this objective, financial support to third parties (cascade funding) (FSTP) is included as part of the grant. This should increase the opportunities for various smaller actors, including those not previously active in the defence sector, to adapt innovative simulation technologies for defence applications and to identify potential business opportunities in the defence sector.

Scope:

Proposals must address studies and design of a reinforcement learning environment/testbed or framework for training of AI agents to develop courses of actions in mission planning, including a flexible and open combat simulation framework fit for RL. It must address the need for rapid and user-friendly creation of scenarios, considering commander’s objectives and intent, rules of engagement and other mission constraints (e.g., speed, resources, attrition). It must also include studies and design of a combat simulation system (not necessarily the same used for AI agent training) including trained AI agents to support mission planning. For the support to mission execution the scope includes studies and design of a digital twin of the ongoing mission for prediction and decision-making support. The proposal must establish a proof-of-concept demonstrator for verification, validation and demonstration.

The learning environment, including the combat simulation framework must be flexible and adaptive for different scenarios and domains. It must take advantage of open standards and open-source frameworks both within AI, simulation technologies (including C2-Simulation interoperability) and mission sensor and mission data to the digital twin.

Types of activities

The following table lists the types of activities which are eligible for this topic, and whether they are mandatory or optional (see Article 10(3) EDF Regulation):

Accordingly, the proposals must cover at least the following tasks as part of mandatory activities:

• Studies:
  • Define scenarios and use cases;
  • Identify general operational needs and requirements, and more specifically, needs and requirements for simulation support in COA development, evaluation and wargaming. Needs and requirements for simulation support during mission execution, such as calibration of simulation models and the combination of the extensive flow of battlespace information in the future and the simulation results;
  • Describe how to use knowledge as constraints for AI models and how to (quickly) update/retrain models for specific missions.
  • Include research and identification of an appropriate Simulation Framework for wargames/combat simulations for mission planning and execution.
  • Explore the availability of data; real and/or synthetic data; validation of data.
  • Contribute to the definition of a concept of operations (CONOPS) for the mission planning and execution support framework, with special emphasis on the human-machine collaboration between the AI-enabled planning and decision-making support functions and the operators. Explore how the CONOPS could change the military decision-making process (MDMP).
  • Further explore existing open standards (e.g., NATO, NATO Modelling and Simulation Group (NMSG), SISO) and the need for new standards for simulation support in defence mission planning and execution applications.
  • Explore means of verification, validation and acceptance (i.e., trust building) of the AI models for mission planning and execution support.
  • Explore how the planning and decision-making support can explain proposed COAs and changes of plans during mission execution.
  • Prepare activities for FSTP in the field of simulation and training and in accordance with guidance described previously in the call text under “Conditions related to FSTP”.
• Design:
  • Propose an architecture design, comprising of both a Reference Architecture and proposed Solution Architecture.
  • Fulfil the requirements for simulation situations at least at the level of EU ambition (concerning number of battlegroups, concurrent operations and missions, potential opponents, etc.).
  • Design AI agents with analytics and predictive capabilities, by studying the three framework components (a testbed of simulation environments from a particular domain, a base line of general-purpose agents for that domain, and, finally, a generic interface between agents and simulation environments) to the level of a proof-of-concept, with particular emphasis on the first component, simulation environments.
  • Propose a design for a baseline of general-purpose agents. AI/RL-enabled modelling of battlespace agent behaviour, by designing and employing simulation approaches to comply with the requirements for AI-supported mission planning and execution, including Modelling and Simulation as a Service (MSaaS), simulation control, speed and parallelism.
  • Provide a clear design for verification, validation and acceptance of the AI models.
  • Organise at least one hackathon through an innovation test hub in the field of simulation and training (cascade funding).
  • Establish a proof-of-concept demonstrator, including a use-case identified in the studies, addressing its operational needs and requirements.
  • Design and execute activities for FSTP in the field of simulation and training and in accordance with guidance described previously in the call text under “Conditions related to FSTP”.

In addition, the proposals should address the following tasks:

• Studies:
  • Explore current and foreseen technologies, for future needs;
  • Study how the AI agents capabilities can be enhanced by a hybrid AI approach combining the symbolic and non-symbolic AI methods and possibly data farming.

Beneficiaries should provide Financial Support to Third Parties (FSTP) in accordance with the following conditions:

• Up to EUR 2 400 000 of the total call topic budget may be allocated as FSTP. The FSTP in the proposals should target but not exceed 16% of the requested EU contribution.
• The FSTP can NOT be provided through services offered by the consortium directly.
• The support to third parties can only be provided in the form of lump sum grants.
• The maximum grant amount per third party is EUR 60 000 (see section 10 Budget categories and cost eligibility rules).
• Third parties must be established in the EU or in EDF Associated Countries.
• Third parties must not be subject to control by non-associated third countries or non-associated third-country entities.
• FSTP must target in priority SMEs, including start-ups. Applicants for FSTP must have self-assessed their SME status. The consortium should perform checks on the basis of random sampling in accordance with the criteria as defined in Article 2 of the Annex to Commission Recommendation 2003/361/EC. Participation of entities other than SMEs can only be accepted where no SMEs are available to demonstrate the capacity or expertise needed for the project during its lifetime.
• Should include a range of entities from different Member States and EDF Associated Countries and different sectors, including those not active in the defence sector.
• Certification at company level or approval as production organisation is not mandatory, but specific business coaching should be provided to non-certified companies. FSTP calls should aim to ensure a balance between experienced SMEs and newcomers.
• Financial support to third parties should be issued in up to two distinct calls with a target from minimum 10 and up to 20 beneficiaries per call, with a view to:
  • give the third parties the opportunity to demonstrate their knowledge, technologies, capabilities and products;
  • foster the possibilities for future involvement of these third parties in the European defence community.
• The following activities, but not limited to this list, may be considered for cascade funding:
  • Boot camps; customised trainings; coaching; technical and business mentoring;
  • Investor pitching events; matchmaking;
  • Hackathons; peer-to-peer evaluation by entrepreneurs;
  • Dedicated business mentors with public and private capital expertise;
  • Organising online training courses, webinars, virtual matchmaking platforms and marketplaces;
  • Technology showcase; internationalisation;
  • Customised support for specific challenges; proof of concept; validation; first client search; innovation management support.
• The beneficiaries may be involved in any type of task within the proposal. Possible tasks at the level of the call for third parties may include, but not limited to:
  • Feasibility studies on alternative solutions;
  • Preparation of sample technologies to be tested;
  • Analysis support;
  • Support the testing or the sample preparation;
  • The use of metaverse for defence applications;
  • Synthetic population in the area of operation/missions; Scenarios for area evacuation;
  • UxVs swarms and/or simulation of remoted pilot/ammunition;
  • Future scenarios and tactics;
  • Algorithms;
  • The weather effect on the area of operations (flooding, fires etc).

Functional requirements

The proposed product and technologies should meet the following functional requirements:

• Frameworks for reinforcement learning, containing three main components:
  • A testbed of (fast executing) simulation environments from a particular domain (here: battlespace simulations for wargames);
  • A base line of general-purpose agents (in the form of reinforcement learning algorithms) for that domain;
  • A generic interface between agents and simulation environments;
• Simulate support for synthetic and realistic data generation for the development of AI models/agent behaviours.
• Collect realistic data and explore potential data sources from the field, pending availability and classification.
• Role for generative AI: Generate simulation environments, models and agent behaviour(s).
• Use MSaaS.
• Support for multi-domain simulation (and as a minimum land, air, maritime).
• Address electronic warfare as a domain.
• Include the weather element.
• The AI agents interface must be agnostic with respect to combat simulations/computer generated forces.
• Use open, commonly applicable standards (as recommended by NMSG, that could include IEEE, SISO, etc.).
• Include an easy-to-use human-machine interface.
• Meet the representation of mission and operations for example as the number of EU battle groups in accordance with the EU level of ambition, at the time of implementation.
• Include in the scenarios the role of UxVs.
• Have the capability for counter play.
• Be tailored for simulation for military operations.
• Consider a decentralised, service-based, architecture for military planning and decision-making support.
• Consider the need for human in the loop for the relevant cases of AI.

Expected Impact:

The outcomes should contribute to:

• Reduce dependencies on non-European suppliers by boosting the EDTIB and promoting the development of a European solution.
• Faster and better planning and decision making (with less personnel) during mission planning and execution, resulting in higher mission success.
• Leverage Reinforcement Learning towards largely automating the modelling and implementation of expert-level (or beyond) competent battlespace agents, thereby greatly reducing the time and cost of course of action (COA) development and wargaming.
• Deliver a proof-of-concept demonstrator at least of TRL 5.
• Increase the opportunities for various smaller actors, including those not previously active in the defence sector, to adapt and apply innovative simulation technologies for defence applications.
• Increase business opportunities in the defence sector for EU and Associated Countries companies and promote technological edge in the field.
• Increasing the interoperability between EU armed forces and with NATO Allies.
• Increase opportunities and future involvement for third parties participating in FSTP in the field of simulation and training within tasks described previously in the call text under “Conditions related to FSTP”.