About a year ago in May 2020, Unity announced the release of ML-Agents – the Unity Machine Learning Agents Toolkit – which was designed to allow game developers and AI researchers alike to use Unity as a platform to train and embed intelligent agents using the latest advancements in machine learning. In case you didn’t already know, ML-Agents toolkit is an open-source project that enables simulations and games to become background environments for the training of intelligent agents. It has an integral C# SDK to construct a scene and define the agents that operate within it. It also has a state-of-the-art machine learning library to train agents for 2D, 3D, and VR/AR environments. ML-Agents is the place to build and test your projects.

The original release of ML-Agents was well received by Unity users and beyond as it flexed powerful muscles but was relatively easy to use. It captured the imagination of game designers and developers, and is now an integral part of the Unity software. The package was published as part of the 2020.2 Editor release and after a number of updates to include improvements to the user workflow, new training algorithms and features, and a significant performance boost, ML-Agents release 2.0 is finally here.

The new package comes with three core developments that further enhance the package and increase its scope. These are: 

• The ability to train cooperative behaviors, 
• The potential to enable agents to observe various entities in their environment, and 
• The harnessing of task parameterization to support training multiple tasks.

These factors add substantial abilities to the initial framework software, and it is worth having a look at each one in turn to understand what the latest update brings.

• Training Cooperative behaviours. While it was previously possible to train ML-Agents with multiple agents in the scene, the user could not define specific agent groups with mutual goals until later updates of the toolkit.  With the latest release, ML-Agents now explicitly supports training cooperative behaviors, and individual agents can work toward a common goal, with the success of each individual tied to the success of the whole group.
• Observing varying numbers of entities. Many game engines allow a central character to react appropriately to multiple antagonists, but this factor was essentially missing from previous iterations of ML-Agents. However, the new release has an included ‘attention module’ built in which processes a varying number of observations by the Agents. This, in turn, allows for situations where a game character must learn to avoid a great number of projectiles or antagonists. 

Task parameterization. The task parameters of a situation usually take the form of positions of objects in the environment or specific landmark points that the Agent should pass through in order to complete their task. The latest version of ML-Agents uses a new type of observation called a “goal signal,” as well as a small neural network called a “HyperNetwork,” to generate some of the weights or significance of another bigger neural network, thereby giving different tasks lesser or greater importance. This, then, impacts the overall behaviour of the Agent.

Other improvements in the latest release include memory allocation and includes significant reductions in what memory is allocated to, and the use of memory can be almost a 100% decreased. Furthermore, the ML-Agents Cloud becomes more pervasive and now lets the user kick off multiple training sessions that run on the Unity cloud infrastructure in parallel, so they can complete a number of developments concurrently. This saves time and allows for cross-experimentation on projects.

As it stands, ML-Agents is a huge boost to the initial release, but there is still more that is likely to be fed out during 2021. Unity is yet to reveal another upcoming showcase environment called Dodgeball, which is a base team-versus-team game within Unity that highlights the way that all three features work together. However, it isn’t certain when this will be included, so keep checking back here at Unity Developers for the latest update. But until then, enjoy the new functionality in ML Agents 2.0 release.