Acme: A research framework for reinforcement learning

Overview | Installation | Documentation | Agents | Examples | Paper

Acme is a library of reinforcement learning (RL) agents and agent building blocks. Acme strives to expose simple, efficient, and readable agents, that serve both as reference implementations of popular algorithms and as strong baselines, while still providing enough flexibility to do novel research. The design of Acme also attempts to provide multiple points of entry to the RL problem at differing levels of complexity.

Overview

At the highest level Acme exposes a number of agents which can be used simply as follows:

import acme

# Create an environment and an actor.
environment = ...
actor = ...

# Run the environment loop.
loop = acme.EnvironmentLoop(environment, actor)
loop.run()

Acme also tries to maintain this level of simplicity while either diving deeper into the agent algorithms or by using them in more complicated settings. An overview of Acme along with more detailed descriptions of its underlying components can be found by referring to the documentation.

For a quick start, take a look at the more detailed working code examples found in the examples subdirectory, which also includes a tutorial notebook to get you started. And finally, for more information on the various agent implementations available take a look at the agents subdirectory along with the README.md associated with each agent.

Installation

We have tested acme on Python 3.6 & 3.7.

1. Optional: We recommend using a Python virtual environment to manage your dependencies, so as to avoid version conflicts:

   python3 -m venv acme
   source acme/bin/activate
   pip install --upgrade pip setuptools

2. To install acme core:

   # Install Acme core dependencies.
   pip install dm-acme
   
   # Install Reverb, our replay backend.
   pip install dm-acme[reverb]

3. To install dependencies for our JAX/TensorFlow-based agents:

   pip install dm-acme[tf]
   # and/or
   pip install dm-acme[jax]

4. Finally, to install environments (gym, dm_control, bsuite):

   pip install dm-acme[envs]

Citing Acme

If you use Acme in your work, please cite the accompanying technical report:

@article{hoffman2020acme,
    title={Acme: A Research Framework for Distributed Reinforcement Learning},
    author={Matt Hoffman and
            Bobak Shahriari and
            John Aslanides and
            Gabriel Barth-Maron and
            Feryal Behbahani and
            Tamara Norman and
            Abbas Abdolmaleki and
            Albin Cassirer and
            Fan Yang and
            Kate Baumli and
            Sarah Henderson and
            Alex Novikov and
            Sergio Gómez Colmenarejo and
            Serkan Cabi and
            Caglar Gulcehre and
            Tom Le Paine and
            Andrew Cowie and
            Ziyu Wang and
            Bilal Piot and
            Nando de Freitas},
    year={2020},
    journal={arXiv preprint arXiv:2006.00979},
    url={https://arxiv.org/abs/2006.00979},
}