gym

🐛 Bug

The documentation of DQN agent (https://stable-baselines3.readthedocs.io/en/master/modules/dqn.html) specifies that log_interval parameter is "The number of timesteps before logging". However, when set to 1 (or any other value) the logging is not made at that pace but is instead made every log_interval episode (and not timesteps). In the example below this is made every 200 timesteps.

The following applies to DDPG and TD3, and possibly other models. The following libraries were installed in a virtual environment:

numpy==1.16.4
stable-baselines==2.10.0
gym==0.14.0
tensorflow==1.14.0

Episode rewards do not seem to be updated in model.learn() before callback.on_step(). Depending on which callback.locals variable is used, this means that:

• episode rewards may n

Add images to ingredients

Per this comment in #12

There seem to be some vulnerabilities in our code that might fail easily. I suggest adding more unit tests for the following:

• Custom agents (there's only VPG and PPO on CartPole-v0 as of now. We should preferably add more to cover discrete-offpolicy, continuous-offpolicy and continuous-onpolicy)
• Evaluation for the Bandits and Classical agents
• Testing of convergence of agents as proposed i