README
¶
Evo-learning
This is a local, non-distributed, Go implementation of the Evolution Strategies as a Scalable Alternative to Reinforcement Learning. The original work from the paper can be found at openai/evolution-strategies-starter.
It uses the openai/gym-http-api, binding-go, and unixpickle/anynet and unixpickle/anyvec for efficient high-level vector computation.
Instructions
The project is aimed to solve CartPole-v0, which requires 195 epochs/reward over 100 episodes.
First, clone or download the repo:
$ go get github.com/KnowmeGPT/evolution
In a separate terminal, open gym from the github.com/openai/gym-http-api directory in your file system.
$ python gym_http_server.py
Now, run the trainer and evaluator with the specifications of your choice.
$ # 200 episodes of training by 2 agents and 100 episodes of evaluation with a single agent
$ # Saving results to a directory "~/agents2eps200"
$ evolution --outmonitor ~/agents2eps200 --finalepisodes 100 --episodes 200 --agents 2
Results
After a trial and error phase of 42 episodes, the agents evolve enough to dominate the game independently. We apply a cutoff average reward of 195 or more for both agents, implying that the parameters should typically be capable of solving the game independently. The average reward achieved is 198.5 over 100 episodes!
Future Plans
- Parallelize where needed
- Add 32/64 bit support
- Support multiple environments
- Allow network serialization/deserialization
- Optimize code further
- Include plotting, statistics, performance profiling, and uploading capabilities
Disclaimer
This is a project initially created for a Complex Systems and Networks class. It is not intended to compare with the original work. Rather, it is an execution with results and interpretation.
Documentation
¶
There is no documentation for this package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package agent implements a neural network worker.
|
Package agent implements a neural network worker. |
|
Package env implements the gym-http-api utilites.
|
Package env implements the gym-http-api utilites. |
|
Package noise implements noise sharing and Gaussian binning
|
Package noise implements noise sharing and Gaussian binning |
|
Package opt implements Adam optimization
|
Package opt implements Adam optimization |
|
Package policy implements a generic policy for evaluation of an environment.
|
Package policy implements a generic policy for evaluation of an environment. |
|
Package util implements utility functions and data structures
|
Package util implements utility functions and data structures |