bandits

Bandits

This project is intented to be the implementation of multi-armed (MAB) bandit framework.
I want to try make an app that would be easy to use if you need MAB.

Glossary

Planned features and scenarios

There will be two APIs(?):

1. Strict API with entity hierarchy

   • One will need to create explicit problem, attach actions and start experiment.
   • IDs of problem and actions should be fixed until the end of experiment.
   • Actions can be added on the fly, but their performance will be up to MAB strategy (UCB, E-greedy, etc).
   • If context is specified then it will affect the stats.

   GET POST /problems GET POST /actions POST /upload_data GET POST /choose GET POST /reward

2. Loose API

   • All you need is to give a list of action IDs to choose from.
   • Actions will be automatically created and accounted.
   • No context support.

   GET POST /choose GET POST /reward POST /upload_data

I need to think about how it would be integrated into the real set ups.
Based on that API will be formalized.

Typical scenarios

Get active problems

given: user wants to fetch active problems
when: issues a request to api
then: problem instances are returned from the storage

GET /problems

Get all problems even deleted ones

given: user wants to fetch all problems
when: issues a request to api with query parameter deleted=true
then: all problem instances are returned from the storage

GET /problems?all=true

Create problem

given: user wants to create a problem
when: issues a request to api with problem_id and problem_description then: problem instance is created and added to storage without any actions added

POST /problems { "name": "problem_name", "" }

Delete problem

given: user wants to delete a problem
when: issues a request to api with problem_id
then: problem instance marked as deleted and won't be shown