Submission for completing the Udacity Project
The agent is a combination of 4 add-ons to vanilla DQN.
- Priority Replay
- Double DQN
- Dueling DQN
- Polyak Averaging
Contains the weights of the trained RL bot to solve the problem. Graphs indicating the progress of the agent and when it solved the problem.
The DQN agent solved the enviroment in 625 steps (Average Reward > 13).
Vector Banana
- State space = 37
- Action space = 4
Visual Banana
- State space = Array of raw pixels (1, 84, 84, 3)
- Action space = 4
Goal is to navigate a 3d world and collect yellow bananas whlie avoiding blue bananas. Each yellow banana is worth +1. Each blue banana -1
In each environment the Agent has 4 actions:
- Move forward
- Move backward
- Rotate left
- Rotate right
The environment is considered to be solved when the agent reaches an average reward of +13
DQN, Priority_DQN
Vanilla ReplayBuffer, PriorityReplayBuffer
QNetwork, Dueling_QNetwork
train.py checkpoint.pth
Clone the repository.
git clone git@github.com:MorGriffiths/Udacity_Navigation.git
cd Udacity_Navigation
Create a virtual environment and activate it.
python -m venv banana
source banana/bin/activate
Install Unity ml-agents.
git clone https://github.com/Unity-Technologies/ml-agents.git
git -C ml-agents checkout 0.4.0b
pip install ml-agents/python/.
Install the project requirements.
pip install -r requirements.txt
Place the environment into the Environments folder. If necessary, inside main.py, change the path to the unity environment appropriately
Each project solution is contained within the notebooks "Navigation.ipynb" and "Navigation_Pixels.ipynb"
Make sure the environment path is correctly set in the desired notebook. Then run the cells as wanted.
The Vector Banana report.md is in the Vector_banana folder. Along with the performance graph and the weights.
Additionally, i tried training visual banana from scratch but likely due to memory constraints it essentially broke in the notebook format. I expect i will be able to train effectively to outside of that. And in addition run some refresh to clear the cache every N epsidoes.
link A medium article describing the different add-ons i implemented to DQN