The AWS DeepRaceing Championship at re:invent in Las Vegas and how I got there
What is DeepRacer?
From the AWS-Documentation:
AWS DeepRacer is an integrated learning system for users of all levels to learn and explore reinforcement learning and to experiment and build autonomous driving applications. It consists of the following components:
An AWS Machine Learning service to train and evaluate reinforcement learning models in a simulated autonomous-driving environment.
AWS DeepRacer 1/18th scale RC vehicles capable of running inference on a trained AWS DeepRacer model for autonomous driving.
AWS DeepRacer for the participants to compete on model performances on a specific track in a simulated or real environment and have the results ranked on a virtual or physical leader board.
AWS-Robomaker provides a virtual environment. It creates a 3d-world with one of a couple of virtual race tracks and a car to race through it.
AWS-Sagemaker provides an environment for training the car. The design of the neuronal network is provided. The input is the image received from a camera mounted on the car. The output is an action. One can choose which actions to use. For instance one might want two speeds, say 1 m/s and 2 m/s. Then one can choose for instance 5 steering angles that the car shall be able to steer to. In this example there would be a total of 2*5 = 10 possible actions available, each represented by an output neuron. The command associated with the neuron that receives the strongest input will be chosen to drive the car.
A 3 layer convolutional neuronal network is used, followed by a couple of fully connected layers. The algorithm to learn the weights is a proximal policy optimization algorithm.
The AWS engineers left open which scoring function one wants to use and how to choose a couple of parameters for the learning process. However, they do provide examples for that that already kind of work.
How did I get into this?
11/12: I receive an email saying “Are your AI skills in pole position? Then come to the Accenture AWS DeepRacer hackathon in Kronberg on 11/14”. I felt challenged. I look through the instructions and log into the AWS console to create my first virtual race car. The setup is super convenient. There is an AWS service that constructs all needed resources automatically.
I run the first couple of trainings with the default-scoring functions. I have a couple of ideas what one could do. But my feeling is that I am far too late to get into this topic simply because I started too late. Fortunately I read in some Q&A section:
Q: I do not have a working model. Can I participate anyway?
A: Just come around. You can still train models on the day and get in touch with other developers.
11/13: I do some client work. In the afternoon I finally decide to participate in the Accenture hackathon. I book a train and a hotel and drive to Frankfurt in the evening. At this point in time I do not even have a model that would manage to finish a single lap. I still only played around with a couple of sample reward functions.
On the train I finish my first reward function. I aim for a simple, robust model that can be trained quickly. When I arrive at the hotel I throw the code into the AWS console to look what happens. The result is: After about 15 minutes of training the car completes its first lap. One minute later it completes another lap and after this 50 more without a single crash! I am really happy with this result, but I still do not anticipate that this model would become the fastest in the worldwide Accenture competition the next day.
11/14: I get up ridiculously early to get to the Accenture office in Kronberg and attend a call with a client. Then at 9:00 the actual race starts. I am surprised to see that my model works very well in the real world as well. It is a known issue, that transferring a model from a simulated environment to the real world can be very challenging. I try out a couple of ways to improve, but apart from some minor parameter adjustments I do not manage to find any better approach. I realize that starting off with a simple robust base model was the way to go for this competition.
And in the end that saves me the first place in the world’s largest private AWS DeepRacer league run by Accenture. I am a bit proud, after all this was a global competition consisting of 30 sites across 17 countries. This also is my entry pass for the AWS DeepRacer Championship Cup at re:invent in Las Vegas.
Vegas Baby
12/01: I arrived in Vegas. My last training runs are two weeks ago. I did not have time to prepare much for what is ahead of me. I arrived at the hotel and finally find some time to train more models. After a little shock that AWS changed the user interface for the training and changed the behavior of the car in the virtual environment a bit, I manage to train a couple of models, none of which differs much from my initial approach.
12/03: There are a total of 64 competitors from all over the world and from numerous companies. I am representing Accenture. All participants are assigned to one of 4 groups of 16. To make it to the next round I have to be among the first 4 in my group. After a couple of warm up runs, the race begins. Each participant gets 4 runs, 4 minutes each and the overall fastest lap counts. I end up making it to the 3rd place in my group. So I am in the final group of 16.
12/04: The last 16 compete in a double elimination system. I drop out in the quarter final, i.e. I made it to the top 8 worldwide. I am pretty happy with making it that far. I am also very exhausted.
12/05: Grand Final. Watch it on youtube.
Details on how the model was trained can be found here:
How to train a model in 15 minutes
The scoring function and parameters used for training can be found here (leave me a star if you learned something ;) https://github.com/TwoDigits/deepracer
