Building a Self-Racing Race Car – The Journey Begins

Hello World!

So today I have decided to actually begin a write up on where I’m headed.  Its taken a long while to come up with what my next adventure after food trucks was going to be, especially with all of the things you can do with embedded technologies and machine learning.  So here it is, a self driving race car.

In this article I aim to lay out the high level plan of attack for what my team and I are building and some of the direction on where we are headed.

Why a Self Driving Race Car?

Honestly, because its freaking cool.  I wanted to have fun with this one and I’ve always liked fast cars and wanted to race a car.  Also, I was accepted into Udacity’s first self driving car curriculum, so it was somewhat fortuitous timing.  Also, I predict the market for self driving cars is going to get flooded.

High Level Plan & Goals

We have already built version 1 (Herby), which can be seen in an earlier blog article here.  That prototype showed that machine learning and MDPs can be used to create a robot that attempts to move as fast as possible around a course.  We have now procured the materials for v2.  Our plan is to compete and win the classic autonomous vehicles race at sparkfun next year using v2.x.  We will remain laser focused exclusively on solving all of the problems on this small, but very high speed platform.  The current race car we have has a top speed of 35 mph, but with some simple component swaps, we can get up to 65 mph.  These limitations will force us to develop better algorithms, processes and tools for our higher end vehicles when we decide to procure them.

Pictures of V1 and V2.


Priority #1 – Race Simulator Development

I will not divulge all of the details here as I envision productizing this system.  The details I can share are that we are developing a simulator based on Unity3D that will allow run time configuration, and training of massive scale clustered AI for reinforcement learning.  For our training purposes the game will be deployed into Azure GPU clusters.  This simulator will enable us to simultaneously train any number of algorithms.  Most everything will be built with Tensor Flow and Python running in Azure.  We will also have diagnostic dumps of every training session.  We will end up with a ton of data, which will be stored in Azure Data Lake.

Priority #2 – Race Car Hardware

We are basing the hardware on the NVidia TX-1 platform.  We simply need to wire up 2 servos (3 wires each) to the TX-1 to gain direct control of the car controls.  Our sensors currently are camera only.  We envision adding gyroscope + accelerometer, but that is about it.  We are using the out of the box camera that ships with the TX-1 dev kit.  Finally there will be a fair amount of live diagnostics on board the car, which we will stream via Azure IoT Hub.  This way we don’t need to catch the vehicle, we can simply push commands to it, and receive data if it decides to go rogue on us.

Priority #3 – Integration

Last priority is taking the best performing AI algorithms from simulator training, and tune them on the the physical vehicle.  What works in simulation may not work on the vehicle itself.  I think we need to be wary of over-fitting on the simulation data.  This will be where some of the art of deep learning comes in to play.


So thats the top priorities.  I’ll post an update each month with progress on the key items.  We have several algorithms we are already wanting to begin testing, but just need to get the infrastructure up and running so we can begin.

Leave a Reply

Your email address will not be published. Required fields are marked *