Search for a command to run...
Few interesting things about me:
No comments yet. Be the first to comment.
Dynamo DB is AWS's NoSQL offering in the vast set of managed databases as a service that they provide. Like most other services, it's fully serverless, flexible and easy to scale. The Data Model As we're working on NoSQL here, there's no real restric...
I was recently a part of a hackathon and was found searching for good Jupyter environments on the cloud to be able to work on my ML Project. Soon enough, I found myself here. However, it took me a while to figure out how powerful it truly can be. F...
I spend 5 days out of a week coding for work, so I'm definitely not motivated to code a lot on weekends. This means that I do not have any software development tools installed on my personal machine. Not even IDEs. However, being the curious little g...
Almost a week to this and not a lot of drama on Indian Twitter, so I thought I could write about it. TLDR: Malicious code with a backdoor was added to one of the core packages of Linux affecting almost all Debian and Redhat distros; was later discove...
If you’re new to machine learning, reinforcement learning, especially, then it’s completely okay to have a hard time understanding these fancy jargons in front of you. Here’s my attempt to simplify these while we work on a little robo-car :)
A contest that is discontinued after this year, unfortunately, DeepRacer is arguably the most entertaining service available on the AWS Console today. This is the arena where you can train a virtual car to run through a racetrack, and the challenge is to complete 3 laps as quickly as possible without going off-road.
—
Before I get to the jargons, let’s first try to understand
Reinforcement learning (RL) is an interdisciplinary area of machine learning and optimal control concerned with how an intelligent agent should take actions in a dynamic environment in order to maximize a reward signal ~ Wikipedia
While I barely made it through that sentence, let’s imagine reinforcement learning as training a dog to fetch the ball for you. Every time the doggo fetches the ball, you give the little guy some treats. You’re giving him positive reinforcement, or as we call it incentive, to perform that action. This is basically what we do in RL as well. We give a reward to our intelligent agents based on their actions to get a desired result.
In case of AWS DeepRacer, the agent is a car, and the desired result is for it to complete the track without going off-track, as quickly as possible. This means that our rewards need to account for the speed of the car, the nature of the track, the turns and more.
Now that we have an understanding of what RL is, let’s try to understand the hyperparameters for our model that may have an impact on our end-results.
Models based on Neural Network train in iterations, meaning they update themselves after every round of learning. This hyperparameter gives you control over how much your model learns in every iteration. Keep in mind that all hyperparameters have their benefits and trade-offs. In this case, a larger batch size (say 128 over 64) would mean that the model would have a wider set of experiences to learn through, and generate a steady output, however, over iterations, it would take more memory and time.
In Deepracer context, the batch size can essentially decide on the exploration that your model will remember across various iterations of its training. Larger batch sizes can delay the convergence times but with every iteration, it’ll get your model closer to the optimal path.
I heard about this term for the first time in Chemistry, and very simply - it’s a measure of randomness. Think about how much you want your model to explore while training. The possibilities it can figure out to eventually find out an optimal result. In case of your robo-car, it can find different ways to cover a track, for example. taking fast, shorter turns, taking slow, longer turns, etc. and it eventually finds out the best option based on the incentive it gets.
You might want to reduce the entropy with time as the model matures, as you’d figure out it’s closer to the optimal path, you wouldn’t want it to diverge from that.
A value that ranges between 0 and 1, it signifies the importance your model gives to short term gains over the long term rewards. Discounting Factor closer to 1 means that you prefer future success over immediate high rewards on that particular action on the iteration. Whereas a factor closer to 0 means that it’ll prioritise the incentives it just received. In DeepRacer, it’s critical that the car has a long-term foresight so as to ensure it doesn’t land itself is bad positions around curves or turns that can lead to crashes.
This parameter basically controls how quickly your model reaches the point of convergence. However, with higher learning rates, you run the risk of overshooting, wherein you might get to an optimal path quicker and immediately diverge from it in the next iteration. The key, like most other parameters, is balance. You can adjust the learning rate based on observation of the patterns that your model creates while training. In DeepRacer, this can potentially define how quickly your model is able to adapt to changes, eg. sharp turns after a straight road, where your car is initially intending to run at full speed, but then has to slow down and turn 140 degrees.
Epoch - can be defined as a collective batch of experiences combined by various episodes in a training window. Your complete process can have many different batches, meaning the model has to move on from one set to another. The number of epochs signifies the amount of time your model spends on one such set. This can essentially refine how your model thinks over a certain set of experiences. Longer epochs naturally mean longer training times and compute usage.
Hope this was a good read to cover various Deep Learning jargons and simplify your experience working with neural networks going forward. Thank you for reading:)
