Search for a command to run...
Few interesting things about me:
Really very informative and useful article. Thanks!!! Looking forward for more such content 🤩
If you’re new to machine learning, reinforcement learning, especially, then it’s completely okay to have a hard time understanding these fancy jargons in front of you. Here’s my attempt to simplify these while we work on a little robo-car :) About AW...
Dynamo DB is AWS's NoSQL offering in the vast set of managed databases as a service that they provide. Like most other services, it's fully serverless, flexible and easy to scale. The Data Model As we're working on NoSQL here, there's no real restric...
I was recently a part of a hackathon and was found searching for good Jupyter environments on the cloud to be able to work on my ML Project. Soon enough, I found myself here. However, it took me a while to figure out how powerful it truly can be. F...
I spend 5 days out of a week coding for work, so I'm definitely not motivated to code a lot on weekends. This means that I do not have any software development tools installed on my personal machine. Not even IDEs. However, being the curious little g...
Almost a week to this and not a lot of drama on Indian Twitter, so I thought I could write about it. TLDR: Malicious code with a backdoor was added to one of the core packages of Linux affecting almost all Debian and Redhat distros; was later discove...
Cloud services can be funny, they obviously provide you with high-quality services for computing, storage, etc. but can also crank up your bills even before you realize it.
AWS, like most cloud services, asks you for your credit card at sign-up. You're charged based on the time and usage of a service.
Any piece of code/event that can potentially take a lot of computing power or storage can trigger a massive bill going ahead. To know what we're up against, here are some of the bill-scare stories you might wanna read -
I don't know about you but those numbers are scary, especially how quickly they go up. Bills can go up exponentially in a matter of hours.
Handling large bills once they are in is a different ball game, but first priority should always be to prevent these bills in the first place. If you are someone like me who is just trying to experience the Cloud for the first time, learning how different services work, you should try not to have any bills at all.
AWS has a pretty generous Free Tier that lasts for 12 months after creating your account.
A lot of services like AWS Lambda, Cognito, Cloudwatch, DynamoDB and SNS are free until you reach a threshold limit.
Whereas, services like EC2, S3, RDS, etc. provide free services until the threshold limits for only 12 months, i.e. until you're a part of the Free Tier.
On the other hand, there are also services like AWS SageMaker, Redshift and Lightsail that are available on a trial period for 2-3 months with their respective limitations.
One solid way to avoid massive bills in AWS or any other cloud service is to set up Budgets and Budget Actions that can alert you when things start to get out of hand.
Search for Billing in your AWS Console and open up the Billing Dashboard.
The Billing Dashboard provides a complete overview of all monetary information about your AWS account.
Under Cost Management, click on Budgets to open up the Budgets page.
Now click on Create to start working on a new Budget.
We'll go ahead with a Cost Budget and give a name to our new Budget. We'll choose Monthly as the period for our Budget. Since I am a student and I don't really have any cloud services running, I'll just choose the threshold to be 1$. We can proceed to the next step now.
This is one of the key steps in creating a Budget, which is to set up an alert that notifies you via email when your bill is close to the budget threshold. Click on Add to create a new Alert.
Here, you can configure the percentage of the bill when there is an alert popping up and also a list of emails to notify. You can also use SNS and ChatBot as notifiers.
So in my case, when the bill reaches 0.75$, I get an email.
In one of the stories, you saw how a startup company got a 72000$ bill in a matter of just a few hours. There are possibilities you end up being too late to see the alert. Here's where Forecasted Alerts come into play.
Every time there is a tiny increment to your bill from 0.00$, a forecasted cost is generated based on your current usage to predict what your bill would be at the end of the month. You can use this metric to create an alert, based on which you can identify if any service is costing you more than usual. Just create a new alert and choose Forecasted instead of Actual in the Trigger parameter.
You can have AWS automatically perform certain actions when a budget crosses a certain alert threshold by attaching an IAM Service Role to the alert under Actions.
With this role, you can target a certain instance of a service or multiple instances and/or services based on your requirements. This would also require the particular role to have all the necessary permissions to perform the said actions.
And that's all you need to do, to set up your budget.
Everything on the cloud is event-based and every time there is a new event, it potentially adds to your bill. This is why it's critical to avoid infinite loops in the cloud, where one service is triggered after the other and there is no end.
Focus on how every time there is a new user, a new record, a new entry, how many extra operations it takes to run the same function. This gives you a good idea of the complexity of your code. Higher the complexity, the larger the bill in the long run.
Use environment variables whenever you're using secret keys or API keys in your code. Use strong passwords and MFA for logging into your AWS account. Make use of IAM Users, Policies and Permissions to access different services. Any vulnerability can be exploited and any misuse can cause serious problems.
Auto Scaling is a very useful feature of the Cloud, but it can be dangerous when not controlled. It is recommended that you only scale as much as required. There are maximum thresholds available for all services and they can significantly affect your bills.
Just like saving electricity, it's recommended to turn off services that you're not using anymore. This is key especially if you are exploring the Cloud like me. Once done using a certain service, it's best practice to just shut it down to avoid usage costs.
Here's one last bonus feature I'd like to talk about in AWS that can help you reduce your costs.
AWS Cost Anomaly Detection, as a part of the AWS Cost Explorer API, uses advanced Machine Learning Techniques to identify anomalous costs that can help avoid your bills skyrocketing suddenly. You can set it up and receive alerts on SNS, Emails even periodically. This not only helps you avoid surprise bills but also analyzes the root cause of the hike.
That's it from me in this thread, I hope it comes in handy in your Cloud journey. Thanks for reading :)
