Cloud changes the planning cycle
With on-premise infrastructure setups, managing utilization would usually mean looking at the code that lives directly on the server. The more data and features you add, the faster you get to around the 70-80% usage amount – the point at which many IT managers start to think about adding more server capacity.
However, purchasing, provisioning and connecting a server in an on-premise data centre can take up to six months. It isn’t an overnight job. So, in the meantime, teams might try to make tweaks to the existing servers to reduce usage, boost performance, and give some breathing room until the new hardware is online.
When deploying code in the cloud, it’s very different. That’s because the cloud’s ability to automatically scale means you never get to the 70 or 80% capacity level. And because the meticulous level of expansion planning isn’t required in the same way as in an on-premise situation, the impetus to tweak and enhance your code to gain more headroom isn’t as great.
Where you might have worked hard to regain 20-30% of capacity on premise, that procedure never happens in the cloud. As such, your cloud services bill gets higher and higher.
The case for cost SLOs
All this requires us to rethink the way we approach the cloud, code, and costs. The common misconception is that cloud is infrastructure when in fact it’s code. And not enough developers think about how much their code costs in production. Usually, they will prioritize the parameters that exist in service level objectives (SLOs) such as CPU and memory consumption, latency, and response times. Usually, cost isn’t regarded as an SLO. I would argue that has to change.
If cost is set as a deliverable from the very beginning, it sets the tone for optimizations that will make cloud costs more affordable – while also unlocking other benefits at the same time. Every execution adds a cost, but what if those costs could be minimized? For example, let’s say an online transaction usually takes two seconds to complete. If you could introduce a line of code that could reduce that two seconds down to 500 milliseconds, you can save 75% while also improving customer satisfaction.
It’s a phenomenon called cost-conscious coding and it couldn’t come to prominence at a better time. While cloud costs increase, AI is becoming more capable, helping developers create code faster than before. According to McKinsey, AI can cut the time it takes to write new code in half. Therefore, it will be possible to become more efficient and cost-effective with code rendered in the cloud, while the people hours to create the code in the first place will be heavily reduced.
Make cost-conscious coding make sense
If you want to manage your cost as an SLO, a good place to start is by defining, measuring, calibrating, and recalibrating it. What do you want the parameters to be and how will you track progress? Cost-conscious coding is an iterative process, so you will continually discover new ways to improve performance.
That said, it’s also worth noting that your returns will ultimately stabilize. There will be a point when the law of diminishing returns kicks in, and results will also vary from business to business. Cost-conscious coding, therefore, should be aware of the effort being put in and the expected outcomes. The results should always justify the initial effort or it will be a wasted exercise, no matter how much AI helped write the code.
If you’ve noticed your cloud costs ballooning recently, cost-conscious coding is definitely worth exploring. It sits well within the broader philosophy of setting cost as an SLO and when there’s such an expectation in place, it’s much more likely that developers will start to address it. Even if your costs aren’t going up now, if you’re planning to grow, there’s a likelihood that they will at some point. Cost-conscious coding is an activity that prepares for the future and can potentially use emerging technologies to make it happen.
