As developers we often ask for more resources from the infrastructure & operations (I&O) teams than we really need so we don't have to go back later and ask for more - too painful and time consuming. We also, often don't know how many resources our code might need so we might as well take as much as we can get. But do we ever give it back when we learn it is more than we need?
On the other hand I&O isn't often any better. The first rule we learned about capacity planning
was that it's more expensive to underestimate resource needs and be wrong than to over estimate and we always seem to consume more resources eventually.
Well, Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) clouds change this equation dramatically andyou can reap big rewards
if you change with them. For example, sure, you can ask for as many resources as you want - there's no pain associated with getting them and no pain to ask for more either. But once you have them and figure how much you really need, it heavily behooves you to give back what you aren't using. Because you are paying for what you allocated, whether you are really using it or not.
, a SaaS-based cloud cost management company knows how much this is costing its enterprise clients as it uses monitoring capabilities to map the difference between what its clients are paying for and what they are really using. It recently shared with Forrester the latest findings in its CloudynDex metrics report that aggregates cloud use and cost data from more than 100 of its clients running on Amazon Web Services' (AWS) IaaS cloud (collected randomly and anonymously with its clients' permission). The data is clear proof that we are bringing these bad habits to the cloud. These clients are spending between $12,000 to $2.5 million per year with AWS and throwing away about 40 percent of that expense. What kind of waste:
- Overallocation of resources. Cloudyn found that the degree of sustained utilization across the 400,000-plus instances being monitored was just 17 percent. The company said the common issue was allocatingLarge or Extra Large instances when a Small or Medium would suffice. This one's easy to find (especially with a cost analysis tool like Cloudyn) and easy to fix. Not surprisingly, Cloudyn also found that the larger the instance, the worse the utilisation with Extra Large instances averaging just 4 percent utilisation. That's worse than the average utilisation of physical servers in 2001 - before virtualisation. For shame!
- Static workloads. Cloudyn also found that many client instances were forgotten and left running not doing anything for days, even months at a time. Cloud vendors will certainly be happy to take your money for this. But, really. Is it really that hard to shut down an instance and restart it when needed?
- Not using Reserved Instances. The statistics also showed the average client had a persistent use of cloud instances that would have benefited from the discounts that come with
AWS Reserved Instancesweren't taking advantage of these discounts which can amount to up to 70 percent lower bills. This one takes longer to assess but once you know you will be staying in the cloud for a year or more, there's no excuse not to take advantage of this. Customers using Cloudyn or similar cost tracking tools that continously track resource activity are quickly getting wise to this benefit. Cloudyn's data shows a big increase in adoption of Reserved Instances from January to May of 2012.
Forrester has found a number of other bad habits from cloud users, some of which were noted in our latest Forrester Cloud Playbook report "Drive Savings and Profit With Cloud Computing
" such as not configuring load balancing/auto-scaling properly
to turn off instances fast enough as demand declined, not leveraging caching enough
between application layers or at the edge and not optimising packet flows
from the cloud back to your data center.
It's understandable why we bring our bad habits with us to the cloud. Heck, simply by using the cloud we're saving the company money. But don't let the optics of the cloud pull blinders over the real costs. A medium instance at $0.32 per hour sounds so cheap but when daily consumption leads to $130,000 in annual spend, which was the average for this group of customers, then 40 percent savings is very, very significant.
And for more on the true costs of using the cloud read Andrew Reichman's report on cloud storageset a Forrester.com research alert on analyst Dave Bartoletti
who's next report will provide a full breakdown of the cost of the cloud versus in-house deployments. You can learn more about your use of cloud by subscribing to Cloudyn - your cloud consumption data will be automatically correlated with others in the CloudynDex report to help you continually get a clearer picture of real cloud use. We all benefit by learning from others.