devopsy.kinsta.cloud

Tag: cloud costs

  • Harness Enables DevOps Teams to Rein in Cloud Costs

    Harness Enables DevOps Teams to Rein in Cloud Costs

    Harness today unfurled its Continuous Efficiency application, which enables DevOps teams employing its continuous delivery platform delivered as a service to monitor the cost of deploying applications on a public cloud.

    Company CEO Jyoti Bansal said cloud costs are spiraling out of control within many IT organizations because DevOps teams lack visibility into the true cost of deploying an application on not just one cloud versus another but also specific tiers of services made available by cloud service providers.

    Continuous Efficiency provides DevOps teams with granular cost breakdowns by cloud provider, cloud service, application, microservice and environment, as well as by Kubernetes cluster, namespace, workload, node and pod.

    Bansal said that capability is crucial for organizations looking to identify underutilized resources that they today typically only become aware of when a monthly bill is presented by the cloud service provider. Even then, it’s too late for the DevOps teams to proactively optimize the cloud environment to minimize costs.

    In fact, it’s currently too difficult to correlate the monthly bill presented by a cloud service provider with specific application services that might have been consuming that infrastructure, he noted, adding Continuous Efficiency is specifically designed for DevOps teams rather than the finance team.

    In the wake of the economic downturn brought on by the COVID-19 pandemic, DevOps teams are under more pressure than ever to reduce cloud costs, said Bansal. Not only has the cloud become the single largest expense for DevOps teams, but also IT leaders are being required to better predict costs at a time when organizations are unsure how much revenue they are able to generate from one month to next. The days when DevOps teams could view the cloud as essentially a free resource are now over, he noted.

    Beta customers who tested Continuous Efficiency have been able to realize as much as $500,000 in annualized savings on their cloud costs, Bansal added.

    Harness is leveraging its continuous delivery service to auto-discover cloud costs without requiring IT teams to manually tag cloud resources. As a CD platform accessed as a service, Harness is able to aggregate all that data in real-time on behalf of its customers. The Harness platform supports Amazon Web Service (AWS), Microsoft Azure and Google Cloud Platform, including the Kubernetes based services provided by those services.

    Much like the rest of the IT environment, Bansal said responsibility for managing cloud costs will increasingly shift left toward DevOps teams. As one of the primary consumers of those services, it’s typically DevOps teams who have the most variable consumption of cloud services. All too often, it turns out that costs are being incurred because developers have reserved instances of virtual machines on a public cloud for sake of convenience but are not actually using. When many organizations are deeply concerned about cash flow, tolerance for that behavior is quickly dropping to near zero.

    Rightsizing of cloud computing environments has, of course, been inevitable. The thing that has changed is the current economic climate has transformed rightsizing from being merely the desired goal into an absolute requirement.

  • NetApp Acquires Spot to Rein in Cloud Costs

    NetApp Acquires Spot to Rein in Cloud Costs

    NetApp this week announced it has acquired Spot, a provider of a service that enables IT teams to continuously optimize costs for cloud workloads.

    Anthony Lye, senior vice president and general manager for public cloud services at NetApp, said the Spot service is based on an application programming interface (API) that IT teams add to each application workload. That API then makes regular calls to Spot, which leverages algorithms that determine when and on what types of instances a workload should run to reduce cloud costs.

    The Spot service will then move workloads automatically between on-demand, reserve and spot instances on Amazon Web Services (AWS), Microsoft Azure or Google Cloud Platform (GCP). It is not designed to enable arbitrage across different cloud services; rather, it optimizes the allocation of compute resources on whatever cloud platform an IT team decides to deploy a workload on.

    Lye said cloud computing often winds up being more expensive than on-premises IT environments because IT teams don’t have the tools required to cost-optimize cloud platforms in real-time. Developers often programmatically deploy workloads in the cloud without understanding many of the implications of using one tier of service versus another. Even when they do appreciate the cost, most IT teams don’t provide them with the tools needed to do anything about it. In fact, it’s usually assumed the IT operations team will find some way to optimize costs as they monitor the cloud environment.

    Then, at the end of the month, the IT operations and finance teams receive a bill from a cloud service provider that is all but indecipherable in determining what workloads consumed what level of compute resources and when. In the absence of the Spot service, Lye said the best IT teams could hope for was an analytics report that offered suggestions about how to optimize cloud instances after the fact.

    NetApp views Spot as a complement to its approach to optimizing the consumption of cloud storage resources by deploying its software on a public cloud to manage those resources, said Lye. Collectively, those offerings enable IT teams to address what has now become the single largest cost in the enterprise.

    In addition, Lye noted that Spot can play a key role in enforcing service level agreements (SLAs) because, among other things, it automatically detects whenever a spot instance of a virtual machine is going away. It then automatically spins up additional instances to ensure a workload remains available.

    The debate over the real cost of cloud computing has been going on for years. Initially, most cloud service providers claimed cloud platforms were less expensive than on-premises IT environments because compute resources were paid for on a consumption basis. While that proved to be true for workloads that ran for a finite amount of time, longer-running workloads have proven to be in most cases more expensive to run in the cloud.

    As organizations look to the cloud to deploy cloud-native applications that are more flexible and resilient in the wake of the COVID-19 pandemic, which has made many on-premises IT environments less accessible, the debate over costs is flaring up again. While organizations need a more flexible approach to IT, they are simultaneously trying to reduce the total cost of IT in the wake of the economic downturn brought on by the pandemic.

    It’s too early to say to what degree IT organizations will embrace continuous optimization alongside continuous integration (CI) and continuous delivery (CD). However, given the current economic climate, it’s more now a question of when, not if, cloud cost optimization will become a major area of focus for IT teams large and small.

  • Self-Service Portal From env0 Reins in Public Cloud Costs

    A platform that provides IT teams with a self-service portal through which they can both control access to and the cost of cloud resource consumption is now available in beta for free from env0.

    Company CEO Ohad Maislish said that while many organizations have implemented self-service portals to control who can access resources on public clouds such as Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP), it’s apparent more workloads will be heading into the cloud in the wake of the COVID-19 pandemic to make organizations more resilient. As a result, there’s now a lot more sensitivity to cloud costs.

    Rather than relying on estimations or manual tagging, Maislish said env0 allows IT teams automatically tag infrastructure in a way that ties cloud resources to a predetermined budget using an engine designed for that purpose. All too often, organizations wind up spending a lot more on public cloud resources than planned simply because there is no way to track spending in real-time, he noted.

    In addition, it is common for organizations to find themselves being billed for “phantom workloads” that are created when developers forget to de-provision virtual machines, for example. The env0 platform provides automatic shutdown and time-to-live (TTL) options to eliminate that issue, said Maislish.

    Env0 is fresh off raising $3.3 million in funding. The env0 platform, expected to be generally available in the fall, also makes available policy controls for managing passwords, keys and tokens along with role-based access controls.

    Prior to the COVID-19 pandemic, there was a tendency for organizations to not pay a lot of attention to cloud costs. Achieving faster rates of building and deploying applications was considered a much higher priority. Most organizations will continue to put a premium on delivering applications faster. However, achieving that goal will be weighted more carefully against the total cost, especially as finance teams start issuing stricter cost mandates. In many organizations, the unfettered access to cloud resources that developers once enjoyed is likely to be more constrained.

    Of course, env0 is not the only platform promising to help organizations rein in cloud costs. However, it has combined access and financial controls in a way many organizations have been unable to achieve. In fact, the biggest challenge when it comes to containing cloud costs may very well turn out to be the will to enforce controls on developers who previously had a wide amount of latitude.

    It’s difficult to assess right now at what rate applications will be moving into the cloud. Many organizations may decide to lift and shift monolithic applications into the cloud as the first step toward making their IT environments more resilient. Others may accelerate the rate at which they are building and deploying new applications as once nascent digital business initiatives are transformed into business continuity strategies. Whatever the motivation, the days when many developers could spin up a virtual machine in a public cloud almost at will are most likely coming to an end.

  • Why AWS Cost Optimization Is About More than Saving Money

    Why AWS Cost Optimization Is About More than Saving Money

    Why is it important to track and monitor cloud costs? The obvious answer is it helps you save money—and everyone wants to save money.

    Yet, cost plays a much larger role in your cloud journey. If you think about cost only in terms of how much you are paying and what you are getting in return, then you’re missing much of the value that careful cloud cost control can provide.

    Let’s take a look at all of the reasons why monitoring and optimizing costs is critical for a successful AWS strategy.

    What Is Cost Optimization?

    In AWS and other clouds, cost optimization is the art and science of identifying and addressing inefficiencies in the way you are using resources.

    Those inefficiencies come in many forms. They could be an EC2 instance that offers more resources than you need for a given workload, which could therefore be replaced by a different, less costly instance.

    They could be a result of using one type of application architecture, like workloads running directly on virtual machines, when another, such as serverless or containers, would deliver the same performance for less overall cost.

    They could be caused by failing to take advantage of the best opportunities available for hosting a given workload. For example, you might be using S3 Standard for storage when S3 Glacier would meet your needs just as well, at a lower cost.

    By identifying these sorts of problems, you can take steps to spend less while maintaining the same level of performance for your cloud workloads. A basic best practice is to use tools such as AWS Trusted Advisor to help ensure that your infrastructure is set up in a cost-efficient way from the start.

    Cost Optimization Insights

    These tools only go so far, and they are not the be-all, end-all of cost optimization. They are designed simply to help choose the right type of configuration for a given workload. However, cost monitoring as a whole goes further, and delivers several other important types of insight into your AWS workloads and strategy.

    Rogue Infrastructure

    By its nature, rogue infrastructure—meaning use of infrastructure for purposes not officially authorized by your organization—is hard to detect. If it wasn’t, it wouldn’t be very good at being rogue.

    If you monitor cloud costs carefully, however, rogue infrastructure will have a hard time hiding. If employees spin up a virtual server in the cloud for personal use, or create storage buckets for organizational data that is not supposed to exist in the cloud, you are likely to discover it when you perform cost optimization operations designed to help identify workloads that should not be running.

    Having a strong IT governance policy in place that requires all cloud-based resources to be tagged is another way to help prevent the creation of rogue infrastructure, and to make unauthorized resources easy to detect. When possible, tagging should be automated to ensure the fastest and most consistent results.

    Security Risks

    Although cost optimization is certainly not the only tool you should use to help identify security issues in your cloud workloads, it provides some useful insight that other tools might lack.

    For example, if you notice that your bills for a certain application or service have increased due to heavier usage without a corresponding increase in legitimate requests, it could be a sign of brute-force attacks or other unauthorized access attempts.

    Neglected Projects

    When was the last time you evaluated each of your applications to ensure it was running as efficiently and securely as possible?

    If you don’t know because you don’t regularly do this, cost optimization can help ensure that you do recognize workloads that could stand to be reconfigured. If you discover that you are paying more than you expect for a given workload, or that one workload’s costs are rising more quickly than those of others without a clear reason, it’s probably time to take a look at that workload and assess whether it should be scaled up or down, migrated to a different type of service, or otherwise adjusted to operate more efficiently.

    Operational Inefficiencies

    Finally, cost optimization can help you identify inefficiencies in the way your organization is run. That is because fees charged by AWS that you could have avoided but didn’t might reflect larger inefficiencies within the organization.

    For instance, consider AWS data storage costs. If your AWS cost analysis shows you are paying early-deletion fees for data stored in S3 Glacier, you should determine why that is happening. It could be simply because you should be using a different storage class, but it could also be because employees are deleting data sooner than intended.

    Conclusion

    Saving money by being cost-efficient should always be one of your goals. But running your overall business in an efficient, transparent manner is another. AWS cost optimization can help you achieve all of this.

    This sponsored article was written on behalf of Eplexity.

    — Chris Tozzi

  • Virtual Instruments Acquires Metricly to Monitor Cloud Costs

    Virtual Instruments Acquires Metricly to Monitor Cloud Costs

    Virtual Instruments has acquired Metricly, a provider of a software-as-a-service (SaaS) application that help organizations to optimize the deployment of cloud application workloads.

    Tim Van Ash, senior vice president of products at Virtual Instruments, said Metricly employs machine learning algorithms to identify opportunities to improve performance and reduce costs. Those capabilities complement an existing VirtualWisdom platform from Virtual Instruments to apply machine learning algorithms to optimize IT operations, otherwise known as AIOps.

    Virtual Instruments focused exclusively on enhancing storage I/O performance before extending the reach of its portfolio to include server performance. With the acquisition of Metricly, the company is further extending the portfolio to span hybrid cloud computing environments, said Van Ash.

    Metricly, formerly known as Netuitive, provides analytics tools that are focused on helping organizations contain cloud costs.

    Metricly CEO Bob Farzami said the company’s tools increasingly are being incorporated via application programming interfaces (APIs) into DevOps processes to provide visibility into costs, as the number of applications deployed in the cloud expands and increases. The acquisition of Metricly adds more than 50 integrations to open source DevOps technologies to the Virtual Instruments monitoring portfolio, which spans databases, messaging platforms, containers and cloud infrastructure services such as AWS Lambda, EC2, ECS, ASG, EMR, Microsoft Azure VMs and Load Balancer.

    Those integrations are critical because as the number of services being made available on AWS increases, the visibility organizations have into their real cloud costs becomes more difficult to both discern and limit, said Farzami. There are already hundreds of cloud services on AWS. Cloud service providers often count on organizations to forget how many instances of, for example, virtual machines they have spun up, only to be presented with an unexpected bill at the end of the month. Those issues will be further exacerbated as organizations more aggressively employ multiple cloud platforms.

    Many organizations are also seeing cloud costs rise over time as the volume of data being dynamically accessed by stateful applications increases. Once those costs reach a certain level, those organizations are realizing it costs less to repatriate those applications back on into on-premises IT environments. Metricly provides a mechanism through which organizations gain more visibility into their cloud costs in real-time, but also identify when specific cost thresholds are about to be reached.

    It will be at least somewhere between five to 10 years before even the majority of application workloads are running in the cloud. Today, after more than 10 years, only about 20% of application workloads are running in public clouds. There are a whole host of reasons organizations still prefer on-premises IT environments, ranging from regulatory requirements and performance to cybersecurity and simple inertia. Wherever workloads wind up being deployed, however, the one thing DevOps teams can count on going forward is that someone in the finance organization soon will be asking more pointed questions about how much it costs to run that workload.

    — Mike Vizard

  • FinOps Foundation to Help Rein in Cloud Costs

    FinOps Foundation to Help Rein in Cloud Costs

    A new consortium has been formed to promote best practices for controlling cloud computing costs.

    Announced at the Cloud Economic Summit, the non-profit FinOps Foundation counts among its founding members Atlassian, Autodesk, Cloudability, HERE Technologies, Just Eat and Nationwide.

    Cloudability co-founder J.R. Storment said the goal is to bring financial accountability to variable cloud spending that can easily spiral out of control if not monitored carefully. The adoption of best DevOps practices makes IT organizations more agile, but not every organization has the financial controls in place to ensure spending on cloud services is aligned with business outcomes, said Storment.

    These days engineers can use their corporate credit card to spin up virtual machines in the cloud. The FinOps Foundation is making a case for more accountability over the spending.

    Gartner estimates worldwide cloud spending will reach $360 billion by 2022, which is five times the compound annual growth rate for overall IT spending. The unfortunate truth, however, is that most finance organizations don’t have a lot of visibility into how those dollars are being allocated.

    To coincide with the launch of FinOps, 451 Research also published a survey of 300 IT and finance executives in the United States and the U.K. that finds more than half (57 percent) of large enterprises worry about cloud costs daily, with 80 percent admitting poor cloud financial management has had a negative impact on their business.

    In fact, the 451 Research report makes it clear most finance departments have little visibility into cloud spending. Almost three-quarters (72 percent) of finance executives acknowledged they have no formal reporting capacity between departments. More than half (51 percent) of respondents admitted to occasionally overspending and 68 percent said they are only alerted to overspend too late. A full 82 percent of finance respondents rely on cloud vendors, spreadsheets and manual processes to track cloud costs.

    Despite much of the variable cost hype surround cloud computing, 73 percent of the finance respondents in the United States and 81 percent in the U.K. still treat cloud as a fixed capital expense rather than an operating expense.

    To help finance teams better understand how cloud spending is being allocated, the FinOps Foundation will provide access cloud economic research and related articles, FinOps training and meetup groups. Early access to the forthcoming “Cloud Financial Management Strategies, Creating a Culture of FinOps” book published by O’Reilly Media will also be provided.

    As IT organizations increasingly employ multiple cloud platforms to run different classes of workloads, understanding the true cost of choosing to use one cloud over another will become more important. At the same time, IT organizations have a greater appreciation for the need to move workloads between platforms as costs rise. Just because an application workload was created in the cloud doesn’t mean that workload should be deployed there. Many long-running applications are less expensive to deploy in an on-premises IT environment. The challenge IT organizations arguably face going forward will be integrating streams of real-time financial data into DevOps processes to make more economically informed decisions about how to allocate their cloud resources.

    — Mike Vizard