devopsy.kinsta.cloud

Category: Infrastructure/Networking

  • Where Should I Store My IaC?

    Where Should I Store My IaC?

    The most successful software companies rely on repeatability, auditability and simplicity when building solutions. The emergence of infrastructure-as-code (IaC) has empowered developers to apply these practices to infrastructure allocation. What are the best practices for storing IaC? Should IaC code be stored with the application and feature code? Should it be stored separately in its own application-specific repo? Or should all IaC code for all applications in an organization reside in the same repo? 

    Let’s explore these questions, determine the tradeoffs between each solution and create guidelines that consider your organization’s needs.

    The Problem With Storing IaC With Feature Code

    The naive option for storing IaC is to keep all IaC code within the application repository that consumes this IaC code as part of its CI/CD pipeline. It’s easy to set up, doesn’t require complex permission allocation and will be fully visible to developers who work on this codebase, thus making it easy to retrieve context. This ease is why 20% of polled developers state they always store feature code and IaC in the same repo. However, there are a number of problems with holding IaC code with feature code.

    Software development best practices include minimizing redundancy. Storing all IaC code within the dependent application’s codebase violates best practices by requiring the infrastructure-as-code to contain not just application-specific allocation information but also all foundational infrastructure. Organizations may classify common infrastructure shared across multiple applications and ecosystems as foundational infrastructure. Examples of such base, foundational infrastructure include Kubernetes clusters and log storage accounts. 

    In addition, storing IaC code with feature code introduces risk. Allowing developers admin-level access to all deployment scripts needed to bring code to production violates the principle of least privilege and makes lateral movement easier for attackers who manage to hijack a developer account.    

    Organizations benefit from a degree of secrecy regarding their deployment processes; the deployment pipeline and its configurations are bespoke and attackers who have a mapping of the services an organization uses can more easily architect supply chain attacks. Storing IaC with feature code makes it so that a code leak reveals all CI/CD configurations for not just the application, but the whole organization. The last thing an organization should do is perform an attacker’s reconnaissance for them, but that’s exactly what happens in the above situation.

    Storing infrastructure-as-code alongside product code may be easier to set up initially, but violates the principle of least privilege and zero-trust principles. This method of storing code introduces unnecessary risk–there must be a better solution.

    Storing All IaC Code Together is Not the Answer

    Since storing IaC with feature code in a common repo isn’t the optimal solution, what about storing all IaC in a dedicated IaC repo? This would allow DevOps and security teams to implement granular access controls and make code redundancy easily identifiable and preventable. However, this organizational structure introduces its own problems.

    When all IaC is stored in a separate, shared repository, it becomes difficult to match feature changes to IaC changes. This scenario introduces friction to the development processes, puts a damper on developer velocity and increases tension between development teams, DevOps teams and security teams. 

    Storing all IaC in a common repository gives developers virtually no control over security decisions affecting their workflows. Between the lack of ownership and the additional steps needed to modify any code-specific deployment scripts, storing all IaC together prevents developers from quickly changing IaC and it unnecessarily hurts the organization.

    IaC Falls Into Two Categories: Base and Feature-Specific

    IaC and other CI/CD scripts ultimately fall into two categories: base and feature. 

    Base deployment scripts may be described as code used broadly by an organization for, among other things, routine deployment actions or to interface with shared resources. Separating this code empowers organizations to allocate base IaC ownership and maintenance responsibilities to their DevOps team, which is well suited to maintain shared infrastructure.

    Feature IaC scripts are the bits of code used to provision and manage resources associated with specific software. Developers frequently interact with the deployment scripts tied to their deliverables; changing release steps, modifying gateways, updating server configurations and many other needs necessitate different handling protocols for feature IaC versus base IaC.

    Now that we understand the difference between the feature and base IaC, we can define proper management strategies for each.

    The Solution: A Hybrid Approach for Storing IaC

    Best practices call for categorizing feature IaC and base IaC differently and allocating the responsibilities to separate teams to maximize security while not adding unnecessary friction to the development and release process. Base IaC should be managed by DevOps teams, while feature-specific IaC should be handled by the code owners of the managed software.

    For your organization, this starts with inventorying the organization’s systems and services. Gaining visibility into how these resources are shared helps inform business decisions regarding IaC code storage, helping minimize redundancies within deployment protocols. Visibility is integral to defining which IaC is foundational to software deployed by the organization versus IaC which is specific to certain applications, libraries and other software deliverables.

    Helping DevSecOps Teams Manage IaC and More

    Best practices don’t mean much if organizations can’t enforce them. Some concluding tips: 

    • Track resources, including registries, repositories, builds and cloud/IaC deployments
    • Implement granular access controls, enabling the appropriate allocations of ownership depending on whether the resource is meant to be shared or application-specific
    • Provide visibility into the use of resources which empowers organizations to adhere to the principle of least privilege without adding unnecessary friction to the development process
    • Provide a comprehensive asset inventory by integrating directly with the components of your organization’s software supply chain

    These integrations allow complex insights to be derived that are simply not possible with other security tools. In addition, these integrations allow for constant monitoring for security updates, thus reducing the time between a zero-day being discovered and patching the vulnerability.

  • How IaC Helps Relieve Development Pain Points

    How IaC Helps Relieve Development Pain Points

    Developers spend much of their day digging into their systems’ guts, allocating storage to an application, connecting the security system to the user interface and tracking which version of the application is where in the development cycle, among other tasks.

    That’s why a new category of software has emerged, infrastructure-as-code (IaC), that promises to automate system infrastructure provisioning and integration. It frees up time so developers can focus more on frontend solutions rather than backend configuration management. First, though, let’s find out what IaC is and how it helps your organization.

    What is Infrastructure-as-Code (IaC)?

    IaC reduces system infrastructure creation and configuration time through the use of sets of tools, protocols, languages and processes. In conventional data center system management, each configuration entry and change has historically required manual action. Now, technology teams can manage everything in an intelligent, automated manner. 

    With IaC, infrastructure configuration management data is stored within standardized files. The tools monitor and maintain the state of the infrastructure and ensure that the right resources are available whenever an application runs. This saves DevOps teams from excessive manual effort. In essence, large-scale configuration and management become dramatically simplified.

    Development Pain Points 

    Currently, development is a cumbersome process because friction arises in many places. A great deal of progress has been made in speeding up the front end of application development—building the feature set. Still, the process slows down on the back end when the software components are tied to the computer infrastructure. 

    Historically, software was deployed as one large, monolithic block of code. Updates were tedious to make because all of the application’s code had to run, even if it wasn’t involved in the update or bug fix. The advent of containers and microservices enabled developers to design applications that are modular in nature and therefore easier to update, add to or debug.

    However, the application still has to be tied into the existing compute infrastructure. Cloud solutions—like AWS, Google Cloud Platform or Microsoft Azure, for instance—provide companies with quick access to large volumes of computing processing power. But that application software must be told how to access the network, present security information, allocate storage and run code on central servers.

    Because of this, developers spend a lot of time building and maintaining infrastructure-as-code. To do so, they use languages like Python and solutions such as Chef and GitHub. They want to use automation to simplify and accelerate the process, but here, again, they encounter some roadblocks.

    Workflow Bottlenecks

    As applications become larger and components become smaller, developers find themselves mixing and matching software from a wide variety of places. Tracking what’s where in the process is a frequent challenge. Current workflows are often non-intuitive and rely heavily on pull requests. Some solutions try to address the problem with multiple workspaces, but the end result is fragile and nondeterministic code.

    Without mapping of projects to branches or tags, someone who comments on an approved pull request can push unapproved code to production. In today’s fast-paced DevOps work environment, an approval meant for a short-lived test run becomes the current release, creating a ripple effect of unintended consequences that create confusion and defective code, slowing development—often substantially.

    Access Control

    Most tools do not offer access control models. Developers can track a single repository and a single Terraform project, for example, but as the number of repositories and projects grows, ensuring that proper access control features are in place becomes more daunting. 

    Workspace Challenges

    Legacy waterfall and static development projects allowed IT teams to set their system infrastructure up once and forget about it. But in a dynamic environment, microservices come and go in the blink of an eye. New environments proliferate and product teams must continually configure their own workspaces, which eats up time and resources.

    Drift Detection

    Generic IaC platforms don’t provide mechanisms to detect if the infrastructure is drifting. Because so much configuration is done at runtime, differences arise between the desired and the actual state of the infrastructure.

    Drift has many causes, such as developer or machine script contingencies or the lack of system resources. Too much drift results in unstable applications. Catching such problems early in the development cycle is more efficient than trying to fix a production system. But needed visibility is often lacking, and shortcomings become clear only when the system is fully in use.

    Resource Visualization

    Given today’s dynamic environment, developers need clear visibility. They must be able to closely examine what happens as their job runs. Though they offer many advantages, CI/CD platforms deliver little insight into resource usage from a real-time or historical perspective.

    So, whenever performance problems arise, these platforms offer little insight into what the source of the problem is or how to fix it. Consequently, developers’ attention shifts from application enhancements to troubleshooting.

    Lack of Reusability

    Legacy infrastructure failed to let teams reuse tried-and-true system images across multiple environments. To address this problem, teams often create a new image for each version of each application, slowing development time. 

    High Prices

    Setting up each IT environment manually requires time and manpower, which raises costs. Companies must have dedicated teams of engineers responsible for creating the infrastructure. But they also need supervisors and data center resources, which ultimately boost technology expenses. IaC tools centrally manage the process and create the right environment. That way, organizations pay only for system resources that they truly need.

    How Does IaC Improve the Business? 

    Legacy IT practices are expensive, slow and inconsistent because they rely heavily on manual input. Infrastructure-as-code automates much of the infrastructure management process, which delivers a range of benefits.

    Speed 

    With IaC, provisioning and management are lightyears faster and more efficient than with a manual system. The environment’s infrastructure is defined in code. That code-based infrastructure significantly accelerates each phase of the software development and delivery cycle. Consequently, DevOps teams deliver applications faster and at scale while improving software quality.

    Consistency

    IaC deploys consistent configurations constantly and with no discrepancies. Because of this, it enables fast delivery of consistent infrastructure and environments. This reduces errors and overhead while boosting productivity.

    Eliminating Management Overhead

    Infrastructure automation helps to reduce the number of people with hands on the routing parts of the software development process. That’s because human resources aren’t needed in the management of networking and storage. Companies free up personnel to work on more strategic initiatives.

    Reducing Risks and Errors

    IaC helps to reduce risks and avoid human errors. It also protects against runtime issues and security weaknesses caused by missing dependencies or configuration drift.

    Visibility

    Developers version IaC configuration files the same way they do any other source code file. This gives them complete traceability of any changes. In this way, IaC ensures efficient, exceptionally transparent workflows and speeds up the development process.

    Finding an IaC Solution 

    Maintaining legacy infrastructure was a time-consuming, complex process. IaC is emerging as a solution that alleviates much of the stress and pain that developer teams feel daily. Now, they can speed up the deployment and connection of system resources to applications. Developers can become more efficient and enable the company to respond to business demands in a faster, more reliable and more cost-effective manner. The development process no longer slows down the business. 

  • Why Object Storage is Best for Cloud-Native Apps

    Why Object Storage is Best for Cloud-Native Apps

    A crucial question that plagues cloud application developers is, “What kind of storage should we use for our app?” Unlike other choices like compute runtimes—Lambda/serverless, containers or virtual machines—data storage choice is highly sticky and makes future application improvements and migrations much harder.

    All three hyperscalers have storage services that present block, file and object-based data access. Each of these storage services are mature and offer different advantages, making the choice even harder. Though block and file-based storage has existed for multiple decades, in this article we will illustrate some key differentiators that should make object storage your default storage choice for new applications written in the cloud.

    Storage Type/Cloud Provider Google Cloud Amazon Web Services Microsoft Azure
    Block Persistent Disk Elastic Block Storage Disk Storage
    File Filestore Elastic File System Files
    Object Cloud Storage Simple Storage Service (S3) Blob Storage

    Easy Scaling

    Scalability is an important requirement for most cloud applications. It is expected that horizontally increasing the amount of compute power available to an application increases its ability to process requests, users, etc. to handle peak workload. Most cloud providers also make it easy to scale up compute resources to meet peak demand. As compute resources are scaled,
    block and file-based storage need to be mounted/attached to the new compute instances.

    However, a cursory search will show that these operations can fail or even hang indefinitely for multiple reasons. They also are often hard to debug. The other issue with using file and block storage solutions is that teardown of the compute instance may fail or hang for the same reasons. These issues immediately negate the application’s ability to scale freely as required. This is, however, not an issue with object-based storage, since there is no mount step involved. Your object storage is instantly accessible to the newly-created compute instances.

    Sharing and Consistency

    Data sharing and consistency are where object storage really shines compared to other storage types. In both block and file-based data storage, one instance of an application can end up seeing partial data written by another instance. Application developers end up having to use persistent locks to get around this issue. However, such schemes come with their own sets of challenges: Performance, correctness, etc. Persistent locks end up making an application severely complicated; I have seen even experienced storage engineers make mistakes while using persistent locks. Object storage avoids this problem by not exposing partially-written objects or objects actively being written. Also, note that objects are typically immutable, so once written they can only be overwritten as a whole and not in parts. This means updating data requires expensive read-modify-write cycles. However, most cloud providers help avoid these additional reads via special APIs that can create an object from portions of an existing object like GCS compose, Azure’s put page blob and AWS multipart upload.

    Data Protection

    Errors are bound to happen during application development or rollouts. These errors can end up impacting critical data and potentially disrupt normal application operations. This is why it is essential to have some sort of backup/snapshots configured on the storage that you use.

    Though most storage services have some form of backup/snapshot mechanism, most don’t make it very easy to configure or restore from them (that is, both require multiple steps or the involvement of a cloud administrator). All cloud object services support native data/object versioning capabilities which are extremely easy to enable. So, basically anytime an application updates and/or deletes, the object storage service preserves the older copy of the data. In case an older copy of the data needs to be restored, you can just read the old version and write it as the new object. The careful reader might see that if an application writes/deletes data often, there may be a lot of older versions of the data left behind. One might think these would be hard to identify and remove when not needed. However, all cloud providers support policy-based data life cycle management (see next section) so you can set up policies to delete unnecessary copies. Note that object versioning also provides an excellent defense against ransomware attacks.

    Policy-Based Data Life Cycle Management

    The amount of data being generated by applications is only going to keep increasing with each passing year. This is the reason all the cloud providers support policy-based data life cycle management for their object storage services. Even if you don’t expect to use more than a few gigabytes of data, policy-based life cycle management can help you keep your storage costs in
    check and can help reduce your code complexity around handling application crashes. Policy-based life cycle management especially comes in handy when you or your cloud admin decide to enable features like object versioning and object holds for data protection and compliance reasons. These policies are very simple to set up and can easily be customized to the needs of
    the individual organizations/applications/developers requirements.

    Conclusion

    As one can see from the above, object storage services have been built to enable the development of simple and scalable applications. So, if you are writing a new application from scratch, choose object-based storage to keep your applications simple and easy to maintain.

  • Exploring the Emerging Role of FinOps

    Exploring the Emerging Role of FinOps

    Cloud overspending is becoming a top-of-mind budget item for the modern CFO. To direct resource utilization in the cloud, some organizations are turning to increased automation and more diligent cloud procurement processes. As part of this momentum, new roles are emerging to oversee cloud financial operations.

    Amid rising cloud costs, FinOps is emerging as a helpful discipline. It is not always a specific role but more often a culture that blends engineers, finance and business areas to participate in dynamic cloud cost management. But getting all these players on the same page isn’t always easy — new findings say influencing developers to actually make optimizations is a top challenge.

    The FinOps Foundation recently released The State of FinOps 2022 report, a deep exploration into the role and the state of FinOps adoption across 1,000 companies worldwide. Below, I’ll highlight the key insights from the report to understand current trends and consider how organizations might implement similar tactics to better operationalize their cloud impact.

    Who Is Practicing FinOps?

    What type of person is involved in FinOps? Well, folks come from diverse backgrounds, spanning cloud financial management, IT or DevOps or consultants. Many are also new to the practice in general—the average career tenure is roughly three years. FinOps tends to be a well-paying field—45% of practitioners report salaries of over $100,000 annually.

    FinOps also tends to be associated with larger enterprises—45% of respondents came from organizations with 10,000 or more employees. In terms of industry, it is most common in financial services and IT companies, although studies show the practice is proliferating in many other sectors too. Geographically, FinOps is most adopted in primarily adopted in North America and Europe.

    Team Dynamics

    The average team size is five people, although many feel they’re a “one-person show.” Interestingly, more mature adoption tends to involve a larger team, upwards of 20+ members for established teams. There’s also a notable increase in FinOps within organizations managing between $50 million and $500 million of cloud costs.

    FinOps must interface with many levels of the company, sharing projections with both development and executive leadership. Most FinOps teams report directly to the CTO or other C-suite members. This makes sense, as an executive directive was found to be the most common way to promote a FinOps culture, followed by organic and grassroots adoption.

    Technology and Tools

    In general, FinOps tends to be achieved through cloud awareness and dedicated tooling. The most mature practitioners are “all-in on cloud.” The largest subgroup (24%) runs 1,000 to 5,000 computing instances in the public cloud, hosting technology like virtual machines, containers and serverless functions. Simply put, the average FinOps practitioner has a lot on their plate.

    In terms of tooling, there’s not one cloud cost management suite to rule them all. Teams use an average of 3.7 tools. A variance of tooling here is likely due to rising hybrid multi-cloud architectures and the need to manage costs separately within each cloud environment.

    As such, FinOps tooling includes a mixture of native cloud provider tools and other middleware. Surprisingly, after AWS Cost Explorer, the second most common tool is a homegrown tool. After that, Cloudablity, CloudHealth and Azure Cost Management rank as popular primary tools.

    Top Challenge: Collaboration

    The top barrier has to do with interdepartmental collaboration. Getting engineers to take action on cost optimization was the biggest challenge, cited by 30% of respondents. Other common challenges included accurately forecasting cloud spending (29%), convincing the organization to adopt FinOps (22%), enabling automation (20%) and reducing waste or unused resources (19%).

    Developers in the flow likely don’t want to be burdened by new cost optimization rules, so FinOps may encounter an uphill battle trying to affect low-level change. It will probably take a combination of shift-left IDE-based cost projection tooling and a decree from upper management to jumpstart this change.

    Besides getting everyone on board, FinOps is often tasked with chargeback models—48% of respondents report using chargeback to report cloud costs, the report found. Yet, correlating cloud costs to a single department or team is challenging with so much shared infrastructure. As such, shared costs remain a challenge; so much so that many practitioners don’t even split them up.

    There is also the need to resolve unexpected anomalies and increases in cloud costs, and it can be hard to distinguish between the two. As a result, 53% of respondents said it takes days for their FinOps teams to respond to unexpected cost increases. In the future, finding ways to shorten this timeframe will help save the company money.

    Next Goal: Getting Engineers to Take Action

    The vast majority (94%) of organizations are wasting money in the cloud, according to the 2022 HashiCorp State of Cloud Strategy Survey. Idle or overprovisioned resources can unnecessarily increase the cloud budget, making the need for FinOps evident. Investing in cloud optimization is also another tactic to meet sustainability pledges.

    FinOps is an emerging role that can help forecast spending and reduce waste. But the practice does require time and resources to reach an ROI. For example, a mature team only has a 5% deviation comparing forecasted spend and actual spend, whereas a less mature team can have as high as a 20% deviation. To improve, teams will require automation for evaluating and communicating cloud projections and more established anomaly management best practices.

    But above all, organizations must grapple with the number-one problem: Getting engineers to take action. Perhaps this is due to the fact that most respondents share cost data with technical teams monthly. Instead, more frequent cost transparency would match the pace of agile development and might empower more engineers.

    The State of FinOps 2022 surveyed 1,056 respondents on their FinOps roles. The yearly study is conducted by The FinOps Foundation, a Linux Foundation program. For more details, you can read the entire report here.

  • How’s Facebook Work? They Don’t Know! | Cali. Pay Law | NASA RISC-V Launch

    How’s Facebook Work? They Don’t Know! | Cali. Pay Law | NASA RISC-V Launch

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Facebook engineers admit they’ve no idea what Meta stores (or where), California requires job ads quote salaries, and RISC-V will power future NASA spacecraft. (more…)

  • How to Go Multi-Cloud: A Simple Three-Step Approach

    How to Go Multi-Cloud: A Simple Three-Step Approach

    Odds are your business already uses cloud technology for automation, cybersecurity, IT infrastructure updates and other purposes. But you don’t have to stick to a single cloud server or provider. In fact, you can take advantage of multi-cloud strategies for your organization and reap the benefits for years to come.

    That said, multi-cloud adoption and optimization can be tricky, especially if you don’t have a plan in mind from the get-go. Today, let’s break down a simplistic yet effective three-step approach to go multi-cloud without running into major problems.

    Step One: Map Your Cloud Zoning Policy

    Your first major step should be to map out your cloud zoning policy and plan. In a nutshell, the cloud zoning choices you make can impact your commitments, costs, and even the eventual functionality of the multi-cloud setup.

    By mapping out your clouds, you also map out which processes and applications will run on each individual cloud server or provider. In essence, you decide what gets locked into one cloud, what data has to be ported between clouds and what applications must run on multiple clouds simultaneously.

    Need an example? A cloud zoning policy might cover whether you keep your web surfing or data analytics on separate cloud providers or on the same cloud servers. 

    You should map your cloud zoning policy regardless of whether you do the setup or hire a service. In the latter case, offering a read-to-go zoning map will make the service’s job easier and minimize the possibility of mistakes and/or hiccups.

    How to Determine Optimal Cloud Zoning

    Naturally, figuring out how to optimize cloud zoning can be difficult. The best way to do this is to determine your special focus areas. Alternatively, consider how you want your multi-cloud strategy to best benefit your organization.

    Say that you want to guarantee 100% uptime for your customers or visitors, even if your service goes down or if you experience a data breach. In this case, you can set up your cloud zoning strategy to balance the data load across multiple clouds at the same time.

    Or say that you want to make sure your users have global availability 24/7/365. In that case, you can set your cloud zoning policies to ensure that users can access your data or websites from anywhere around the globe at any point.

    Essentially, figure out what matters most to your organization–and what you want the most from multi-cloud optimization–and zone your cloud applications and policies accordingly.

    Step Two: Architect the Multi-Cloud Environment

    The next major step is to architect your multi-cloud environment. This involves looking at the high-level design of the environment and creating a rock-solid foundation for multi-cloud servers.

    At this stage, you should have at least some idea of how your business will grow and how the multi-cloud environment will assist with its scalability and resource needs. You need to know:

    • Where your data science and machine learning applications need to be
    • Where your product application serves from
    • Where your data warehousing takes place
    • Where cloud security is run from
    • How each of those functions grows alongside the others

    In the architecture phase of a multi-cloud setup, you need to figure out which projects and apps are so-called “cloud agnostic.” Cloud agnostic projects and applications don’t have to be portable; because of this, they can lean on managed services or proprietary IT infrastructure from your organization.

    How to Set Up an Ideal Multi-Cloud Environment

    To reap the most benefits out of a multi-cloud environment, you should adopt a containerized and flexible approach. Not only is this cost advantageous, it also allows you to set up your multi-cloud environment with as much versatility as possible.

    If you architect or plan your multi-cloud environment so it is containerized and flexible, you can work with practically any infrastructure-as-a-service (IaaS) provider. Therefore, you can transition to different cloud hosts or service providers whenever necessary according to your budget or other factors.

    To do this, be sure to perform heavy forecasting. You need to know how much data storage your computing will require, the database utilization your organization will need, the number of computer nodes you’ll likely require and so on.

    As a side bonus, containerization in your multi-cloud setup means that if one server or process goes down, the others are less likely to fail from a cascade effect as well.

    Step Three: Prep for Contracts and Forecast Costs

    The last step in going multi-cloud for your organization is handling the financial aspect of the process. You need to prepare for contracts and commitments alongside forecasting costs.

    Forecasting is critical in this phase because you’ll prepare for contracts and select different cloud services. Because of this, you need to understand how flexible your budget is relative to your infrastructure requirements.

    Specifically, you’ll need to match the costs to every multi-cloud forecast and build your overall resource and money consumption budget. Basically, you need to know:

    • How much you’ll likely spend on multi-cloud processing and services
    • Whether you need certain features, such as website monitoring tools like Visualping
    • Whether your budget can take that forecasted strain

    If the answer to the third question is “no,” you may need to go with a more affordable service or adjust your multi-cloud environment architecture and zoning policies. By forecasting costs ahead of time, you won’t run into a disaster scenario where you already have your multi-caught environment up and running but can’t pay for it, forcing you to scramble as a result.

    Minimize Commitment Risk

    Luckily, there are ways to minimize commitment risk and avoid budgetary disasters. For example, you can use commitment buy-back guarantees, or you can use flexible commitment options like that from AWS. These include computing savings plans, which consume cloud resources in any region and which result in very low savings rates.

    Of course, you can and should also practice very careful budgeting and accounting. Once you ensure that your commitment costs and savings are attributed to the right services, server resources, applications, etc., you’ll have a better idea of how much money and other resources you really need and avoid over-committing to a provider that is too expensive.

    Then you can further avoid commitment risk when planning your migrations of applications between different vendors carefully. Budgetary costs can skyrocket if migrating applications and data between vendors take longer or result in unexpected complications. Therefore, you should make sure that your migrations go quickly and easily or that a cloud service provider offers assistance during this phase (possibly as part of a deal to get you to sign with them in the first place).

    Wrapping Up

    As you can see, you can transition your organization to multi-cloud in a matter of months. But even if you follow the above steps perfectly, remember that you won’t optimize your commitments, performance and costs up to 100%. Still, you can prep your organization for long-term success and multi-cloud benefits with the right preparation and planning.

    With the right cloud services provider, you’ll gain additional assistance and support throughout this process and be able to take advantage of the extra resources from a multi-cloud setup quickly and easily. 

  • 90% Of Orgs Say Multi-Cloud Helps Achieve Business Goals

    90% Of Orgs Say Multi-Cloud Helps Achieve Business Goals

    Multi-cloud is the new normal. And thankfully, this new normal isn’t just happenstance—it’s driving positive outcomes.

    New findings show that most organizations benefit from using multiple cloud providers. Multi-cloud adoptions are helping achieve business goals, such as improving reliability and scalability and increasing overall security and governance. However, nearly all organizations (94%) admitted that they have some avoidable cloud spending and skills shortages continue to hold back a fully-realized multiple-cloud success.

    HaschiCorp recently released the 2022 State of Cloud Strategy Survey: Making Multi-Cloud Work. In addition to showcasing the positive outcomes of multi-cloud adoption, the report found a trend toward automation frameworks that help operationalize multi-cloud. Below, I’ll explore the report’s key takeaways to see what CloudOps and platform teams can garner from current multi-cloud trends.

    Multi-Cloud is Achieving Business Goals

    First off, the use of the public clouds is increasing dramatically—the majority of public cloud workloads will likely double in the next two years. To support this influx, more organizations are moving toward multi-cloud. According to the HashiCorp survey, 60% are already multi-cloud and 21% plan to use multiple clouds within the next year.

    The good news is that a multi-cloud strategy appears to be working. A full 90% of respondents said multi-cloud is helping their organizations achieve their business goals. This figure is an impressive jump from HashiCorp’s 2021 survey, in which only 53% of organizations said multi-cloud was helping them achieve their business goals.

    So, what exactly are companies extracting from multi-cloud adoption? Well, the topmost business goal driving multi-cloud is reliability, at 46%. This is followed by digital transformation (43%), scalability (42%), security and governance (41%) and cost reduction (39%). As we can see, multi-cloud equally benefits various areas.

    Centralized Automation Supports Multi-Cloud

    With the rise of multi-cloud, CloudOps is becoming its own discipline. As such, it’s common for an organization to build a dedicated Cloud Center of Excellence (CCoE) or strategy team to manage cloud operations—86% said they relied on a cloud platform team. Such a team is often in charge of standardizing cloud services, developing cloud best practices, creating operational policies and related duties.

    To aid cloud platform teams, cloud automation is becoming increasingly important. The survey found that 99% see automation as important for operationalizing multi-cloud infrastructure. Multi-cloud automation could help solve inconsistencies across varying cloud workflows. Respondents also found it can be applied to deliver more flexible infrastructure, improve security and governance and better utilize cloud resources. Improving security is especially important here, as 89% said security is a key driver of cloud success.

    So how are these automations built? Well, folks are rarely developing cloud automation tools from scratch—only 27% do so. It’s far more common to either use open source or commercial tools for cloud automation. It’s pretty equal across the board whether these tools are run ‘as-a-service’ or are self-hosted.

    Regardless of the tooling type, companies are experiencing a broad set of benefits from using middleware to manage their multi-cloud operation strategy. 49% rank scalability as a benefit. This is followed by infrastructure provisioning (42%), cost optimizations (40%) and more flexible deployment options (39%).

    (Nearly) Everyone is Overspending on Multi-Cloud

    Unsurprisingly to some, the study found that 94% of organizations are wasting money in the cloud. The following were the top cited areas that contribute to cloud overspending and waste: Idle or underused resources (66%), overprovisioning resources (59%), a lack of needed skills (47%) and manual containerization (37%).

    The cloud overspending dilemma necessitates forethought into optimizations and reduction of idle resources. As I’ve previously covered, some organizations are even tinkering with employing a cloud economist, or Cloud CFO, to keep escalating cloud fees in check. To help this mission, 43% say consistent and automated tooling has or would lead to better utilization of cloud resources.

    Although wasteful spending is pervasive, the majority of companies are forecasting their cloud spending correctly. More than half (57%) said that in 2021 their organization’s projected cloud spend aligned with their actual cloud spend.

    Getting the Most Out of Multi-Cloud

    Multi-cloud is becoming more common across all IT divisions—as is middleware to operationalize and standardize logic between multiple clouds. And, widespread business success with multi-cloud signals increased maturity in this area.

    Yet, multi-cloud doesn’t come without its challenges. Skill shortages rank as the highest factor impeding an organization’s ability to operationalize multi-cloud. Businesses must face other challenges, such as siloed teams, compliance and risk management and lack of training in order to fully realize multi-cloud. Standardized operating models, along with shared workflows and policies, are possible ways to combat these roadblocks.

    The HashiCorp study surveyed 1,000 technology practitioners and decision-makers from around the world. HashiCorp also partnered with Forrester to provide additional analysis of the findings. To read the results for yourself, check out the full report here.

  • VMware Embraces Graph Technology to Manage Multiple Clouds

    VMware Embraces Graph Technology to Manage Multiple Clouds

    At the VMware Explore 2022 conference today, VMware launched a portfolio of multi-cloud management tools, dubbed VMware Aria, based on a graph-based data store that captures the resources spanning multiple clouds and their relationship to one another.

    Accessed via a VMware Aria Hub platform previously known as Project Ensemble, VMware Aria Graph provides a common definition of applications, resources, roles and accounts in near-real-time across a multi-cloud environment that can consist of millions of nodes.

    VMware CEO Raghu Raghuram told conference attendees the Aria Graph is at the core of an effort to enable IT organizations to move beyond “cloud chaos” to embrace a next-generation “cloud smart” approach to managing distributed computing environments that span multiple platforms.

    Based on an event-driven platform, the VMware Aria Graph aggregates data pulled from public application programming interfaces (APIs) into a data store residing on that cloud. The data store is based on a federated model so there is no need to move data into a central repository. Instead, VMware is making use of GraphQL to launch queries across a federated data model. In addition, IT teams can discover dependencies and analyze historical configuration data to discover the root cause of any issue. All operational data is now aggregated in a way that creates a single source of truth.

    VMware has also provided a set of API services that can be used to integrate the Aria Graph data store with third-party platforms.
    In general, the company is committed to accelerating “the great replatforming” of IT environments spurred by digital business transformation initiatives, said Raghuram. The challenges IT organizations are encountering are roadblocks that involve everything from a lack of skills, the weight of existing enterprise applications and fragmented operations and cybersecurity models, he noted.

    The VMware Aria graph data store will enable the company to automate a wide range of management tasks, Raghuram said. That includes using the right cloud for the right application and governing and accessing them at the lowest cost possible, added Raghuram. The ultimate goal is to enable a unified developer experience that provides a frictionless multi-cloud experience across a consistent set of enterprise infrastructure, he said.

    VMware offerings that leverage the VMware Aria Graph include VMware Aria Guardrails for enforcing IT policies, a VMware Aria Migration tool for automating assessment, planning and execution using the VMware HCX platform and a VMware Aria Business Insights tool that surfaces business insights using machine learning algorithms. The rest of the VMware Aria portfolio is anchored around extensions to existing platforms in the form of VMware Aria Cost powered by CloudHealth, VMware Aria Operations and VMware Aria Automation.

    VMware also announced today that its existing VMware Cross-Cloud Service offering for deploying its stack of software across multiple clouds has been extended to add support for the Microsoft Azure cloud.

    Overall, VMware claims there are now five million developers in its ecosystem that collectively launch 350,000 projects every week. That equates to roughly 35 new applications a minute, according to Raghuram. All told, VMware claims there are 85 million workloads running globally on its platform.

    The bulk of those workloads today still run in on-premises IT environments. In fact, Raghuram noted that cloud platforms still only account for $413 billion in IT spending each year, which is only 10% of the total IT spending that reaches $4.4 trillion each year.

    VMware, which is expecting to complete its acquisition by Broadcom in the next few months, is betting that as more of those workloads move to the cloud, the need to centrally manage heterogenous IT environments will play to its strengths.

  • TidalScale Enables Hot-Swapping of Failed Servers

    TidalScale Enables Hot-Swapping of Failed Servers

    TidalScale has updated its namesake software for automating server management to include the ability to detect and prevent impending failures and enable IT teams to hot-swap another server.

    Version 4.1 of TidalScale also adds server wear leveling and preventative health checks that identify servers that should be replaced based on their remaining estimated life span.

    TidalScale CEO Gary Smerdon said that capability is especially critical given the high rate of DRAM memory failure that results in a high rate of downtime caused by server failures. The latest update to the company’s software-defined server management software reduces the failure rate in on-premises or cloud computing environments by a factor of 100 to ensure 99.999% uptime, he said.

    The company also announced that its software is now available on the Amazon Web Services (AWS) Marketplace.

    TidalScale employs machine learning algorithms to aggregate and virtualize CPU cores, memory and I/O across servers to present operating systems with a single software-defined server. That makes it possible to hot-swap individual servers to ensure availability.

    The single biggest cause of downtime in IT environments is memory failures; they are now 100 times more likely than they were a decade ago, noted Smerdon. In most cases, the root cause of the issues is the demands of modern applications running on legacy infrastructure that was not intended to support the latency requirements of those applications, he added.

    That issue can be especially acute in cloud computing environments where infrastructure is often shared by multiple applications, said Smerdon.

    It’s not clear at what rate cloud service providers are replacing legacy server infrastructure, but the ability to determine when to replace infrastructure gives IT teams an added measure of control over servers regardless of where they are deployed. The biggest issue, of course, is not so much replacing a server as it is determining which one actually failed. It often takes longer to identify a failed server than it does to bring a new one online.

    TidalScale is made available by both cloud service providers and manufacturers of servers used in on-premises IT environments, but it needs to be configured and deployed by an IT team. The overall goal is to not only increase system availability but also reduce the total cost of IT by managing infrastructure more efficiently across the life span of a server, said Smerdon. Some IT teams routinely replace servers at specified intervals, regardless of actual usage, to avoid downtime. TidalScale enables IT teams to determine which servers to replace based on the actual wear within the context of a self-healing server farm, he noted.

    Given all the things that might potentially go wrong in an application environment, not having to worry about server failures should enable DevOps teams to spend more time optimizing application performance.

    In theory, of course, modern applications based on microservices should be able to reroute calls to application programming interfaces (APIs) anytime a server becomes unavailable. However, over time it’s not uncommon for single points of failure to emerge as applications are updated. In the meantime, the majority of applications running today are based on legacy monolithic architectures that are much more susceptible to server failures that, unfortunately, are still far too common.

  • OpenText to Create Software Powerhouse – Acquires Micro Focus for $6B

    OpenText to Create Software Powerhouse – Acquires Micro Focus for $6B

    OpenText this week announced it will acquire Micro Focus for $6 billion. The acquisition is part of an effort to reinvigorate a broad portfolio of information management, application development and testing, integration and cybersecurity platforms by accelerating the rate at which they are transitioned to OpenText’s managed private cloud.

    The acquisition is expected to close in the first quarter of 2023, and the combined company expects to reduce costs by an additional $100 million on top of the $300 million Micro Focus previously committed to cutting as part of an ongoing effort to return to profitability and growth.

    OpenText CEO Mark J. Barrenechea told analysts the ultimate goal is to enable organizations to accelerate their digital business transformation initiatives using a set of application programming interface (API)-centric offerings that reside on a private cloud.

    Digital transformation represents the largest opportunity in the history of computing, he said, but Barrenechea also acknowledged that a lot of companies are still struggling with making that transition. “There has to be a better way to get digital transformation right,” he said.

    The Micro Focus portfolio includes everything from a robotic process automation (RPA) offering, a recently-launched IT service management (ITSM) offering—dubbed Service Automation Management X—along with the Vertica database and LoadRunner testing tools. There is also a slew of cybersecurity platforms that include ArcSight, Fortify, IDOL Zenworks and Sentinel along with legacy COBOL application development tools and even the remnants of Novell.

    In effect, OpenText now has one of the broadest software portfolios in the industry after previously acquiring Documentum, Carbonite, Hummingbird and Zix. “The strategy appears to be to simply gather up all the pieces on the chess board,” said Mitch Ashley, principal analyst for Techstrong Research, an arm of Techstrong Group that is also the parent company of devopsy.kinsta.cloud.

    Despite the disjointed Micro Focus portfolio, OpenText has no intention of selling off any of those offerings, said Barrenechea. Going forward, the strategy is to not only accelerate the transition to the cloud but also expand application license renewal services, he added. OpenText also sees an opportunity to expand the appeal of the Micro Focus portfolio into the small-to-medium business (SMB) market, noted Barrenechea.

    All told, it will take six quarters once the deal is closed to align the Micro Focus business model with the one OpenText currently employs, which Barrenechea said should result in growth starting eight quarters after the deal closes.

    There’s no doubt that, once the deal closes, IT organizations that rely on Micro Focus will be closely watching the level of ongoing investment in the portfolio for any signs of one platform being deemphasized in favor of another. It’s also probable that many customers will be relieved to see the stewardship of those platforms assumed by a company that is on firmer financial footing.

    Regardless of how the OpenText portfolio evolves in 2023, however, one thing that is certain is IT organizations have no shortage of alternatives to consider.