Category: Cloud Management

  • VMware Enhances Management Portfolio

    VMware Enhances Management Portfolio

    VMware today rolled out a series of updates to its automation and monitoring portfolio as part of an ongoing effort to simplify management of highly distributed IT environments.

    Updates to the VMware vRealize Cloud automation portfolio include VMware vRealize Automation 8.4’s tighter integration with the open source Salt automation framework the company gained by acquiring SaltStack last year, in addition to a range of tools for provisioning and governing instances of VMware software running on Google Cloud, Microsoft Azure and VMware Cloud on Amazon Web Services (AWS).

    There is also a set of tools to enable organizations running NSX network virtualization software for vSphere to migrate to NSX-T, an instance of network virtualization software that can be deployed anywhere.

    VMware vRealize Operations, meanwhile, is being updated to support a total of 57 AWS services along with the ability to predict the cost of deploying VMware Cloud on AWS. The latter capability includes tracking potential savings generated by powering off idle or unused virtual machines as well as snapshots, orphaned disks and oversized virtual machines. There are also now ten additional dashboards for tracking various costs, including return on investment, total cost of ownership and potential cost savings. IT teams can also more easily track capacity utilization trends on AWS.

    VMware is adding support for the open source Telegraf Agent to enable IT teams to collect metrics from any platform, in addition to providing an update to VMware vRealize Log Insight 8.4 and vRealize Log Insight Cloud that enables these monitoring tools to store data collected from multiple cloud services more efficiently. IT teams will also be able to collect logs from both Microsoft Azure and AWS.

    Finally, VMware is expanding its network monitoring tools to provide greater visibility across multiple clouds.

    Fidelma Russo, senior vice president and general manager for cloud services at VMware, said the rate at which IT teams are consuming the software-as-a-service (SaaS) editions of VMware management and observability tools or full-blown managed services has increased sharply since the start of the COVID-19 pandemic, as IT teams managed on-premises and cloud instances of VMware software from home.

    It’s not clear what percentage of VMware’s installed base are now consuming the company’s automation and monitoring tools, but as the number of VMware instances deployed on public clouds increases alongside on-premises IT environments, so, too, do the management challenges. The choice IT teams generally have to make is deciding between automation and observability tools that are optimized for a specific platform versus adopting frameworks that work more broadly. With the acquisition of Salt and support for Telgraf, it is apparent VMware is moving to expand the reach of its management portfolio beyond its own software.

    In the meantime, IT teams are not only being asked to manage more complex IT environments, but the expectation that, through automation, they will be able to more effectively implement cost controls. This is happening at a time when many organizations have not been quite so sensitive to the total cost of IT. The challenge, of course, is those organizations are being asked to spend money on automation today in the hopes of spending less because of it tomorrow.

  • How to Know if You Need Multi-Cloud — and Succeed if You Do

    How to Know if You Need Multi-Cloud — and Succeed if You Do

    There are many good reasons for running in multiple clouds, as well as times when multi-cloud environments are unavoidable due to organizational history or legal reasons. However, operating in multiple clouds adds complexity, increases operational overhead and can become expensive. Organizations should only pursue it when they have clear business reasons for doing so.

    In this article, I’m going to cover some common situations when organizations can get value out of multi-cloud as well as strategies for success. Because multi-cloud is hard, the first question organizations should ask themselves is, “Do we really need multi-cloud?” So, the first order of business is to outline when technology leaders are better off avoiding multi-cloud altogether.

    When Multi-Cloud is a Bad Idea

    A single application and its data should never span multiple clouds, if it can be avoided. Here are some common reasons organizations adopt a multi-cloud approach when they would be better off avoiding it.

    Short-term bursting (in hybrid cases). “I’ll just tap into a public cloud if my private cloud suddenly needs extra capacity.”

    Disaster recovery. “Let’s run this application on AWS and Google in case one of them disappears overnight.”

    Staying cloud-agnostic. “By running on multiple clouds, we are no longer locked in.”

    In all the cases above, network latency and egress cost will hurt your application and your business. In the best-case scenario, you will spend a lot of time and effort prematurely optimizing for unlikely scenarios.

    Unless you’re perfectly prepared, please, don’t go there.

    When Multi-Cloud is a Necessary Evil

    There are also plenty of times when multi-cloud is a drag on the engineering organization and a drag on the business — but can’t be avoided. Here are totally legitimate reasons to have a multi-cloud setup that will still require more effort to maintain than operating in a single cloud.

    Mergers and acquisitions. Some organizations can end up operating in public clouds and private clouds as they acquire other companies. The effort required to migrate everything to one cloud provider often isn’t worth it, so everyone continues using the same cloud they used pre-acquisition.

    Legal. Depending on the type of data you gather and the jurisdictions you operate in, you may have to worry about where your cloud provider has regions available. There will be times when you need a data center in multiple jurisdictions; none of the cloud providers are in all of them. In those cases, you don’t have much choice, you need to be multi-cloud.

    If you need to use multiple cloud providers, either for legal reasons or because of mergers, it’s best to focus on making multi-cloud management as painless as possible — while still accepting that it will not be as smooth an experience as running all workloads in a single cloud.

    When Multi-Cloud is a Solid Strategic Move

    There are also some very good reasons to adopt a multi-cloud approach on purpose. For example:

    Facilitating engineering speed. Engineering speed is paramount. Companies who focus on using multi-cloud to become cloud-agnostic and avoid lock-in often sacrifice development velocity, but if the organization instead uses multi-cloud to allow everyone on the team to use the tools and environment they are most comfortable with, it can increase velocity.

    Best-of-breed tooling. The cloud providers’ offerings are not totally identical. Multi-cloud can let organizations use the best tools from each cloud provider; for example, machine learning technology from Google, compute from AWS and business intelligence from Azure.

    Better customer experience. Especially if latency is a big concern, a multi-cloud approach can help organizations get physically closer to users, providing a better customer experience.

    There is a common thread here: When evaluating whether or not you have a legitimate need for multi-cloud, the key question is whether you will be fulfilling a psychological need, a theoretical need (in other words, something that you do not actually need at this moment but might at some future date) or a concrete legal or technical need that you can easily describe. If there are no concrete legal, technical or business reasons for pursuing multi-cloud, don’t. There are costs to pursuing multi-cloud, so organizations that won’t actually reap benefits from it shouldn’t attempt it.

    How to Succeed

    Multi-cloud is hard. Here’s how to increase your chances of success so you can get the business benefits of better tooling or better customer experiences from your multi-cloud deployments.

    Use a cloud management platform. Getting a single pane of glass for all your deployments is critical to managing multiple clouds without pulling your hair out. This abstraction, however, comes at the cost of supporting very deep and cloud provider-specific use cases, for example, using obscure service X.

    Standardize workflows with Terraform and Ansible. You need to have processes that are as repeatable and predictable as possible. Even with standardized workflows, you still have to adjust for each environment, but the more standardization the better.

    Use Kubernetes. Kubernetes lets you abstract the infrastructure layer and move workloads around more easily, even across clouds. The caveat here is that you have to adopt microservices and deal with the increased complexity of managing Kubernetes and its underlying infrastructure.

    Each of the above approaches has its merits and flaws. They are not mutually exclusive, and you should not treat them like they are. Be prepared to mix and match depending on your needs.

    Weigh the Costs and Benefits

    Sometimes companies are not realistic about their technical capacity or what their needs are. They want the ability to do the latest thing they read about without thinking about whether or not that technology or approach will provide business benefits.

    Multi-cloud adds a huge amount of complexity to your deployment. If multi-cloud truly is essential, however, use tools and platforms that hide that complexity from users and can simplify the management process.

  • Cloud Migration Monitoring: A Practical Guide

    Cloud Migration Monitoring: A Practical Guide

    Almost every organization has transitioned some operational systems to the cloud. DevOps professionals are an essential part of migration projects, facilitating the migration itself, and making sure applications work as expected and provide satisfactory performance after the migration.

    To make your next cloud migration a success, you must get monitoring right. Applications must be monitored at every stage of the migration—when the application is still running on-premises, during migration and once the new workload is running in the cloud.

    In this article, we will explain key metrics to track during your migration, monitoring considerations at every stage of the migration cycle, and free tools offered by the major cloud providers that can help you monitor your migration.

    What to Monitor During a Cloud Migration

    There are many components involved in a cloud migration process. There are several approaches to cloud migration—each migration is unique, and might necessitate setting up different metrics. However, there are several aspects that can be considered mandatory when monitoring your cloud migration process:

    • Tracking application traffic. Tracking application traffic can help you validate that application configurations are properly defined and validate the availability of the entire application. You should ensure that all requests arrive to the new server, and keep an eye on the error rate. Additionally, watch for response times to ensure your application is not exceeding the planned thresholds.
    • Tracking your costs. Tracking costs can help you ensure that you are not incurring unexpected losses. The majority of organizations migrate to the cloud to reduce their costs. However, incorrect implementation of cloud resources may lead to losses. You can gain visibility into your costs by leveraging first-party cloud tools, like AWS Billing, which natively integrate into the pipeline, monitor costs, and show visualized data in customized dashboards. You can also use these tools to set up alerts.
    • Monitoring application security. Application security monitoring can help you identify security vulnerabilities and incidents during and after the migration process. You should monitor infrastructure configurations, such as internal traffic, host open ports and instance processes that are actively accessing different services. Additionally, keep watch for API hits per second, which might indicate a distributed denial of service (DDoS) attack.

    Monitoring Aspects of the Migration Life Cycle

    Cloud service providers typically identify three main phases of the migration process, which are referred to as the plan, migrate and run phases.

    Plan Phase

    The plan phase helps you assess the performance levels of your applications before migrating. The goal is to achieve an in-depth understanding of how workloads are functioning pre-migration, as well as the current state of user experience. This data can then be used to establish performance benchmarks and baselines.

    Performance baselines, like error rates, page load times and slow transactions, can help you set up configuration alerts that keep track of outages across the cloud migration process. You can then quickly diagnose the situation and apply fixes that ensure application health during and after the migration.

    Here are several questions that can help you during the plan phase:

    1. What is the application baseline?
    2. How will the migration process affect user experience?
    3. What are the potential technical challenges expected during the migration process?
    4. Can the corporate network support additional bandwidth needs?
    5. What are the steps needed to start the migration process?

    Migrate Phase

    The migrate phase involves all processes required to securely move components from one location to another. During this process, you should continuously monitor performance to ensure user experience remains within the pre-established baseline. However, migrating to the cloud means you no longer have control over all configurations.

    To ensure application performance and stability during the migration process, you need to identify potential roadblocks and issues, and continuously monitor and validate the process. You should test critical aspects like application response time and call counts, and troubleshoot issues quickly.

    Run Phase

    The run phase starts right after the migration process is complete. During the post-migration phase, you need to establish proactive monitoring processes. The goal is to ensure your application performs well in the cloud; using the expected resources and without incurring overhead.

    When monitoring your application, keep track of service level agreements (SLAs) to ensure the cloud vendor meets the requirements agreed upon. You should also monitor for changes and review individual variables that might cause issues.

    Additionally, you should make sure all connections between applications are maintained after the migration.

    Free Cloud Migration Monitoring Tools

    The three major cloud providers—Amazon, Microsoft Azure and Google Cloud—provide tools that help automate migration to their clouds, and also assist with the crucial aspects of monitoring. These tools are offered free, because it is in the best interest of the cloud providers to allow easy, frictionless migration.

    The following table summarizes migration monitoring capabilities available from each cloud provider.

    Cloud Provider Tool

    What it Helps Monitor

    AWS Migration Services

    Helps automate migration of applications that rely on databases.

    • Continually monitors source and target databases
    • Able to restart failed migrations
    • Oversees continuous data replication
    • Integrates with first-party AWS monitoring tools

    Azure Migration Tools

    Provides a migration hub that lets organizations manage large migration projects.

    • Tracks migration process and current status for server and database migrations
    • Allows external tools to integrate and track migrations
    • Reports on compliance aspects of migrations

    Migrate for Google Compute Engine

    Formerly known as Velostrata, this tool automates migration of on-premises systems, with automated roll back in case of errors.

    • Validation of workloads for migration compatibility
    • Ongoing oversight of streaming data transfers
    • Ability to roll back migration to on-premises systems in case of error

    Monitoring is critical for a successful cloud migration, and it’s important to use monitoring at every stage of a cloud migration:

    • During the plan phase, establish a baseline of application performance before it is migrated to the cloud.
    • During the migration phase, carefully monitor application behavior and troubleshoot issues.
    • During the run phase, validate that the application performs as expected and is meeting SLAs.

    I hope this will be useful as you plan and execute your next cloud migration project.

  • LogDNA Adds Usage Quota Tools to Help Control Costs

    LogDNA Adds Usage Quota Tools to Help Control Costs

    LogDNA announced today it will enable IT teams to apply usage quotas to help rein in log management costs.

    Tucker Callaway, CEO, LogDNA, said as applications become more complex – thanks, in part, to the rise of microservices – the amount of log data that a DevOps team can collect has significantly increased. IT teams can now assign DevOps teams both hard and soft quotas to limit the amount of log data collected during a specific time period, Callaway said.

    IT teams can set a daily or monthly hard limit on the volume of logs stored or, alternatively, they can also set soft daily or monthly quotas that can apply throttling logic to ensure mission-critical log data will continue flowing through LogDNA as usage volume approaches the limit threshold. It will then be up to each IT team to determine, based on the tasks at hand, whether to enable a DevOps team to exceed those thresholds, Callaway said.

    IT teams can also make use of soft limits to reallocate storage resources from one DevOps teams to another based on how much log data might be generated in a specific time period, Callaway noted.

    LogDNA will notify team members any time a quota is hit, or when an exclusion rule is triggered because a quota has been reached. Those alerts can also be integrated with email systems or the Slack messaging platform. Logs that are not retained are not counted toward the monthly bill, but are displayed and will trigger other relevant alerts.

    In the wake of the economic downturn brought on by the COVID-19 pandemic, IT organizations are, generally, more sensitive to costs. At the same time, many of them are encouraging developers to assume more responsibility for application life cycle management as they embrace DevOps best practices. The usage quotas are intended to enable IT teams to strike a better balance between what can be two competing agendas, Callaway said. IT leaders are not necessarily trying to discourage DevOps teams from collecting log data as much as they are trying to make sure costs are predictable, Callaway added.

    Log data, of course, is one of the core elements of any observability strategy. While observability has always been a core DevOps principle, the number of organizations that have been able to meaningfully achieve it has been limited. However, as organizations start to deploy microservices with multiple dependencies, the need to provide context when troubleshooting those applications has become more acute.

    It’s not clear to what degree more granular control over log data might sway an IT organization toward one platform versus another. However, when it comes time to justify a choice to senior IT and business leaders, it’s always a good idea to address, at some point, how one decision versus another might have a more positive impact on the bottom line.

  • Cloud Migrations Demand Risk and Compliance Maturity

    Cloud Migrations Demand Risk and Compliance Maturity

    The COVID-19 pandemic brought undeniable disruptions for organizations and their employees whether business, personal or otherwise. Across the globe, businesses and governments alike were forced to try and manage these disruptions. For nearly all organizations, digitization initiatives were accelerated in a short period of time, from implementing work-from-home policies and launching new applications to support a distributed workforce to adopting artificial intelligence (AI) to adapt supply chain processes and more.

    According to Gartner, by 2022, 30% of all security teams will have increased the number of employees working remotely on a permanent basis and by 2023, 40% of all enterprise workloads will be deployed in cloud infrastructure and platform services. The distributed cloud enables organizations to provide products and services when they’re needed in this era of work-from-anywhere, whether to their employees or customers.

    Even in sectors such as energy and utilities, which are historically heavily reliant on standard on-premises installations and had often avoided cloud adoption, business leaders are realizing the value of migration, especially in light of the global pandemic. These businesses are able to deliver digital products with speed, experiment with tools such as artificial intelligence (AI) and robotic process automation (RPA) to increase productivity and to lower the total cost of ownership (TCO) across assets, among other benefits. Still, cloud migration risks abound, and new challenges may arise. A risk that cannot be ignored is the cybersecurity risk. Meeting regulatory IT compliance and managing risks involved with cloud computing are top challenges facing those migrating their workloads to the cloud.

    Cloud Migrations and Security: Clear Risk and Compliance Gaps

    Many cloud providers such as Microsoft Azure, AWS, Google Cloud and others have a global network of service models that include compliance teams and consulting organizations that help with risk and compliance for their cloud instances, whether public, private or hybrid cloud. Many of them have even built tools for customers to use to implement basic risk and compliance management in-house, leveraging relevant data and applications. However, monitoring and meeting security and compliance controls that span people, processes and technology for cloud environments, and in the broader context of the enterprise, is complex. This is one cause of cloud migration risk, and is a challenge for many other reasons; lack of measurement, visibility and accuracy are three of the greatest risks when migrating to the cloud. The point solutions that currently support most cloud instances don’t elevate the posture of cloud environments to that of the enterprise risk posture, and the majority of assessments still remain point-in-time and qualitative. Metrics are fractured and far from holistic, and very few, if any, solutions can provide insight beyond compliance and into real-time risk management.

    IT and security regulations and standards are filled with requirements that were created before the cloud became a commodity. In the energy sector, for example, cloud security isn’t taken into account because regulators and industry leaders couldn’t fathom those platforms becoming as pervasive as they have, because on-premises installations were standard to the industry. On-premises installations are still a mainstay in energy, power and utilities, and for those who have become more comfortable with cloud migration processes, there is a clear and pressing need to leverage their human capital, processes and technologies to implement robust risk management practices.

    Beyond regulatory compliance lags, many distributed organizations opt to have multiple providers in place, requiring a multi-cloud approach to compliance requirements and risk assessment. As more organizations consider cloud migration risks and begin shaping their cloud migration strategies, there are some innovations that address risk management and compliance in the cloud, but not many. Measuring, managing and reporting on compliance frameworks, making the shared responsibility model actionable, and getting a view into risk are all serious challenges. Cloud providers will continue to mature and bring new innovations to their services, but, to date, there hasn’t been a lot of anticipatory work done in this area. The focus has largely been on creating reactive solutions. In heavily regulated countries, the challenges only become greater.

    Leverage AI Automation for Compliance and Risk Management

    There is a shift occurring in cybersecurity and IT risk management, calling for the dramatic disruption of the legacy IT governance, risk and compliance (GRC) space and demanding a reevaluation of how we manage compliance and risk in the digital age. For years, data has been aggregated manually and analyses performed on out-of-date information. With the increasing availability of automation, the five functions of the NIST Cybersecurity Framework – identify, protect, detect, respond and recover – are becoming more continuous in nature and shifting into real-time management, from assessment to reporting and more.

    Leveraging this technology in the cloud is no exception, but those who look to reinvent their approach must look for solutions that go beyond the siloed capabilities of cloud security posture management solutions and similar markets.

    Ultimately, the true test of this next-generation approach comes when organizations are able to roll all of this data up to risk. With risk metrics that are supported by drill-downs, trend reports and risk profiles, executives can get the visibility they need into their posture with the most up-to-date data, informing their key business decisions. Using this next-generation approach to risk will inform global expansion, allow executives to evaluate risk across lines of business, and increase cyber maturity in any cloud-based organization.

  • Too Many Organizations Get Public Cloud Wrong

    Too Many Organizations Get Public Cloud Wrong

    The cloud has been getting a lot of press in the past few years—and with good reason. The elasticity and flexibility of resources are transformational for businesses. However, there are also drawbacks. For most enterprises, it’s a significant undertaking to migrate on-premises applications into a public cloud environment.

    Modern applications today are rarely self-contained entities. Application dependencies occur when software components, applications and servers rely on one another to deliver a service. The typical enterprise manages hundreds, or even thousands, of interconnected components and services, forming a complex web of dependencies.

    If an application is migrated to the public cloud without understanding the dependent services, severe performance degradation can occur with applications that are still on-premises. For example, moving an e-commerce shopping application to the public cloud but leaving a key database on-premises will result in consumers experiencing a slower response.

    The Repatriation Problem

    IDC research shows that fewer than 20% of enterprise applications have been moved from onsite data centers to the public cloud, not because companies do not see value in the cloud, but because the process of getting those applications to the cloud is complicated.

    A survey of 350 cloud decision-makers in the United States and UK commissioned by Virtana and fielded by Arlington Research in November 2020 confirmed this. More than two-thirds (68%) of respondents have already migrated 25% or more of their applications to public clouds, supporting the IDC finding.

    Clearly, businesses recognize the value of the cloud. But another surprising result confirms that the migration process may be problematic. A whopping 72% of respondents said they have moved one or more applications from the public cloud back on-premises. Considering the time and effort cloud migrations require, this is a striking result. While there may be cases where repatriation is undertaken strategically to support evolving business needs, the reasons cited by respondents show most were rollbacks required because of unanticipated problems.

    There Is No Single Reason Why

    Almost half (41%) of respondents said they repatriated applications that should have stayed on-premises. The reality is that not all workloads are a good fit for the public cloud environment. You need a solid understanding of the various attributes of different workloads, such as data, back end, privacy and security requirements, to determine their inherent suitability for a public cloud environment.

    Technical provisioning issues challenged 36% of respondents. The sheer number of configuration options available—combined with the variety of computing dimensions that you have to take into account—means there are a lot of puzzle pieces that have to fit together, or you could run into problems.

    Almost one-third (29%) of respondents reported application performance degradation in the public cloud. The public cloud is a fundamentally different environment than an on-premises data center, and creating a hybrid environment further alters the nature of the infrastructure, so you cannot simply expect workloads to attain the same performance levels as you migrate them. And if performance slows to an unacceptable level, any benefits you gain by moving applications to the cloud are overshadowed.

    Around one-fifth (21%) admitted that they selected the wrong public cloud provider. Cloud providers are not all the same, and picking the wrong one, for any reason, can be an expensive and disruptive mistake.

    Finally, one-fifth (20%) of respondents repatriated applications because of unexpected costs in the public cloud. The public cloud’s pay-for-what-you-use model is appealing compared to the on-premises capex model, where you pay for capacity regardless of whether you use it. But there is a dark side; you have to pay for what you use in the cloud even when that usage is unexpected. Whether that is due to poor planning, an unanticipated surge or workload drifts over time, getting a nasty surprise in your end-of-month bill can put a big dent in your cloud budget.

    Eliminate Repatriation With Better Planning and Observability

    Any one of these reasons alone reflects a challenging setback to overcome. Unfortunately, many organizations run into more than one of these problems. In fact, one-third of respondents cited two or more of these reasons for repatriation, and 12% experienced three or more of these issues. This seems to indicate that organizations’ migration planning efforts are inadequate. It is not for lack of investment—78% of survey respondents said that even during the current challenging economic climate, they are continuing their migration to the public cloud at the same or an accelerated pace, signaling they believe public cloud deployments to be a strategic investment.

    The real problem is a lack of precision in observability—of workload characteristics in the data center pre-migration and of workload utilization and performance in the public cloud post-migration. The better you understand your applications, in minute detail, within the context of the environments they are moving from and to—a ‘know before you go’ approach—the less likely you will have to hit the “undo” button on your public cloud migration.

  • Why Every DevOps Team Needs a Spot Instance Strategy

    Why Every DevOps Team Needs a Spot Instance Strategy

    Most DevOps teams use the public cloud extensively and focus a lot of energy on reducing cloud costs. According to one estimate, U.S. businesses spent $14.1 billion in 2019 just on wasted, unused cloud resources. Spot instances are one of the most important ways to reduce the cost of public cloud services.

    Spot instances are not a new development. Amazon was the first to announce this pricing option in 2009, turning cloud computing into a market influenced by the dynamics of supply and demand. Microsoft Azure took more than a decade to follow suit, announcing its spot virtual machines offering in May 2020.

    The spot pricing model is simple on the surface but can be complex to implement. Spot instances are how cloud providers sell their unused capacity. When compute instances are not ordered by anyone via the regular, on-demand pricing model, the cloud provider puts them up for auction at very favorable prices, usually around 10% to 20% less than the on-demand cost.

    However, it’s not so simple to get this 80% to 90% discount. When another cloud customer requests the spot instance, the cloud provider sends a notification, and you need to immediately move your workloads before the instance is terminated. This makes it difficult to use spot instances for stateful, database-driven applications, or those that require high availability.

    This is not the only challenge. The market price fluctuates and is much more volatile than on-demand pricing. This means the level of discount that can be achieved over time is relatively unpredictable.

    However, there are at least three ways savvy DevOps teams can make great use of spot instances:

    1. Automation—DevOps teams are proficient with a range of automation tools, including configuration management, infrastructure as code (IaC) and cloud provider auto-scaling tools. These can be used to manage workloads in clusters and automatically fail over when a spot instance terminates.
    2. Dev/test environments—DevOps teams are responsible for setting up development and testing environments, which are extremely suitable for spot instances because they can usually tolerate brief interruptions, and, in many cases, are not stateful.
    3. CI/CD jobs—running jobs on Jenkins, GitLab and similar tools can be easy to scale on spot instances. These jobs are stateless, and if an instance drops, it’s easy to rerun the job on another.

    AWS Spot Instances

    AWS Spot Instances let you buy unused Amazon EC2 computing power at a significantly discounted price. You can specify a price, and when a spot instance is offered at that price, it is launched with the Amazon Machine Image (AMI) of your choice.

    How Spot Instances Work on AWS

    Spot instances are priced at a variable spot price, which is adjusted according to supply and demand conditions. To see current rates, use the AWS Spot Instance Advisor.

    You create a spot instance request, specifying what instance types you are interested in, and the availability zones (AZ) in which they should run. If capacity is available, and their current price is less than your maximum bid, instances are launched.

    Spot instances continue running until:

    • Capacity is no longer available (because the instances were requested by on-demand customers).
    • The price has risen over your maximum bid.
    • You request to terminate the instance, or it is automatically terminated by auto-scaling.

    You can also order spot instances with a predefined duration—you then pay a static hourly rate for that entire duration (even if market price changes in the interim).

    Automation Options

    You can automatically scale instances on Amazon EC2, including both on-demand instances and spot instances in a single auto-scaling group. If spot instances are not available when scaling up, the group can use regular on-demand instances.

    Amazon also supports mixing in reserved instances and savings plans (additional ways to save on on-demand instances by committing to a certain period of time or total capacity). So you can combine multiple saving methods in the same auto-scaling group.

    You can improve availability by deploying applications across multiple instance types running in multiple AZs. By allowing multiple instance types, you tap into multiple pools and increase the chances of obtaining a spot instance when you need it.

    Azure Spot Instances (Spot VMs)

    Azure offers Spot VMs that give you access to unused compute capacity. You can request a single spot VM, or launch multiple spot VMs using an Azure VM Scale Set (VMSS). Spot VMs replaced the previous Low Priority VMs feature, which let you purchase VMs that were in low demand on Azure for a reduced price.

    The spot price of VMs on Azure depends on the total capacity available for that specific instance size and SKU (instance type) in the Azure region. Azure commits to changing pricing slowly—avoiding sudden spikes—to maintain pricing stability and make it easier to manage budgets.

    Like on Amazon, discounts fluctuate significantly, and spot VMs can be up to 90% cheaper than the base price of the same VM.

    How Spot VMs Work on Azure

    The Azure Portal provides access to Azure spot VMs. When you create a spot VM, you can see the current price for the selected region, image and VM size. For consistency, prices are always in U.S. dollars, even if you use a different base currency for billing.

    There are two options for eviction of spot VMs—you can choose the condition on which spot VMs will be evicted:

    • Maximum price eviction—You set a maximum bidding price, and when the spot VM rises over that price, it is evicted.
    • Capacity eviction—You always pay the current price of the VM (without setting a maximum price), and when Azure does not have sufficient capacity of the requested VM type, your VM is evicted.

    When a VM is evicted, Azure applies an eviction policy called Stop / Deallocate. This means the instance is paused, but attached disks remain, and you are still charged for them. When the price goes down or capacity becomes available, the instance is restarted and continues working on the same disk data.

    Automation Options

    Azure provides virtual machine scale sets (VMSS), which can automatically increase or decrease the number of VMs running your application. You can create a scale set that includes spot VMs, and as your application scales, more spot VMs will be added as they become available. Spot scale sets operate in a single fault domain and do not guarantee high availability. Unlike AWS, Azure currently does not allow you to mix on-demand VMs and spot VMs.

    Both Amazon and Azure provide robust capabilities for cost savings using spot instances. Azure’s offering is newer and provides less-advanced bidding and auto-scaling capabilities, but these are expected to be added as the service matures.

    Whether DevOps teams choose to run in AWS, Azure, or both, they literally cannot afford to ignore spot instances, especially for low-criticality workloads like dev/test and CI/CD job execution.

  • Security Policy Management and Hybrid Cloud with Tufin

    Security Policy Management and Hybrid Cloud with Tufin

    In this TechStrong TV / DevOps Chats interview, Colby Dyess, director of cloud product management at Tufin, joins Mitch Ashley to discuss security policy management across hybrid cloud, multi-cloud and cloud-native security. Recorded in late 2020, Colby shares some experience from working with customers during the rapid migration to the cloud in 2020.

    The video and podcast audio of the conversation is below, followed by the transcript. Enjoy!

    TechStrong TV Video

    DevOps Chats Podcast Audio

    Transcript

    Mitch Ashley: I’m joined today by Colby Dyess who is director of cloud product management at Tufin. Welcome, Colby. Welcome, it’s good to be talking with you. Would you introduce yourself, tell us a little bit about you and of course, tell us about Tufin.

    Dyess: Yeah, well, you’ve got my name and my title, so I’m dialing in here from beautiful Southern New Hampshire where it’s fantastic weather. So, I’ve worked for Tufin Technologies, Tufin is really well known in the industry for providing security policy management, mostly in the organizations like the Global 2000 complex environments where you need to make changes to the network environment and you need to be able to do it quickly, safely, ideally you do it automatically and it’s secure. The Tufin product suite helps organizations do that and we’ve done that in the traditional space. Of course, our customers have adopted cloud, cloud native platforms, Kubernetes, and we’ve been taking these same capabilities for securing the environments and bringing them into the cloud, so no matter where you put your workloads, we can help you, and that’s mostly what we’ve been focused on.

    Ashley: Good stuff. Boy, the center for the universe for a lot of folks. Kinda speaking of that and just thinking about the timing of where we all are, you know, there’s a lot of statistics. I run an analyst firm as part of MediaOps organization called Accelerated Strategies Group. Anyway, we have a lot of data showing some of the acceleration that’s happened, both in digital transformation and also move to the cloud, just in the last six to nine months. I don’t think that’s news to anybody, right? I don’t know if you need data to say that, because it’s happening probably in most people’s organizations.

    Dyess: Right.

    Ashley: If I put on my security hat for a moment, we always get plenty of notice when we’re making changes and doing new projects and accelerating stuff—sarcasm—so, I’m sure there’s a lot of security folks who are scrambling still just to not only keep pace, but make sure we’re all doing a good job of securing environments that our developers are taking this into, making sure we’re understanding the services we might be using now that we weren’t before, cloud services that we’ve been in. I’m sure you’ve seen a ton of that happening with your customers.

    Dyess: Yeah. It’s no surprise. Organizations have been moving to public cloud environments for a long, long time, at least, technically, for the past 15 years, they’ve been doing it, but it hasn’t always been sanctioned, right? Let’s say over the past five or eight years, organizations have really begun to intentionally adopt the cloud. And now this year, of course, everybody’s moved to the cloud about as quickly as they could.

    What we’re seeing, at least what I’ve seen a lot is, security is now scrambling to try and get ahold of some visibility—what’s really going on inside these cloud environments—and trying to put some sort of guardrails around the services that the Dev teams are using. That’s a real challenge because they’ve been focused so much on how to secure most of the on prem environment or use firewalls to protect the cloud space, but now they realize they’ve got to go a bit deeper, because the threats are pretty real out there and they’re really dynamic. And if you’re not sure how to secure the cloud environment, you’re going to leave yourself open to threats.

    So, security has really been spending a lot of time trying to learn more about cloud and cloud native security controls. We spend a lot of time helping customers figure out the difference between cloud native security controls and on prem stuff and then how to combine them so they can be assured it’s all secured. We see that a lot, yeah.

    Ashley: Well, you know, I mean, we’ve all been doing firewalls and intrusion prevention and access controls and the list goes on.

    Dyess: Yeah.

    Ashley: So, the topics aren’t new, but the environments are different and the services. It’s not like you can go into the cloud—no, let’s use all the things I’m normally using in my on prem environment. Maybe you can get some software versions of that. Sometimes it’s just better to use because it’s better integrated, you spend less time making the environment work by using what’s provided by the cloud provider.

    Dyess: Yeah. That pattern is the same everywhere. I’ve seen that you’re gonna go to a new environment, you wanna take some of the things that have worked for you in the past and carry them forward. And a lot of times, that makes sense as you try to move quickly into the cloud. If you’re not familiar with cloud native security controls, you probably think, “I’ll just use firewalls and the process for managing firewalls in the cloud.” And there are a lot of options to do that, a lot of the traditional vendors in that space have tried to make a pivot to offer their same services, same capabilities in the cloud environment. So, it’s no surprise that we see a ton of that.

    At the same time, those processes around managing all that stuff still takes too long. If you moved to the cloud, you went there for agility, right? You want to rapidly build applications and get them out into the market as quickly as possible. But if you’ve got to fill out a ticket to get a firewall changed and it takes three days for that stuff to be processed, you’re no longer as agile as you used to be when you could twiddle a little configuration in a Terraform template, redeploy, and—poof, you’re up and running all over again. That’s something that security teams—

    Ashley: If you take another two days to answer that ticket, you’re probably gonna have a fourth after your third _____ while you’re add it to the mix, right? Some developers are gonna be like, “I’m outta here, let me go set up the next environment.”

    Dyess: Yeah, it’s too difficult. You can’t wait. You don’t want to see shadow IT any longer, because IT was taking too long to respond, but you’ve got to respect that IT’s there, the security site of IT is just really trying to help protect the business.

    But you know, you also saw Amazon and AWS, they both announced their own firewalls, right? Azure did a little bit back and AWS just did within the past month. And I think that’s really just paying homage to the fact that those are really good technologies. Firewall technologies really are good stuff and it’s more than security groups or NSGs, it goes beyond that, and the businesses really need these. These do address customer problems. So, we’re seeing a lot of folks ask about how do they adopt those new to the cloud provider, new technology, and how do they integrate them into the very agile processes? And I expect we’ll see a lot more of that.

    In fact, what we are finding that we do often is try to help organizations see how they can get the most value out of the cloud native security controls, because these cloud environments are really, they’re pretty solid. And like you pointed out, the security controls are integrated into the services. So, it makes a lot of sense to use the services, they’re there to protect your assets and they’re so integrated in the services, you really should be trying to use them, but it’s a learning curve for security, so it means a lot to try and help them get the visibility and then start to affect control over that, so that they don’t have to slow down the rest of the business, swapping out, let’s say, a cloud native control with a legacy control [Cross talk].

    Ashley: Mm-hmm. It is that tradeoff, and I think that’s one of those where it’s sort of, I think it puts security teams in a position where, are we gonna stick to our guns of this is the environment we’re gonna have consistent across all of the environments we operate in, or for sake of speed but not sacrificing security, maybe we do look at some other options and can those integrate into the monitoring and management systems as opposed to using the same firewall in every environment that we take with us there, right?

    Dyess: Yeah, yeah.

    Ashley: Easier said than done. I appreciate it’s not an easy job.

    Dyess: Right. You know, whether you decide to use the cloud native security controls or use the traditional kinda controls, the virtualized versions of those or a combination of them, one thing I think we’re sharing is, we see a lot of conversations around how—which technologies to adopt and then how to actually manage those technologies.

    And from my point of view, it seems like we’re almost too focused on getting down into the weeds about the specific security control and, as a security person, I think it makes a lot of sense to step back a little bit and try and focus on what’s the overall security policy. Instead of thinking what firewall should I use, should I use one from the cloud or should I use from my traditional vendor, maybe a better question to help organizations remain agile is for security to be able to focus on what the security policies are, what the guardrails should be, and then begin to get the practice of checking cloud environments against those guardrails, against these policies as part of your CI/CD pipeline, the whole buzzword of shift left.

    That’s a real thing, right? That’s really important. That security can focus on defining security policies where they have the strength, but the actual implementation of enforcing the security policy is probably handled by the cloud operations or whoever actually twiddles the bits on the security controls. Like, if you can marry those two things together and you can do it in the CI/CD pipeline, then that’s a huge win.

    We’ve seen organizations move a lot faster because security focuses on what they do and they get it codified in a way that can play in a pipeline, and now the app teams, they can make changes to an application or a cloud Ops can change the configuration of an environment and get an alert saying it doesn’t comply with the security policy. That’s great. To step away from choosing which sort of security device you want to use and focus on do things comply with the security policy? That’s a major shift, and one that seems to be really hard for organizations to do. I think to kinda get away from security being hands on with the control and now focused on the policy—it’s trust but verify, I think.

    Ashley: [Laughter] Back to that old saying, right?

    Dyess: Yeah.

    Ashley: You know, in a way, it also comes down—you know, we’re used to, let’s first take an inventory and make sure we know what’s all out there that we’re trying to secure.

    Dyess: Sure, of course.

    Ashley: But it’s equally just as important to have, whether it’s change management practices or software that can help you with notifying when changes occur or understanding, so when a change occurs, how does that map back to policies that we have in place and are we still in alignment, are there gaps that we have to take care of? Because the environment isn’t—it isn’t an environment we control everything in any more. It changes by itself, because other people are changing it along with us. So, it seems like it’s a more dynamic environment than maybe what we had 10 years ago, if you reflect back a bit.

    Dyess: Intentionally more dynamic, right? Businesses have to move faster, we also have to react more quickly to customer demand or competitive threats or whatever. So, the fact that we’ve got these dynamic environments is just a reflection of what we actually needed to have built, anyway. And now, we need to find a way to incorporate that back into our regular practices.

    So, you made the comment about being able to track what’s changed and when and all that, but if you adopt more of an infrastructure as code model, you already have that. You don’t need a CMDB—maybe you do, but you don’t have to have a CMDB in place when you could go back to your Git logs and see, oh, here’s the pull request for the change that we made that allowed for this security group to be open or for this Azure firewall to have this particular rule. You can actually see that information, but these are, on the security side, these are new tools. As a security person, when was the last time you looked into a Git repo to see what the configurations were? [Laughter]

    Ashley: The repo—right, GitOps and security, right?

    Dyess: Right. Right, but we’ve got to get somewhere around there, as a group collectively, whether we’re writing the code or we’re managing the operations or we’re managing the security collectively, we’ve got to be able to collaborate and use these tools that I can’t say are merging, they’ve been around a long time. Terraform and Helm, Ansible, Chef, Puppet, the whole bit—these things have been around for a while, but to incorporate them now into our security practices is a pretty big leap for folks.

    Ashley: Mm-hmm.

    Dyess: So, we really try to encourage that the security team gets the visibility that they need, that’s the first thing, as you pointed out, right? You absolutely need to understand what’s deployed, what’s talking to what and whether or not things are properly segmented.

    You’ve got to first get that visibility, but I think before you start jumping into let’s throw a bunch of other control points in there, consider what resources are there and how can you, on the security side, participate in automation, participate in these flows that your CloudOps, DevOps, SRE, whomever had set up. Because there’s a really good opportunity now on security to be part of this agile process and to allow the business to be secure, but to do so in a way that won’t break the developer productivity.

    Ashley: Mm-hmm.

    Dyess: And that’s, it’s uncomfortable, because it’s a big change, but it’s a change that I know it can be done successfully. We’ve worked with companies that have done that, you know, make the conscious decision.

    Ashley: I almost wish we had stepped back and redefined shift left, because if, you know, I were king of the world for a nanosecond, I would think about it as, it’s not just about getting security people to work with the software teams earlier in the life cycle of software—yes, it’s that. But think about it this way of, the old world was, set the policies and then how do we implement the enforcement mechanisms, right, to make sure that those are being followed.

    Dyess: Mm-hmm.

    Ashley: I think in the world we’re in now, it’s, security people are, if you were gonna set up a security practice in an organization today, what I would do is, let’s start with how software is created, let’s work with the team about how the entire tool chain is set up, how that flow happens, both for application software, but also for the infrastructure and how that gets created where we’re talking about test environments and production environments and maybe you’re stepping into infrastructure as code if it really, truly is kinda dynamic Terraform or whatever kind of environment.

    But start there, because if you understand how that’s done, now you know how to secure it. You know how to work with teams to help them secure it, then the next layer would be getting into the software architecture itself around microservices and things that are more porous than we’re used to, kinda the hard outer shell of monolith applications of the past.

    That, to me, is the mental shift, the big mental shift for a security team to not necessarily be an expert at all that stuff, but understanding that’s what you’re really securing now.

    Dyess: So, two things that stand out, like responses I can imagine people having right now and that’s, I don’t have time to go and learn all that stuff. [Laughter] I’ve already got 20 other tickets for all high priority things. I’ve gotta get on that right now, so I can’t go and learn all that stuff. Valid, that’s definitely the case in a lot of organizations, yeah.

    And another problem related to this is security teams who maybe do want to know, maybe have some time to do that, but struggle that the application teams themselves don’t even know what makes up the application and what connections do they need. So, even the folks who you think would know because they built it, you’d think they would know the answers to some of these questions, but it turns out that they don’t—in a shockingly large number of scenarios, that’s the case.

    Ashley: Mm-hmm.

    Dyess: And so, if you’re gonna shift left, you’ve got to address not just the problem of the security team getting visibility in this—or maybe, actually, it’s the same thing. The application teams have to be able to come to the table to say, “This is what the application looks like. Here’s how it’s put together. Here are the resources it needs and here are the services that will rely upon it.”

    Ashley: Mm-hmm.

    Dyess: And so, if application groups can come to the table with that, then a security team can look at it usually very quickly, look at a map and say, oh, I can see where it needs to be segmented this way. I can see where he’s a resource that has too close of an access to PII, PCI, right? And so, that conversation could start a lot sooner if it’s part of your normal bill practices, you were generating these views, you had some way to produce an item that a network team or security team could look at, right? They don’t need to look at the source code, they need to look at your network topology, they need to understand what you’re connecting to.

    So, if you can bridge that gap, that goes a long way. So, to your point, it’s not just like, “I just need to check if it doesn’t comply with the policy.” Sometimes, it’s even just trying to define the policy at the outset.

    Ashley: Yeah, on what you’re trying to secure.

    Dyess: Yeah, but also, while security never seems to get the heads up about the things that are coming down, the joke that you made at the beginning, it’s also the case that when the application team is building something that they don’t really have all the rules, they’re not sure what all the restrictions are, and there are certainly plenty of guardrails, guidelines that they could have if you give them access that they could pull from an API to do their own checks.

    So, you know, there’s a model. We’ve done this before in the industry, back when I wrote code, we had a separate QA team, so we’d write code, we’d do a little bit of testing, but just enough, frankly, to get us through the build, right? And then we throw it over the wall and it went over to QA. And then thankfully, our development practices matured to a point where we realize we actually need to integrate testing as part of what we do. So, as developers, we had to own some of that testing, some more of the testing, but also importantly, the test team got involved earlier in the application life cycle, the model you were talking about. So, the QA teams could come in and learn a bit more and begin to set up some of the additional tests and participate in the build process, so what they did showed up in NMake.

    So, I think security is on a path like that as well. I think security, instead of app teams just throwing it over the wall and asking security to figure out how to lock it all down, I think we’re going to, and in some places, some customers for sure are moving to a place where security is part of the process earlier, and if they can say, “Here’s some of the tests,” so now the guardrails, they can contribute some of that and they can go into this build process, then now you’ve really begun to get that shift left.

    But I know this can happen. We’ve seen it happen before. Now QA is just integral to what we do. You can’t even imagine putting a product out without some part of QA being part of the build process and tests happening really early on, test planning happening early on. I imagine security just going in that same sort of loop.

    Ashley: Yeah, of course Tufin’s been in this space for a while, has built up, I’m assuming, a body of knowledge about this. If you were gonna advise maybe a new customer or an existing customer on—okay, we’re always playing catch up, we’re really playing catch up now with how much more cloud services that organizations are using in such a short period of time, what are some of the sort of best practices or recommendations of, “Here’s to sort of get the first thing established, here’s the next thing to work on, here’s the next thing to work on.”

    Dyess: Yeah.

    Ashley: Any suggestions about that?

    Dyess: Sure. We have this whole crawl, walk, run methodology, and it’s designed to help organizations take these small steps towards getting to really secured environments. And the first step right away is just to get visibility, which is kind of a “no, duh” statement, but it’s actually one of the big problems that orgs have. You’ve got to be able to see what’s deployed inside your cloud, what’s deployed inside your cluster, and which resources are communicating with each other, where are the dependencies.

    So, understanding that is a really major first step. It’s true for the application teams, but for the security, it’s to know are things properly segmented, where are the biggest risks? So, getting that awareness, that’s really the crawl phase. And in the walk phase, that’s where you take what you’ve learned about what’s deployed, what talks to what, and whether or not things are segmented properly and then you begin to isolate. Here are specific accounts that we need to tackle or here are clusters that we need to tackle—like, hone in on the higher priority items and begin to define a security policy for those cloud native environments.

    And as you develop the cloud native policies, which you then share with the application teams or the cloud operators, whoever’s responsible for configuring it. As you develop that sort of, that skill, you’ve gotta get to the run phase, and the run phase is when you begin to automate this. You automate generating security policies, you automate validating security policies, but at the very beginning, where most organizations are at right now is, first get the visibility and then the second step of defining a security policy. Not defining what is my security group definition, not defining what is my AWS firewall definition, but what is my security policy? What are the rules that should govern access to applications, assets, resources of certain categories? Those are two big steps to take right away.

    Ashley: I think that’s some great advice. I’m curious, any thoughts as we—I think we all want to turn the page on 2020 and kinda get to 2021, and hopefully, we’re looking at a different kind of a year. But one of the things about 2020 is, we’re not gonna go back, I don’t think. We’re not gonna go back to, let’s rewind to January and start working in that environment again, right? Everything is changed with remote work or how much we’re using the cloud.

    What are some things that you’re anticipating for 2021 as you look forward in the next 6 to 12 months. Do you kinda think it will be topics maybe not as talked about yet but will be more topical in 2021? I’m just kinda asking you—

    Dyess: Are you asking for predictions of 2021?

    Ashley: You know, I don’t see it, but I’m pretty sure, there might be a crystal ball right behind your chair, just look back a little bit there and kinda check back. [Laughter]

    Dyess: [Laughter] Well, I think, given the maturity of the market adoption of public cloud environments, I think we’re going to see more security teams getting their arms around how to help in securing these cloud environments without necessarily always relying on the traditional devices. We’re already starting to see that even now. So, I think we’ll see more of the cloud native security control adoption. I know organizations have seen the benefits of doing that to lower cost of operations, just total lower cost of ownership, plus the benefit of being able to say when there’s a problem, I can call my cloud vendor, I don’t have to worry about vendors poking at each other. So, I think we’ll see more of that.

    I think it’s the Kubernetes space that’s probably the most interesting, because it’s the least well known for most folks on the security side.

    Ashley: Mm-hmm.

    Dyess: So, I’d say maybe over the past year, there’s been a lot of emphasis, just like with cloud, slap a firewall on it and we’ll solve the problems of micro-segmentation later, but later is gonna happen real quick, and I’d expect in 2021, you’ll see securities start to get more involved in getting visibility into the clusters and then looking to define the security policies.

    And not—again, not to try and define the policy of an individual cluster, but just the general rules, because there’s no way we can keep up with what will really be thousands of services running inside of a cluster.

    Ashley: Yeah.

    Dyess: So, we’ll see that. I think we’ll see more organizations at the security side have to deal with the scale and the rate of change and begin to participate in the automation processes that the Dev teams are using to manage it.

    Ashley: Maybe that’s something we’ll hear more frequently. I wouldn’t be surprised if the—let’s think of it as cloud native security, not just as security, sort of think about it as one of you architectural approaches to how you do security.

    Dyess: Cloud native. The native controls of the cloud provider—yeah, that’s a big shift for folks, I think.

    Ashley: It is. We’ll see if something like that happens. And I think the things you said made a lot of sense. Well, it’s been great talking with you, it’s the first chance you and I have had to talk before, and—

    Dyess: Yeah, it was good to talk to you, Mitch.

    Ashley: – a lot of fun. Tell us a little bit about where folks can learn more about Tufin.

    Dyess: Yeah, of course, you can go to Tufin.com, that’s T-U-F-I-N dot com. On the cloud side, like what’s really relevant to what we’re doing here, we’ve got a lot of resources at Tufin.com/try-securecloud. It’s cloud native security controls. It’s really designed to help you get visibility and control the cloud native security components. Great stuff there, free to use product. There’s a great trial for people to give a go. Tufin.com would be the place.

    Ashley: Great. Perfect. Thanks. Well, we wish you the best and a even better 2021, we’ll look forward to that.

    Dyess: Yeah. From your lips, man.

    Ashley: [Laughter]

    Dyess: Good luck in 2021, everybody.

    Ashley: There you go. Hang onto that, hang onto that. Alright, well, take care. It was very nice talking with you, Colby—Colby Dyess from Tufin.

  • Cloud Misconfigurations Threaten Cloud Migration

    Cloud Misconfigurations Threaten Cloud Migration

    If we’ve learned one thing during the COVID-19 pandemic, it’s that digital-native business models are essential to survival. That means cloud adoption is more important than ever before. Research shows that the public cloud market is expected to grow to $308.5 billion in 2021, an increase of 16% from 2020.

    But while the cloud holds incredible potential, we have an outstanding security issue to address. That issue is cloud misconfigurations. Through 2025, 99% of cloud security failures will be the responsibility of the customer, quantifying the need for security professionals to turn their attention to security hygiene issues like cloud misconfigurations.

    Digging Into the Cloud Misconfiguration Problem

    To get to the root of the problem, we must fully understand the risk of cloud misconfigurations and how they differ from more traditional vulnerabilities. While traditional vulnerabilities can be patched, cloud misconfigurations can create vulnerabilities in otherwise secure applications and infrastructure. Imagine cloud infrastructure like a highway system and cloud misconfigurations as road hazards. They can be incredibly dangerous and lead to accidents.

    As a specific example of cloud misconfigurations, let’s look at identity and access management (IAM). Poorly configured IAM, such as misconfigured roles or a lack of multi-factor authentication (MFA), can lead to compromised administrative accounts. If a threat actor hijacks a legitimate administrator account, they potentially can take full control of an entire cloud environment. Because IAM sits above the cloud infrastructure layer and all workloads and data within, once it is exploited a threat actor will often be able to circumvent your other security barriers, such as network segmentation, leaving you defenseless.

    Cloud Misconfigurations Can Easily Scale

    Where misconfiguration risk dramatically increases is through the adoption of cloud-native applications and practices like infrastructure as code (IaC) templates used by DevOps teams. These templates offer teams greater speed and scale for building and managing applications, but the downside is that misconfigurations can be unknowingly replicated from development environments to production environments (where sensitive data is stored) at greater velocity.

    In short, the biggest risk to organizations right now is scaling these misconfigurations through the cloud. As the cloud grows in adoption and scale, we’re witnessing these misconfigurations scale right alongside it. Where before, a misconfiguration might have been limited to a siloed application or environment, today, that same misconfiguration can impact the entire organization without checks in place to catch it. Worse, many cloud storage buckets have logging disabled, meaning once threat actors are able to identify a misconfiguration and access an internal cloud bucket, an organization won’t even be able to see what data was accessed.

    Understanding the Shared Responsibility Model

    This all points to a higher-level, hygiene-related issue to keep in mind when securing cloud environments: responsibility. Particularly, organizations struggle to understand the shared responsibility model and how it applies within their own organizations. While some may fail to delineate between cloud provider responsibilities and their own, the major issue at stake is the shared responsibility between various internal teams that often goes undiscussed and undocumented until there is a security event.

    When thinking about software-as-a-service (SaaS), to a much lesser degree the responsibility falls on consumers. However, when considering infrastructure- and platform-as-a-service (IaaS/PaaS) and all the moving parts such as network, user credentials, resource configurations, workloads, identity configurations and more, cloud consumers become responsible for much more. A key consideration to note is that one can never outsource accountability, no matter which cloud model is used. To put this more simply, if one puts data in any cloud provider, they are still accountable for that data.

    Enabling a Holistic Cloud Security Strategy

    When advising security and business leaders on how they can better secure their cloud environments and applications, I highlight what my team calls “The Big Cloud 5,” a set of recommendations developed to help organizations adopt a holistic cloud security model that accounts for proper security hygiene and shared responsibility.

    • Gain awareness and deep cloud visibility
      The very first step to ensuring cloud security is understanding how teams are using cloud technologies, leveraging shadow IT and cloud provider APIs. This allows you to get situational awareness and make informed decisions today as well as in the future. This is not a one-time event, but something you’ll need to do continuously.
    • Set guardrails to automatically prevent the most serious cloud misconfigurations
      Drawing lines in the sand around the most offensive (and potentially destructive) misconfigurations that should never exist in an environment is key to automating protection in the cloud. This will help keep templates and practices controlled, so that poor hygiene doesn’t inadvertently take root and spread. Think of this as your “dirty dozen.” What configurations should never exist in your cloud environments?
    • Standards are the precursor to automation
      One can’t automate what hasn’t been standardized, and while there aren’t widely accepted security standards yet, key stakeholders in an organization must be in agreement about how to secure cloud infrastructures.
    • Train and hire security engineers who code
      To fully leverage APIs, security teams must have engineers who know how to code and automate security processes. An assessment of skills that exist across your security team (e.g., knowledge of coding in the likes of Python or Ruby) can point to areas in training and hiring that need further investment.
    • Embed security in the development pipeline
      Map out who, what, when, where and how your organization pushes code into the cloud. Once mapped, identify the least disruptive insertion points for security processes and tools, so that they can exist in as much of the development pipeline as possible.

    If you feel your organization is a step behind on proper cloud security hygiene, know that you’re not alone. Earlier in 2021, the Cybersecurity and Infrastructure Security Agency (CISA) released a report highlighting the importance of strengthening security configurations based on the uptick in successful cloud attacks, often attributed to poor hygiene and the mixed use of computing devices in the remote work environment. While this is a growing problem, it is also a reversible trend. Best practices, cyber hygiene and a shared responsibility model can help companies safely and securely migrate to the cloud.

  • Rewind Acquires BackHub to Protect GitHub Repositories

    Rewind Acquires BackHub to Protect GitHub Repositories

    Rewind, a provider of a software-as-a-service (SaaS) platform for backing up cloud services, today announced it has acquired BackHub to gain control of a platform for backing up GitHub repositories.

    Based in Germany, BackHub adds a data protection platform to a Rewind portfolio that includes offerings for backing up everything from QuickBooks to Shopify. Fresh from raising an additional $15 million in funding, Rewind CEO Mike Potter said BackHub addresses a critical issue involving the protection of some of the most valuable intellectual property any organization has in a cloud service. GitHub, a subsidiary of Microsoft, is the largest host of source code in the world, with over 40 million users and 190 million repositories. BackHub claims it protects data in those repositories for more than 1,000 organizations.

    Rewind GitHub backup

    While GitHub itself is a reliable cloud service, the code residing in GitHub is subject to the same potential threats as it would be in any other cloud service. It’s not uncommon for a malicious actor to gain access to credentials that they exploit, for one reason or another, to delete or alter data – or, in the case of GitHub, code. BackHub makes it possible to back up code stored in GitHub to another cloud service where it can be easily recovered.

    Potter said the ultimate goal for Rewind is to make it possible to back up the entire cloud. Achieving that goal will require both additional acquisitions and internal product development, said Potter.

    In the meantime, the company is focused on providing data protection for specialized cloud services usually overlooked by other backup and recovery tool providers. For now, it leaves the backup and recovery of data stored on widely used cloud services, such as Microsoft 365, to other vendors. Rewind claims more than 80,000 organizations are using its platform to protect data, stored in a wide range of cloud services, that address a specific niche use case.

    Given the prominent role GitHub plays in software supply chains, it is not surprising that the code stored in Git repositories becomes a target for cybercriminals. Much of the intellectual property stored in those repositories is a high-value target for organizations looking to reduce the amount of time required to re-engineer a rival company’s functionality.

    Nation-states around the world have cyber-espionage programs aimed at gathering any type of digital asset that might benefit their military or economy. Naturally, copying code isn’t always enough when, in one fell swoop, add code that might derail that project long enough for them to catch up.

    Unfortunately, there have been some high-profile breaches involving software supply chains. The best way to determine if a project has been compromised is to compare code bases. Repositories, by definition, provide application development teams with version control capabilities. However, backing up a GitHub repository provides one extra layer of security that can make it easier to recover from a breach.