devopsy.kinsta.cloud

Category: Cloud Management

  • CloudBees Acquires Neuralprints to Shift Compliance Left

    CloudBees Acquires Neuralprints to Shift Compliance Left

    At the online DevOps World 2021 conference today, CloudBees revealed it has acquired Neuralprints to provide the core technology for CloudBees Compliance, a real-time compliance and risk analysis platform that it will roll out in the first quarter of 2022.

    At the same time, CloudBees has enhanced the feature management capabilities of its namesake continuous integration/continuous delivery (CI/CD) platform to provide greater visibility into and control over feature flags that are now widely used to add new capabilities to applications.

    Prakash Sethuraman, chief information security officer (CISO) for CloudBees, said CloudBees Compliance will make it possible for DevOps teams to continuously enforce compliance policies as part of an effort to better secure their software supply chains. Those DevOps teams will be able to enforce compliance across code, binary artifacts, data, identity and infrastructure environments in a way that provides developers and IT operations teams with instant actionable feedback to enable issues to be addressed long before an application is deployed in a production environment.

    CloudBees Compliance is based on a mix of open source and proprietary technologies developed by Neuralprints and will be made available for on-premises IT environments as well as via a software-as-a-service (SaaS) application. The goal is to enable organizations to manage compliance as code as part of any effort to shift more responsibility for application security further left toward DevOps teams, said Sethuraman.

    CloudBees Compliance

    A survey of 500 C-level executives published today found nearly half (45%) admit that initiatives to secure their software supply chains are halfway or less-than-halfway complete. Nevertheless, 95% claimed their software supply chains are secure (95%), with more than half (55%) posting they are very secure.

    A full 93% said they are prepared to deal with an issue such as ransomware or a cyberattack on their supply chain. However, nearly two-thirds (64%) admitted they are not sure who they would turn to first if their supply chain was attacked; with 58% admitting that, if they experienced an attack, they have no idea what their company would do. An equal percentage (64%) said it would take more than four days to fix the problem if they did experience an issue.

    Almost all C-level executives (95%) said they think more about securing the supply chain now than they did just two years ago, and 92% said a security issue would impact their brand. A total of 83% said security issues cause their developers to drop everything to review code, with 82% noting that caused the organization to spend less time on innovation.

    On the plus side, 95% of executives said container images are checked for high or critical vulnerabilities, while an equal number said automation access keys are set to expire automatically. A total of 92% said their company only accepts commits signed with a developer GPG key, while 90% said dependencies to trusted registries are limited at their organization, while 89% said administrative access to CI/CD tools is restricted (89%).

    Of course, what C-level executives believe and what actually occurs inside an organization are not always one and the same. A series of high-profile software supply chain breaches resulted in a number of organizations reviewing their software supply chains. At the same time, the Biden administration has issued an executive order requiring all federal agencies to conduct similar reviews.

    One way or another, most organizations will soon find themselves embracing a wider range of DevSecOps best practices to ensure the integrity of software supply chains no matter how secure they may think their software development platforms are today.

  • Avoiding Budget Overruns During Cloud Migration

    Avoiding Budget Overruns During Cloud Migration

    Many organizations have already started their migrations to the cloud, or soon plan to. It makes sense: The cloud is an attractive option for multiple reasons, including its potential for cost savings, scalability and support of continuous application development.

    However, benefits aside, it can be very easy for enterprises to lose financial control of a cloud migration. The potential for costly mistakes when shifting to the cloud is significant, and organizations often end up getting hit with surprising bills that ultimately stop them in their tracks. In fact, in a 2020 survey, 94% of participants claim that it is a challenge for them to take full advantage of public cloud resources, with 40% citing the need to control cloud costs as their biggest obstacle.

    These financial mistakes are typically made due to the pay-per-use model of cloud services and the growing cloud skills gap. Too often, IT team members are forced to work in cloud environments they are not proficient in, and as new features are released every week, it’s almost impossible for these individuals to keep up. One recent study claimed that 90% of enterprises found that their cloud skills gap doubled over a three-year period.

    The Two Different Types of Cloud Migrations

    There are two types of cloud migrations that typically occur, a lift and shift, or a replatform/rearchitecture migration. The lift and shift method is fast and relatively easy, but it doesn’t enable organizations to truly optimize their cloud usage. The rearchitecture method is the most popular as it gives an organization the opportunity to fix any outstanding issues by harnessing the benefits of cloud-native technologies and services.

    However, once a rearchitecting migration begins, organizations often find it takes significantly more time, effort and resources to complete than they originally thought. This can become a real challenge, especially when facing critical deadlines or experiencing major events like the COVID-19 pandemic or the wild weather events experienced around the globe.

    Understanding and Assessing Performance

    For both types of cloud migrations, understanding and assessing performance and underlying architecture is an ongoing task, as cloud providers are always releasing new or improved services or lowering prices. To make sure they’re achieving the best performance and building a resilient architecture, application developers and IT teams must always be looking for ways to improve their applications. Within the cloud, this is typically focused on four key areas: Networking, computing, storage and application architecture.

    From a network perspective, understanding where data is being used and where data is being accessed is critical and can help organizations cut costs exponentially. For example, cloud providers often have costly data ingress and egress charges, so assessing how best to access and use data when necessary can help a company navigate those charges.

    Regarding computing, there are multiple resources that can help manage system load, make tweaks to computing infrastructure and find performance and cost benefits. For instance, leveraging the data and metrics from native cloud reporting tools can help organizations understand and predict demand, and taking advantage of services like auto-scaling enables them to scale up or down appropriately depending on the changes in application use and demand.

    On the storage front, cloud providers offer a number of ways to save money and boost performance. From the release of GP3 volumes to tiered long-term storage options based on the frequency of access, storage is an easy way to find cloud savings opportunities. Additionally, cloud providers will usually make recommendations and set up tiering policies to help move data automatically into colder and deeper storage.

    As cloud providers continue adding new services, looking at your application architecture is essential to innovation and cost savings—especially when including labor and effort—and truly achieving the promise of the cloud.

    Solving Common Costly Mistakes

    While there are numerous examples of common financial mistakes companies make when managing cloud instances, lack of visibility and understanding are the leading pain points for the developers who stand up and maintain their organization’s architecture. With the demand for cloud so high and with their attention directed elsewhere, developers and engineers often take the mentality of “If it isn’t broke, don’t fix it.” This hampers the company’s ability to assess and optimize their cloud operations. They don’t have time to consistently review their applications for ways to save money or even know how much their decisions cost.

    This ubiquitous issue can be solved by bringing greater visibility into the cloud migration and management process and aligning real-time finances directly with established corporate budgets. In addition, organizations struggling with the cloud services’ pay-per-use model should take the necessary steps—whether internally or with a third-party vendor—to ensure cloud resource and service use is being effectively monitored. This makes it easier for developers to understand where and how their actions affect the financial posture of the organization before their boss comes knocking on the door.

    If enterprises provide the resources necessary for IT teams to become more knowledgeable about how their applications are being used within the cloud and address the cloud skills gap in their workforce, they will be able to prevent cost overruns, experience cost savings and take back control of their budgets. This can take a variety of forms, but enterprises need to establish continual training programs to ensure team members are up-to-date on the latest cloud features and services, as well as leverage tools and automation to reduce manual intervention. Providing this critical visibility throughout a cloud migration can often mean the difference between a successful, on-budget migration and one that is stalled and costly.

  • Google Donates Sqlcommenter to OpenTelemetry Project

    Google Donates Sqlcommenter to OpenTelemetry Project

    Google announced today that Sqlcommenter, an open source project that provides access to an object-relational mapping (ORM) auto-instrumentation library, is being merged with OpenTelemetry, an open source project for building agent software to instrument applications.

    Nimesh Bhagat, a product manager for Google, said Sqlcommenter is designed to make it simpler to correlate application and database telemetry. OpenTelemetry provides a mechanism for collecting logs, metrics and traces that is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The challenge organizations using OpenTelemetry to collect that data will face is the need for tools to query that data in whatever platform it ultimately resides, noted Bhagat. OpenTelemetry previously lacked a common standard by which application tags and traces can be sent to databases and correlated with an application stack. Google envisions that IT teams will either employ Sqlcommenter to explore data or access via platforms such as Cloud SQL Insights or a third-party application performance management (APM) platform.

    Google made Sqlcommenter available as open source software earlier this year. It is designed to enable object–relational mappings (ORMs) to augment SQL statements before execution by, among other things, providing access to comments containing information about the application code that caused its execution.

    Sqlcommenter also allows OpenTelemetry trace context information to be propagated to the database, enabling correlation between application traces and database query plans that ultimately should make it easier to, for example, determine the root cause of a performance issue involving a microservices-based application making calls to a database. Sqlcommenter is primarily designed to be employed by full-stack developers that have some familiarity with how to use SQL to query data residing in an observability platform, noted Bhagat.

    Specifically, Sqlcommenter is designed to provide deeper insights into the performance of backend databases by simplifying the process of correlating slow queries with source code. It currently supports Python, Java, Node.js and Ruby languages along with ORMs such as Django, Sqlalchemy, Hibernate, Knex, Sequelize and Rails.

    As the number of applications that are instrumented using open source software continues to steadily increase, the amount of data that a DevOps team could potentially query also will steadily increase. Cloud storage platforms have made it easier to store that data at an affordable cost but the next challenge is clearly going to be finding a way to surface meaningful insights from all that data. Undoubtedly, achieving that goal will require everything from query tools to machine learning algorithms capable of identifying patterns at a level of scale that would not otherwise be possible.

    In the meantime, DevOps teams can look forward to being able to attain a level of observability that previously could only be achieved at a cost that tended to limit the number of applications organizations would instrument. At a time when the dependencies between applications increases with each passing day, that level of observability will soon prove indispensable.

  • 4 Ways to Simplify Cloud Performance Monitoring

    4 Ways to Simplify Cloud Performance Monitoring

    Over the past 18 months, technologists across the globe have played a crucial role in the survival of brands and organizations via digital strategies. Across all industries, their skill and dedication have enabled businesses to rapidly launch innovative new digital services to meet huge fluctuations in customer needs and allow entire workforces to operate from home during the pandemic.

    This hasn’t been easy, even as end users’ expectations for seamless digital experiences are on the rise. In fact, the recent 2021 App Attention Index reaffirmed found 76% of consumers’ expectations of digital services have increased in the last year. And an overwhelming 85% noted that applications and digital services have become a critical part of their daily lives. This has placed even greater pressure on the technologists who are responsible for keeping applications and cloud experiences running smoothly.

    Often IT teams are challenged with monitoring companies’ digital services, but they lack the visibility they need to monitor health and performance across an increasingly sprawling IT estate, exacerbated by a seismic shift toward cloud architectures over the last 18 months. IT departments don’t yet have the right tools and data to optimize health and performance in their cloud environments.

    As a result, technologists are constantly addressing the same issues because they can’t easily isolate and identify them, and don’t know where to prioritize their actions. Hiccups can lead to brands losing customers—the same report revealed that over half of consumers indicated that brands have one shot to impress them, and that if their digital service does not perform, they won’t use them again.

    After a year and a half since the start of the pandemic and with adoption of cloud technologies set to accelerate even further over the next few years, it’s time for technologists to get ahead. Here are four things technologists can do to take the stress out of managing performance across the cloud.

    Implement Visibility from the Ground Up

    Having visibility into applications and the underlying cloud infrastructures that support them can be challenging without the right tools, which has contributed to some of the pressure technologists are experiencing. Traditional monitoring has allowed technologists to identify and catch issues in certain areas of an IT stack, but this approach is of little use in dynamic, distributed and software-defined environments where organizations are continually scaling their use of IT and the cloud up and down.

    As a result, the performance of the digital services can be impacted, potentially hurting end-user experience and eroding trust in the brand.

    To keep up with the rising demands from consumers in the digital world, technologists need the ability to monitor the full IT estate, from cloud infrastructures to customer-facing applications through to core infrastructure such as compute, storage and network; including the open web and security. With access to tools like observability platforms, technologists can lessen the burden of individually monitoring the performance of each item supporting the digital service. Instead, they can achieve one holistic view into alerting, root cause and analysis of correlated metrics, events, logging and tracing (MELT) data from the cloud to enable early and easy troubleshooting.

    Consider tools that link infrastructure (compute, storage and network) with the services and applications they execute. When there is an application-level issue, it is easy to take a closer look and identify problematic areas in what makes the application run or the application itself.

    Align IT Performance to Business Results

    After aggregating a full-stack view of applications and the cloud infrastructure behind them, technologists can then link performance data identified by the observability platform with real-time business metrics.

    Linking the two provides technologists with the ability to identify where and how the digital services are impacting the business and end-users. Full-stack observability automatically provides technologists with these linked insights, enabling them to address the areas that matter most.

    Educate Teams

    In addition to having the right tools to help take advantage of the cloud, it is important that an IT team is well-versed in the technology itself. Managing the performance of the cloud is complicated and there is a finite pool of talent available to businesses. Therefore, business leaders should provide appropriate resources to ensure teams are educated and equipped to succeed.

    To support a seamless implementation of cloud infrastructure to support a business’ digital services, business leaders must invest in upskilling their teams or recruit others who are knowledgeable about cloud monitoring.

    This will ease the pressure on existing teams and give technologists the skills and confidence to progress in their careers.

    Leverage Automation

    Even with the best observability solutions in place and teams equipped with the necessary skills, there is still concern that there will be an overwhelming amount of data generated by accelerated digital transformation over the coming years, especially those involved in large-scale public cloud environments. It can become a challenge for technologists to dedicate the level of resources and skills required to manage and optimize performance across the IT estate.

    To avoid these potential circumstances in the future, it’s worth considering how to minimize the impact of a data deluge by reducing an IT department’s reliance on manual interventions to monitor health and performance and resolve issues.

    Two things that will play an increasingly vital role in tracking health and performance across the entire IT estate are automation and AI. This is particularly the case in large-scale public cloud environments, which are fast becoming the biggest headache for some IT teams.

    With AI deployment to identify and fix issues in real-time, technologists can spend less time firefighting thanks to the AI/ML guiding hand. This will lead to increased job satisfaction and fulfillment and a better work-life balance.

  • Progress Expands Chef Automation Portfolio

    Progress Expands Chef Automation Portfolio

    In advance of its ChefCon 2021 conference, Progress today unveiled a series of enhancements to its Progress Chef automation platform that includes managed services and a software-as-a-service (SaaS) edition of its offerings as well as tighter integration with cloud services and other third-party tools.

    Sudhir Reddy, vice president of engineering for the Chef business at Progress, said the SaaS platform, now available in beta, combined with the managed services provided by Progress will make the automation platform much more accessible to a wider range of organizations.

    Progress is now also extending the reach of its Chef Compliance Automation for Cloud Resources offering across Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) to include Chef premium content that aligns with best practices for each cloud environment in addition to providing integration with Azure Policy, a suite of tools from Microsoft that compares resources to a set of business rules that define how cloud infrastructure should be employed.

    Chef

    Finally, Progress has enhanced Chef Enterprise Automation Stack to provide tighter integration between compliance and infrastructure automation along with adding infrastructure operations dashboard, support for Windows patch management, integration with third-party secrets management tools, and improvements to Chef Workstation and Chef Test Kitchen to simplify testing within application development processes.

    With the release of a SaaS platform and the availability of managed services, Chef is joining a long list of IT vendors that are removing responsibility for deploying and securing their platforms from internal IT operations teams. It’s clear more organizations now want to spend more time on building and deploying software than managing the platforms that automate those tasks on their behalf, said Reddy.

    It’s not clear how many IT teams that have already deployed a platform might shift over to a SaaS platform or rely on an IT vendor to manage that platform on their behalf. However, in the wake of the COVID-19 pandemic, many IT teams continue to work from home to help combat the spread of the virus. Many of those organizations have either shifted platforms to the cloud themselves, adopted SaaS platforms or outsourced the management of IT platforms altogether. There are also many small organizations that simply don’t have the internal expertise and resources required to stand up an IT automation platform on their own.

    That approach also makes it simpler to converge security, compliance and IT operations management within the context of a larger DevSecOps workflow, noted Reddy.

    The great paradox of IT automation has always been the level of expertise required to achieve it. Organizations that lack the expertise required to create automated playbooks, for example, don’t derive much value from an IT automation platform. It’s only been larger companies that have mostly derived the most benefit from investments in IT automation. However, as IT automation becomes more accessible via SaaS platforms and managed services, the level of expertise required to benefit from it naturally drops. In fact, with the advent of artificial intelligence (AI) for IT operations (AIOps), many manual IT tasks are increasingly being automated.

    It’s too early to say whether the IT industry as a whole is on the cusp of the next wave of automation, but the raw ingredients that make up the primordial soup needed to achieve that goal is, at this point, only awaiting a catalyst.

  • SRE’s Guide to Pragmatic Incident Response

    SRE’s Guide to Pragmatic Incident Response

    In my past experience as an SRE, I learned some valuable lessons about how to respond to and learn from incidents. If you want the TL;DR, I’ll summarize them here:

    • Declare and run retros for the small incidents. It’s less stressful, and action items become much more actionable.
    • Decrease the time it takes to analyze an incident. You’ll remember more, and will learn more from the incident.
    • Alert on pain felt by people – not machines. The only reason we declare incidents at all is because of the people on the other side of those machines.

    Now, let’s dive into each of these lessons a little deeper, and explore how they can help you build a better system for pragmatic incident response.

    1. Focus on the Small Incidents First

    The bad habit of ignoring small issues often leads to bigger issues. You should run retrospectives for small incidents (slowdowns, minor bugs, etc.) because these often have the most actionable takeaways—instead of shooting the moon and creating “rearchitect async pipeline” as a Jira ticket that never happens. Focusing on low-stakes incidents and retrospectives are a great introduction to behavior change across your organization.

    Let’s look at an obvious example of how a focus on small incidents can have an outsized impact. Case in point—my apartment. I live in an old candy factory retrofitted into apartment units. We have an elevator (thank God), but some of the buttons don’t light up when you press them. The LED display doesn’t match up with the numbers on the floor buttons. Yes, the elevator goes up and down, but overall, you can tell that things are wrong with it.

    One day, I came home and noticed an “Out of Order” notice on the elevator door. Was I surprised? Not at all. Of course an elevator with mislabeled buttons and broken LEDs would stop working!

    This isn’t dissimilar from an important software lesson—ignoring small issues often leads to much bigger issues. What starts as an inconvenience can lead to a complete breakdown. This is why you should focus on the small incidents first. I hear a lot of companies say, “We need to fix incident response! Every time we have an incident, it’s just chaos!”

    They are talking about unexpected downtime during a high severity level incident and they want to improve it.

    I say that’s the wrong type of incident to use to fix your incident response system and process. Focus on the small incidents first. Running retros for small incidents can help you build strong incident response models, because they have the most actionable takeaways and they are the best way to change behavior through repetition and practice.

    If you have a high-stakes, 12-hour incident and you run a one-hour retrospective you’re not going to get the results you want. You need to start small. Run retros for bugs that were introduced; maybe a bad data migration that didn’t really impact anything but took up a couple of hours of your day.

    Heidi Waterhouse captured this idea really well in this piece on reliability. Every airplane you’ve ever flown on has many tiny problems, Waterhouse said, “… like a sticky luggage latch or a broken seat or a frayed seatbelt. None of these problems alone are cause to ground the plane. However, if enough small problems compound, the plane may no longer meet the requirements for passenger airworthiness and the airline will ground it. A plane with many malfunctioning call buttons may also be poorly maintained in other ways, like faulty checking for turbine blade microfractures or landing gear behavior.”

    I couldn’t agree more. Extrapolating to software: The small things are typically indicators of bigger issues and could cause catastrophes down the road.

    2. Track Mean Time to Retro (MTTR)

    It’s important to think about what you measure in your organization. You should be measuring how you’re improving, and the most important metric here is mean time to retro (MTTR). Everyone should be tracking MTTR. It’s a great statistic to improve incident response in your team because it helps you understand the delay between incidents and retrospectives.

    The easiest way to have a bad incident retro is to wait two weeks. It’s better to get into a room quickly and hash out what happened than wait a long time until you’ve got everything perfectly prepared.

    Tracking MTTR can help you hold prompt and consistent retrospectives after incidents. Set a timer and make an SLO or SLA for yourself that says, “This is how long we take for retros.”

    Retro time will vary depending on the severity of the incident itself. If it’s a SEV1, clear everyone’s schedules, because you need to have retro within 24 hours of the incident. For SEV3 incidents, you have much more leniency.

    I also like tracking the ratio of retros to declared incidents. This is a metric that should go up; you should see your ratio of retrospectives to incidents increasing. You can break down that number by severity, as well. If you have a low retro ratio for a SEV1 versus a SEV3 incident, that might be okay at first (remember, start small) but you want those metrics to eventually become equal.

    3. Alert on Degraded Experience with the Service, and Not Much Else

    The severity of incidents is directly linked to customer pain. We would not declare SEV1s if there weren’t a lot of people feeling a lot of pain.

    Alerting on computer vitals is an easy way to create alert fatigue and burnout. As your company starts to scale, you are going to use more CPU; you’re going to use more memory. Tying alerts to computer vitals is not a good sign.

    If I run, my heart will beat faster—it’s just doing its job. Paging people at 2:00 a.m. because disc capacity is at 80% even though you won’t run out of space until next month is a good way to lose great teammates. I have worked with people who left their previous companies strictly because they got paged too many times for stuff that didn’t matter.

    This is why you need to alert on a degraded experience with the service and not much else. A CPU burning hot at 90% is not necessarily a bad thing, but you need context to decide. Create SLOs that are tied to customer experience and alert on those. People experiencing problems with the service is the only thing you should alert on—for the most part.

    One of the best ways I’ve seen to think about this came from Soundcloud developers, who explained that you should alert on symptoms, not causes. My fast heartbeat is not necessarily a problem. But if my elevated heart rate leads to lightheadedness and I fall—that’s a problem. So, I need to be able to alert on something like that. Paging people at 2:00 a.m. because disc capacity is at 80% and you won’t run out until next month is not good. But paging people because you know that disc capacity problems cascade into other, systemic problems is. You can apply the same thought to other potential causes of an outage. Paging alerts that wake you up in the night should only be based on symptoms.

  • How to Improve Cloud Management

    How to Improve Cloud Management

    Cloud technology underpins emerging technologies like artificial intelligence (AI), IoT, edge computing and containerization. Organizations must prioritize implementing effective cloud management practices to avoid spiraling costs and potential security and compliance violations.

    The COVID-19 pandemic has vastly accelerated cloud adoption. To support remote work, many businesses have moved workloads from on-premises servers to the public cloud. At the same time, thanks to the proliferation of technologies like AI and machine learning, organizations are generating more data than ever, much of which is also being stored in the cloud. It’s no surprise then, that the explosion in demand for cloud services is showing no sign of slowing down. Gartner predicts that worldwide end-user spending on public cloud services is forecast to grow by 23.1% in 2021 to total $332.3 billion.

    But, it’s not all smooth sailing. As the average organization’s cloud infrastructure grows in complexity and size, it becomes increasingly difficult for teams to manage it effectively. This has significant security, compliance and cost implications. According to Accenture, an astronomical 30% of organizations’ cloud spend is currently wasted.

    So, what can you do to ensure you avoid the most common cloud management pitfalls?

    See the Big Picture

    It’s surprising just how little visibility many businesses—even those with large teams of cloud architects—have into their cloud infrastructure. For instance, one recent survey found that over 25% of IT departments have no idea what their monthly cloud spend actually is.

    Before taking steps to optimize your cloud management practices, it’s crucial that you gain full visibility into your current situation. You need to understand your cost history, audit trail and CPU utilization patterns to identify where you’re overspending.

    Plus, make sure you’re using tagging consistently. A well-defined organization-wide tagging policy is essential, enabling you to accurately visualize expenditure by category. It’s also good practice to set up alerts which are triggered when a user launches instances without tags. This prevents any potential oversights.

    Develop a Change Management Process

    Gone are the days of rigid change management processes handled solely by IT teams. Nowadays, business units can acquire cloud services without even involving IT.

    In the cloud, change management is a series of continuous integrations, deliveries and deployments. Given this fast-moving environment, it’s crucial to implement a robust change management process or risk spiraling costs and potential compliance violations.

    It’s important that your change management process gives you visibility into and control of each infrastructure change without compromising on delivery speed. To achieve this while establishing strong policies and rules, it’s worth considering automation for at least part of the application delivery process. By auto-approving all routine and low-risk changes, you can free up time for your developers to focus on more complex projects.

    In addition, it’s worth integrating your change management process with your DevOps tool stack. You can use collaboration tools like Slack, Teams and/or Jira to ensure everyone has visibility into changes and resolve any issues quickly.

    Continuously Optimize your Cloud Infrastructure

    Unfortunately, when it comes to effective cloud management, your work is never truly done. Conducting a one-off review of your existing cloud infrastructure will only get you so far.

    Many businesses make this mistake and find themselves falling back into their old bad habits a few months down the line. To keep your cloud infrastructure in check, it’s best to continuously track a whole host of factors, including costs, changes, users, permissions and compliance breaches. After all, it’s much easier to solve an issue when you are able to spot it before it gets out of control.

    Now you have an idea of where to start when it comes to improving your cloud management practices. However, manually juggling all of the above considerations is a significant undertaking. It’s prohibitively time-consuming for many organizations whose DevOps teams are already overstretched. This is why a whole host of SaaS-based cloud management tools have sprung up in recent years. These offer unparalleled visibility into changes, resource utilization, costs, security, compliance, performance and more. Yes, they come at a cost. But, most businesses find that the savings such tools enable them to identify significantly exceed any outlays.

  • Majority of Organizations Now Adopt Multi-Cloud

    Majority of Organizations Now Adopt Multi-Cloud

    As IT’s digital footprint grows, its cloud footprint is rising both in size and complexity. As a result, most organizations have matured into multi-cloud operators. A multi-cloud approach empowers different divisions with the autonomy to select the right computing environment for their needs. It also enables a lift-and-shift ability to port workloads across cloud service providers (CSPs) when needed. Adopting a multi-cloud model could bring performance benefits, cost savings or better scalability, all while encouraging a more vendor-neutral stance.

    We’re in a multi-cloud world. A recent HashiCorp study found three out of every four organizations are already multi-cloud. This is a staggering increase from just a few years ago when mono-cloud was more accepted, and operators had zombie-like allegiance to AWS. The uptick in multi-cloud signals the maturity of offerings at other CSPs, like Azure and GCP. It also signals differentiation among clouds as each hones in their unique value propositions.

    HashiCorp’s inaugural State of Cloud Strategy Survey offers some very clear results about the state of multi-cloud adoption. Below, I’ll review major takeaways from the report to better understand the state of multi-cloud adoption. To sum it up, large organizations tend to benefit most from multi-cloud adoption, as there are still significant roadblocks to truly reaping a sizable return from a multi-cloud status. Yet, more infrastructure automation may help companies of all sizes respond to multi-cloud complexities.

    The Multi-Cloud Era

    DevOps truly operates in a multi-cloud era. The report revealed 76% of organizations are already multi-cloud; that leaves only one in four companies leveraging a single cloud. And, that trend won’t stop—within two years, 86% predict they will be multi-cloud operators. The study also confirms that the larger a company is, the more likely it will adopt multi-cloud—94% of large enterprises with 5,000 or more employees are multi-cloud.

    So, what’s driving this cloud fragmentation? Well, 34% of respondents cite digital transformation as a top driver for multi-cloud adoption. In the race to meet new digital innovation demands, multi-cloud has emerged as a strategic force. To enact this rapid change, many organizations are investing in a cloud center of excellence (CCoE) to guide cloud transformation efforts. A CCoE is most common within large companies with a cloud budget of $5 million or more.

    Speaking of budgets, cloud spending is going through the roof. In 2020, only half of companies met their projected cloud spend, and 39% of companies overspent their allotted cloud budgets. As I’ve covered before, cloud-native costs are notoriously difficult to identify and trim. Without the proper guardrails to manage spending, surprise data egress fees or hidden server costs can strain the cloud checkbook. As it stands now, 40% of companies lie in the $100,000 to $2 million per year range, and 6% of enterprises are investing $50 million a year or more into multi-cloud. This spending will likely increase in the coming years as cloud reliance expands.

    Enterprises Benefit Most

    So, which clouds are companies using? If we compare cloud adoption, AWS is a clear market leader; 88% of companies use AWS, 74% use Azure and 61% use GCP. AWS’s hegemony is not expected to fade in the next two years. Yet, while AWS usage will remain the same, competing CSPs will experience modest increases. This is likely due to Azure and GCP making gains in geographical zones outside North America, such as Asia-Pacific and Africa.

    In terms of technology used in cloud environments, open source software (OSS) is a popular choice. Cloud-native open source projects, such as Kubernetes, OpenStack and Terraform, have taken on a life of their own, with vibrant community support. However, not all shops build upon OSS and run it themselves. Somewhat ironically, we’re seeing more managed OSS emerge. One stand-out area is provisioning, where 42% of organizations build with open source and run it themselves.

    On the whole, multi-cloud is working at most enterprises, with 53% of survey respondents saying multi-cloud has helped achieve their business goals. The top benefit of multi-cloud is digital transformation, at 34%. This is followed by avoiding lock-in (30%), reducing cost (28%) and scalability (25%). But multi-cloud does not treat all companies as equals; the likelihood of multi-cloud success increases as the company size increases. Less than half (42%) of small companies say they have multi-cloud success, which jumps to 64% for large enterprises. With fewer resources for support, multi-cloud may be less beneficial for startups and small to medium businesses. In fact, 41% of professionals say a lack of in-house skills inhibits their cloud success.

    Challenges in a Hybrid Multi-Cloud World

    An ongoing talent gap is just one potential challenge to cloud success. Just over half (51%) say cost concerns are a top inhibitor to their cloud program success. Hybrid cloud complexity is another prominent challenge, with 35% of organizations citing it as a major cloud obstacle. Many enterprises are still grappling with the remnants of legacy on-premises estates. But technical debt is not the only reason for hybrid modalities. Surprisingly, large platforms even have constructed their own data centers or used colocation as an intentional strategy.

    Security is a top cloud inhibitor, according to 47% of IT professionals. The leading cloud security concerns include data protection (40%), data theft (33%) and regulatory compliance (31%). The myriad cybersecurity threats evolve each day, and cloud technologies are not immune; for example, web APIs are now a significant threat vector. A separate report found a 300% increase in malicious API traffic in the past six months. Other issues, such as exposed keys, insecure defaults and misconfigurations are known to hamper cloud-native infrastructure.

    Contrary to popular belief, nearly half (46%) of companies reported that COVID-19 has not quickened their cloud adoption. This reality could indicate that cloud migration has been planned for years, and recent adoption is only part of an ongoing trend. The study found that the pandemic did, however, accelerate things like infrastructure-as-code (IaC), container orchestration, compliance and governance and other operational factors.

    Infrastructure Automation Supports Multi-Cloud

    Multi-cloud feels like an inevitable result of accelerating cloud adoption in a more diversified market. Rising multi-cloud use is also arguably in line with Conway’s Law. As organizations develop more and more digital experiences across new verticals, the platforms and backend processes to support these initiatives will inevitably grow more and more complex.

    To mitigate some of the issues posed by multi-cloud, many organizations are turning toward increased infrastructure automation. In fact, 70% of those surveyed said infrastructure automation tools are extremely important to their organization. Of these tools, provisioning automation (75%) is the most popular kind in use. Automation for other areas, like networking (58%), security (50%) and app development (69%) follow.

    Hashicorp’s State of Cloud Strategy Survey boldly announces, “Welcome to the Multi-Cloud Era,” and its findings back up that sentiment. For further insights, you can view the results freely on this website.

  • Infrastructure-as-Code Security will Improve Your Cloud Security

    Infrastructure-as-Code Security will Improve Your Cloud Security

    Moving to the cloud creates a tremendous opportunity to get security right and reduce the risk of a data breach. Most organizations start their cloud security efforts by ‘shifting security left’ thereby addressing issues in the CI/CD pipeline. Fixing known vulnerabilities and risky configurations before pushing images to production makes sense. But there is an opportunity to shift security even further left—as you set up your cloud infrastructure—to reduce risk. Infrastructure-as-code (IaC) security allows you to define security controls as you configure your cloud infrastructure.

    Getting Started on Your Cloud Journey

    Before we dive into IaC security, let’s talk about why platform teams want to deploy infrastructure-as-code as they move through their cloud journey. Most organizations have not yet tackled IaC. They start their journey by experimenting with cloud configurations until they have defined a stack that works for their teams. In this early stage, application teams may be defining their own cloud stacks. They are manually configuring infrastructure or using scripts. Configurations may not be standardized across teams.

    Standardization of Infrastructure Saves Time

    Although experimentation is reasonable early on in the journey, standardization of infrastructure saves time. Cloud services are no different than applications that run on premises. When your teams are using different versions of tools in their stack, getting an application up and running can be a nightmare. Interoperability between services is challenging, and it can feel like you are fixing a different flavor of the same problem over and over.

    Mature cloud platform teams typically prefer to create a ‘golden image’ with a baseline set of services for development teams. They may create a menu of a few standard options to meet different teams’ needs. Because they are standardized, the platform team can take time to validate that they interoperate and are secure. For example, these golden images will be regularly scanned for vulnerabilities and updated to ensure they meet internal security standards and any relevant regulatory requirements.

    IaC security cloud

    Automating Infrastructure Deployment Reduces Errors

    Manually configuring the environment in the cloud for each team is tedious. Once these configurations have been standardized, the cloud infrastructure can be provisioned as code using templates. A declarative approach defines the desired end state, whereas an imperative approach defines the detailed steps to achieve the end state. The templates are normally stored in a Git repository and version controlled. Automating the configuration of infrastructure not only saves time but also reduces errors that can disrupt operations.

    IaC Security Shifts Security Further Left

    Once you have templates that define your infrastructure, standardizing permissions and access rights for services-as-code is a logical next step. The security team typically defines the guardrails or policies for accessing cloud services. These policies can then be defined as code along with the infrastructure configurations. This IaC security is a further ‘shift left’, allowing security to be embedded into infrastructure, even before an application can be deployed. Implementing policy-as-code minimizes the opportunity for human error in configuring security services. By defining these policies and enforcement guidelines together, the DevOps and security teams can establish a trusted partnership.

    Although some organizations may start their journey by implementing IaC security, for most organizations it happens later in the cloud journey. The order is less important than ensuring you take a DevSecOps approach and embed security at each stage to manage risk. Be sure to address the following areas as part of your cloud and container security program:

    Continuously Check Cloud Permissions and Configurations

    Even if you have IaC templates, you need a program to regularly take inventory of all cloud assets and confirm permissions and configurations. The Center for Internet Security (CIS) benchmarks are commonly used to check passwords and access rights. Beyond enforcing these best practices for security on cloud services, continuously checking for changes to permissions and suspicious activity is critical. The cloud vendors capture logs across all of their services, and your security tool should continually monitor and alert using those logs.

    Scan Images for Vulnerabilities and Risky Configurations

    Scanning for vulnerabilities and ensuring configurations use best practices allows you to shift security left into the development pipeline. Scan everywhere you have images—including within registries, pipelines and also check for new vulnerabilities identified for images running in production. Be sure to scan hosts as well as containers. Defining scanning policies and enforcing through the development process creates guardrails for the development team and clarifies expectations. Most regulatory compliance standards have image scanning requirements as well.

    Detect Threats and Respond to Alerts

    Prevention is fundamental but must be supplemented with threat detection and response. Alerting on unexpected behavior can identify the presence of a threat. Initially, teams may use out-of-the-box detection rules as a way to get started and then customize rules over time. Checking configs against dockerfile best practices is a good way to get started. Overly permissive access rights for containers is common in Kubernetes environments and should be allowed only when required.

    As your team develops more expertise with containers and Kubernetes, you can also implement zero-trust network security policies to block unexpected communication between pods, services and applications.

    Some DevOps teams handle triage of alerts, as they can quickly determine if the issue is a configuration issue or an indicator of compromise. Teams often map detection rules to the MITRE ATT&CK framework, allowing their security teams to manage alerts consistently across their environment. For any team to effectively respond to a security alert, it is important to capture a detailed record of exactly what happened before, during and after that event. This visibility can be particularly challenging with containers but it is critical because containers have short lives. The 2021 Sysdig Container Security and Usage Report revealed that about half of containers live less than five minutes.

    Complying with Regulatory Standards

    Complying with regulatory standards can be painful. Some standards were written with cloud computing in mind, like NIST 800-53 and SOC2, and others were not. With those broader standards, it is important to understand the controls that are relevant to containers and the cloud.

    As you consider your compliance program, be proactive—don’t wait until you are ready to roll into production. Start early by engaging your compliance team and helping them get up to speed. Your role is to provide evidence of compliance to pass an audit at any time using the documentation from your security platform.

    Getting Cloud and Container Security Right

    The move to the cloud creates a tremendous opportunity to get security right and reduce risk. The fact that physical security and a chunk of operations are delegated to cloud vendors frees up security and development teams to focus on their more narrow scope within the shared security model. By taking a secure DevOps approach and inserting security into every stage of the application life cycle, your team can confidently and securely run cloud and containers.

  • The SolarWinds Query Report 2021: Database Priorities and Pitfalls

    The SolarWinds Query Report 2021: Database Priorities and Pitfalls

    The rise of hybrid IT and the accelerated adoption of cloud-native apps have evolved traditional database design, management and monitoring. Workloads are rapidly shifting to the cloud and application performance is heavily dependent on the performance of the underlying database. Meanwhile, the global COVID-19 pandemic has forced companies to make significant changes in the last year. Although it was and is a necessity, the transition to remote work and many organizations’ rapid pivots in strategy posed significant challenges for the technology professionals tasked with supporting new employee demands and shifting business priorities.

    As a result, organizations are increasingly acknowledging the importance of database performance management and strategy to ensure their most valuable asset: Data. SolarWinds recently revealed the findings of The SolarWinds Query Report 2021: Database Priorities and Pitfalls, which examines the current state of database management, evaluates the primary challenges database professionals face today and outlines how database professionals can best overcome these challenges.

    The North American results are based on responses to an online survey fielded in February 2021, which included participation from 331 technology professionals responsible for database performance management in the U.S. and Canada from public and private sector small, mid-size and enterprise organizations.

    Among many other key findings, this year’s report revealed the following:

    Increasing Complexity

    Database design and management are evolving in tandem with increased complexity, and IT professionals are on the verge of a significant shift in strategy that embraces cloud database-as-a-service (DBaaS) platforms as critical database performance success factors.

    • Though nearly 80% of the respondents said Microsoft SQL Server is the most critical platform in use today, there’s a relatively even split between the second, third, and fourth most-used platforms: cloud DBaaS (54%), Oracle (47%) and MySQL (43%).
    • As tech pros keep pace with advances in database platforms and indicate an intended shift toward cloud DBaaS adoption (33% of the respondents ranked cloud DBaaS as the number-one priority database platform to adopt over the next three years), the new mix of traditional/cloud DBaaS platforms will also require new skills.
    • Complexity increased in response to COVID-19 and the transition to remote work over the past year, according to 31% of respondents, who cited increased data demands from users, increased volume of data and security/compliance issues (such as GDPR) as the top three drivers of added complexity.

    The industry’s acknowledgment of cloud DBaaS platforms as critical success factors for database performance is now underway. When it comes to cloud migration (especially amidst a pandemic dramatically accelerating planning and execution of cloud computing and digital transformation), database professionals should start with a basic data and code cleanse and ensure only what’s truly needed is being migrated to reduce complexity and overall cost (we know cloud computing costs can add up). A comprehensive, successful migration strategy should also include a detailed assessment of the data estate—what dependencies or complex coding features exist? Remember, everything interconnected with a database has to go with it to the cloud, so taking these preliminary steps will help organizations avoid surprises down the road.

    Database Security/Compliance Influence Priorities

    The cloud and security/compliance are key factors influencing priorities and resource allocation in 2021 and beyond.

    • Nearly 60% of tech pros surveyed stated that of the data/systems currently managed within their organization, less than 25% are in the cloud. But this is expected to change, as 36% of respondents reported integrating cloud DBaaS is one of their organization’s top priorities in 2021.
    • Two-fifths of the respondents anticipated bigger budget allocations in 2021 (compared to 2020) for cloud services and security/compliance (42% and 39%, respectively).

    Managing database platforms is a big undertaking—a balancing act between proactive and reactive responses. But tech pros must make sure they’re covering all their bases, especially when it comes to security and compliance (and, on occasion, cost and licensing). Time and time again, we see these four big-ticket items pushed to the bottom of the priority list, and falling at one of these hurdles can do a great deal of damage. The new generation of legal compliance has real teeth if you fail to comply with the law. A simple cost-benefit analysis can be the proof point you need to highlight the potential impact of not upskilling in one of these areas and can give you a reason to move it back to the top of the priority list.

    Maintenance Constraints

    Database professionals devote more than a third of their day to database maintenance. This constrains both time and opportunity for activities such as innovation and upskilling in response to increased complexity and data-driven initiatives.

    • Maintenance work accounts for a greater portion of database professionals’ time when compared to other tasks. One-third of the tech pros surveyed stated one of the top areas where they spend the most time as a database/data/tech pro is maintenance (33%) and database development (22%).

    Daily database maintenance consumes time tech pros could otherwise be spending on things like upskilling or innovating new ways of working. For example, nearly half of those polled stated performance/query tuning was one of the top three skills/tools they believed they could develop/improve to better manage increasing database complexities, followed by security management and data engineering (45% and 41%, respectively). One of the best ways to work smarter, not harder: Mitigate downtime spent on maintenance and enable efficiency is to implement appropriate automation and monitoring tools. When implemented properly, automation and monitoring solutions can improve overall performance and eventually lead to reduced costs. Additionally, they can free up tech pros’ time, allowing them to focus on more proactive database performance management.

    The report reveals just how quickly the database architecture and operations tech pros are managing in today’s organizations are changing—and the web of management complexity accompanying this change. With application availability and performance as critical business drivers, the database has long been the linchpin of today’s IT environments. However, against the headwinds of increasingly diversified platforms and reduced resources, tech pros must be equipped to do more than reactive maintenance in environments that keep adding more databases with each passing day.

    Tech pros must capitalize on the opportunity ahead to maximize the performance potential of cloud-hosted databases and ensure the successful optimization of their organizations’ data.