devopsy.kinsta.cloud

Category: DevOps Practice

  • Unleashing AI in SRE: A New Dawn for Incident Management

    Unleashing AI in SRE: A New Dawn for Incident Management

    In my recent blog, Revolutionizing the Nine Pillars of SRE with AI-Engineered Tools, I indicated that AI could assist with incident management by automating the detection and triage of incidents and helping to quickly identify the root cause. In this blog, I explain in more detail how AI-engineered tools can be used to improve the SRE incident management pillar of practice.

    AI Use Cases for Incident Management

    Here are use cases for SREs to use AI to assist with software deployments.

    Detection of Incidents: AI can be instrumental in identifying and predicting incidents before they cause severe damage. Tools like Moogsoft and Datadog use AI to analyze system behavior and predict potential failures. However, a challenge could be ensuring these AI models are correctly trained to understand what constitutes an incident. To overcome this, invest time in training these models with historical incident data and continuously fine-tune them.

    Communication of Incidents: AI can automate the process of notifying the right teams when an incident occurs based on its severity and impact. Tools like PagerDuty use machine learning to streamline incident communication. A challenge might be setting up correct escalation policies. This can be overcome by carefully analyzing past incidents, understanding their impact and setting up escalation rules accordingly.

    Triage of Incidents: AI can assist in categorizing and prioritizing incidents based on their potential impact on the business. IBM’s Watson AIOps uses AI for effective triaging. However, defining the right parameters for triaging could be a challenge. It can be overcome by understanding the nature of incidents, their potential impact on the business and defining the rules for triaging accordingly.

    Identify the Root Cause: AI can help quickly identify the root cause of incidents by analyzing patterns in historical data. Tools like RunWhen and Zebrium use machine learning for root cause analysis. A potential challenge is ensuring that the AI model has access to all necessary data for analysis. You can overcome this by setting -p comprehensive logging and monitoring systems that collect all relevant data.

    Blameless Postmortems: AI can provide data-driven insights to facilitate blameless postmortems, helping teams understand what went wrong and why. OverOps and Blameless are examples of such tools. The challenge here is ensuring an unbiased analysis. Overcome this by setting up AI models that rely on factual data rather than assumptions.

    Documentation of Incidents: AI can help in auto-documenting incidents, capturing important details that could be crucial for future reference. JupiterOne, for instance, uses AI for this purpose. However, capturing all relevant details in documentation can be a challenge. To overcome this, configure the AI tools to collect comprehensive details about the incidents.

    Automation of Playbooks: AI can help automate the execution of incident response playbooks, reducing human intervention and response time. Tools such as StackStorm and Blameless Insights are tools that can help here. A potential challenge is ensuring the right playbook is executed for the right incident. Overcome this by training the AI model with historical incident data and the corresponding successful playbook execution.

    RoadMap for SREs to Transform Incident Response Using AI

    Needs Assessment and Gap Analysis: Start by understanding the current incident management process and identifying areas where AI can improve. Also, analyze the types of incidents you face most frequently and the time taken to resolve them. This will help create a clear problem statement and set expectations from AI implementations.
    Tool Selection: Based on the needs assessment, identify the right AI tools. Look at AI tools specializing in incident detection, communication, triage, root cause analysis and automation. Tools such as Moogsoft, PagerDuty, IBM’s Watson AIOps, Zebrium, OverOps, JupiterOne, RunWhen, Blameless and StackStorm might be good starting points.
    Model Training and Configuration: Configure the chosen AI tools as per your incident management process. You’ll need to train the models using historical incident data so they accurately reflect your environment.
    Integration with Existing Systems: Ensure the chosen AI tools can integrate seamlessly with your existing systems, such as your ITSM tool, monitoring and logging systems and communication channels. This is crucial for the automation of the incident management process.
    Pilot Implementation: Start with a pilot implementation on a small scale, maybe for a single service or a part of your infrastructure. This will help you understand the effectiveness of the AI tools and make necessary adjustments before a full-scale rollout.
    Measure and Optimize: Regularly measure the performance of the AI tools and their impact on your incident management process. Make necessary optimizations based on feedback and performance metrics.
    Full-Scale Rollout: Once you’re confident about the performance of the AI tools and their effectiveness in improving the incident management process, plan for a full-scale rollout.
    Continuous Learning and Improvement: The power of AI lies in its ability to learn and improve over time. Keep feeding it new incident data and train the models for better accuracy and effectiveness. Also, keep an eye on new advancements and features in AI tools and update your implementations accordingly.

    Summary

    Modern incident management is being revolutionized by AI, bringing unprecedented improvements in detection, communication, root cause analysis, postmortem reporting and automation of playbooks. The power of AI lies in its ability to learn and adapt, making it a powerful ally for SRE teams dealing with complex and evolving systems. But the journey to effective AI-based incident management isn’t without challenges. With the right tools like Moogsoft, PagerDuty, IBM’s Watson AIOps, Zebrium, OverOps, JupiterOne and StackStorm and a well-planned implementation roadmap, organizations can successfully overcome these challenges and unlock the full potential of AI in incident management.

    The roadmap to AI-based incident management starts with a thorough needs assessment and gap analysis, followed by tool selection and configuration. An integration with existing systems ensures seamless data flow, while a pilot implementation allows for tweaks and adjustments. Measuring and optimizing the performance of AI tools is a crucial step before a full-scale rollout. And the journey doesn’t end there—continuous learning and improvement is key to harnessing the full potential of AI. With the right approach and tools, the transformation to AI-based incident management can significantly reduce downtime, speed up resolution times and improve system reliability—a win-win for both SRE teams and their users.

  • The Skills Successful DevOps Pros Need in 2023

    The Skills Successful DevOps Pros Need in 2023

    In the last decade, DevOps adoption has grown considerably. An increasing number of companies across a wide range of industries are embracing the benefits of DevOps, and we have not seen the end of the growth cycle yet. Indeed, the total DevOps market is expected to continue to grow significantly in the next few years, with one recent market survey projecting an annual revenue growth of no less than 24.7% from 2023 to 2030 (which will bring the market value to about 51 billion dollars).

    Further, DevOps is continuously integrating with and accommodating new technologies, which are themselves evolving quickly. This means DevOps professionals will have to invest more and more time each year to stay current and effective at their job. But, this predicament comes with a silver lining. By staying current and even ahead of the game, DevOps professionals are better able to differentiate themselves and deepen their level of expertise—and with that comes the ability to increase their income potential over time.

    In this article, we take a look at what knowledge, skills and abilities (KSAs) DevOps professionals should bring to the table if they want to keep up with the fast pace of change in the industry in 2023—and beyond.

    Knowledge – What Pros Need to Know

    As always, DevOps professionals are expected to have a baseline level of knowledge before taking on a new role, internally or externally. But, as DevOps adoption grows, professionals will need to make a concerted effort just to stay abreast of current trends.

    Platform Nuances

    Each platform places limitations on DevOps processes, and their successful implementation within a company will vary depending on the platform. It’s not uncommon to have to tailor your approach in particular circumstances. In the case of Salesforce, for example, successful DevOps processes must begin with improved deployment speed and accuracy. Only once this is achieved can Salesforce DevOps professionals move onto more mature processes such as automated testing and releases.

    Industry Standards

    DevOps professionals should understand the constraints of the technology they are working with. and within their industry. Both factors may place limitations on or add additional requirements to what they do. Any given technology platform comes with its own unique constraints, such as file sizes or limits on API usage. This is especially true of cloud-hosted systems like Salesforce, where there are multitenant architectures and resources are allocated to customers. Similarly, different industries are faced with differing constraints, such as the HIPAA requirements for companies handling health care data in the U.S. As a result, DevOps professionals will need to understand these tied to specific standards or processes of their given industries.

    Transferable knowledge

    As a final word on knowledge, it’s critical to make sure you are aware of DevOps trends and best practices on platforms other than your own. There are times, more often than we realize, that your knowledge of what has been successfully utilized on other platforms can be transferred to your own. Even if these practices are not directly transferable, they can point you in directions that can ultimately prove beneficial to your DevOps work on a completely different platform.

    Skills—What Professionals Need to Master

    While there are a huge number of skills that DevOps professionals need in order to stay current and further their career growth, I’d suggest paying particular attention to…

    Version control

    This skill is fundamental—you just can’t do the job without it. Make sure you thoroughly understand its nuances: Branching strategies, rebasing or merging, rollbacks, cherry-picking, etc. It’s also valuable to understand different approaches’ limitations and use cases.

    Automation

    Become well-versed in the differences between continuous integration and continuous deployment/delivery (CI/CD) and how they form the CI/CD pipeline. Knowing where and when to apply it and how to apply it are also critical skills to master.

    Successful teams also benefit from building automated workflows. A recent survey Gearset ran with Salesforce DevOps professionals showed that 85% of teams were either currently using or planning to use source control over the next year. Automated workflows are critical to optimizing ROI in DevOps, as they reduce manual processes and unlock some of the most valuable benefits of DevOps for teams of all sizes.

    Testing

    Test automation is increasingly central to achieving DevOps success. As such, you would be wise to develop expertise in environmental testing, unit testing (integrated with automation and release gating) and UI test automation.

    Monitoring

    Application monitoring for performance, errors, data anomalies and security is arguably as crucial as testing. DevOps professionals must apply the same level of diligence to routine system operations as they do to deployments. As DevSecOps becomes more prominent in 2023, it’s expected that there will be increased collaboration and integration with infosec teams.

    Backup

    It’s critical to master the backup and restore process, no matter which platform you work on. It should absolutely be included as part of a comprehensive DevOps skill set. This is because skill underpins the mean-time-to-recovery metric. DevOps professionals must be familiar with the processes and tools necessary for restoring systems and data to achieve a rapid and secure return to business as usual.

    Demonstrating ROI of DevOps

    Finally, it’s essential for team leads, or those hoping to advance their DevOps career, to effectively convey the performance of DevOps, which can be measured using indicators such as Google’s DORA metrics. This means translating DevOps metrics into quantifiable business value that can be easily understood. It is crucial for managers to not only motivate their teams but also to validate the return on investment (ROI) in DevOps to critical stakeholders. By connecting the performance improvements resulting from DevOps practices with tangible business outcomes and demonstrating the ROI, managers can secure continued support for the DevOps approach within their organization.

    Abilities—What Professionals Need

    This final aspect of KSA plays an important role in fostering the collaborative culture that is the hallmark of DevOps success.

    Communication

    DevOps is not just about the tools—you need to maximize communication effectiveness so that the team aligns their goals with the least amount of friction. If everyone is using the same model to reach the same goal, you will achieve success that much faster.

    Continuous improvement

    DevOps success is not a fixed goal. Rather, there are always elements of the process, or your team’s skills, that can be tweaked, changed or improved.

    Making training a priority

    As per the aforementioned Gearset survey, 45% of teams who train once a month rate their team’s collaboration as ‘excellent’ compared with only 13% of untrained teams. Teams that prioritize training outperform all others on a wide range of metrics, including but not limited to seeing fewer bugs and data loss incidents.

    This Challenge is an Opportunity

    There’s a huge range of KSAs that DevOps professionals will need to master if they want to thrive in 2023. Clearly, working in DevOps has never been more demanding than it is today. But, from this challenge, there’s also a tremendous opportunity for professionals to make themselves indispensable.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    Well
    Raleigh-Durham-Chapel Hill, North Carolina
    Principal DevOps Engineer – Team Lead
    $180,000 to $200,000

    CareerBuilder.com

    Apple
    Cupertino, California
    Cloud DevOps Engineer – Enterprise Systems
    $138,900

    Indeed.com

    General Dynamics Information Technology
    Indianapolis, Indiana
    Capado Salesforce DevOps Engineer
    $116,000 to $174,000

    SimplyHired

    John Hancock
    Boston, Massachusetts
    Senior DevOps Engineer
    $109,000 to $138,000

    Dice.com

    IT Trailblazers
    Houston, Texas
    DevOps Lead
    $100,000 to $120,000

  • Teleport Assist Leverages Generative AI to Democratize DevOps

    Teleport Assist Leverages Generative AI to Democratize DevOps

    Teleport has made available a chat interface, dubbed Teleport Assist, that makes use of OpenAI’s generative artificial intelligence (AI) application programming interfaces (APIs) to enable DevOps teams to troubleshoot infrastructure issues more easily.

    Using the natural language interface (NLI) provided by Teleport Assist, DevOps engineers can ask questions and give instructions in English to massive fleets of servers. Capabilities include being able to suggest commands and scripts for debugging and resolving common issues and then generating commands for review before they are applied.

    In future versions of Teleport Assist, support for other cloud resources, such as SQL databases and Kubernetes clusters, will be added. Teleport Assist currently supports Secure Shell (SSH) and bash scripts, with support for additional languages and interfaces to be added over time.

    Teleport CEO Ev Kontsevoy said it’s already clear that many of the scripts that DevOps engineers rely on today for IT management will be replaced by generative AI platforms. The average IT administrator will be able to invoke DevOps platforms via an NLI that eliminates the need for extensive programming expertise, he added.

    In effect, the NLI surfaced by generative AI platforms will serve to democratize DevOps expertise, Kontsevoy noted.

    It’s still early days as far as the adoption of generative AI by DevOps engineers is concerned, but it’s already apparent that its impact will be profound. It will increase the pace at which applications are built while making it easier for software engineers to manage large codebases. Most of the generative AI advances thus far improve developer productivity, but tools such as Teleport Assist show how generative AI will also make it easier for more organizations to adopt DevOps best practices that previously were limited to organizations that could afford to hire and retain significant numbers of software engineers.

    At their core, generative AI platforms are fundamentally changing the way humans interact with machines. Instead of requiring a developer or engineer to create a level of abstraction to communicate with a machine, it’s now possible for machines to understand the language humans use to communicate with each other. A a natural language interface will soon enable DevOps engineers to ask generative AI platforms to not only surface suggestions but also debug applications and even write code. The accuracy and quality of those results will vary depending on how the large language models (LLMs) that trained those platforms were, but over time there will be a wide range of LLMs that are trained to be used within specific domains such as DevOps.

    The issue is determining how to organize DevOps teams as more tasks become automated. Generative AI doesn’t replace the need for IT professionals, but it does make it simpler to manage IT at unprecedented levels of scale. In fact, at this point, the proverbial generative AI genie is out of the bottle, so there is no going back. The only thing left to decide is how quickly generative AI becomes pervasively applied.

  • Mezmo Simplifies Management of DevOps Telemetry Data

    Mezmo Simplifies Management of DevOps Telemetry Data

    Mezmo has added additional capabilities to streamline the flow of telemetry data within DevOps workflows as part of an effort to make it simpler to surface insights and reduce the total cost of observability.

    The company has also added integrations with more data sources to the Telemetry Pipeline platform, along with controls that make it simpler to optimize data storage and usage. For example, an Events-to-Metrics Processor capability identifies and extracts metrics from logs to make them easier for third-party tools to consume.

    Mezmo CEO Tucker Callaway said these additions collectively make it simpler to surface insights that will make DevOps workflows more efficient. Previously, the company added capabilities such as rollback and redeploy, sequential parsing, error history management and data sample management to enable DevOps teams to manage telemetry data more efficiently.

    In effect, Mezmo is making it easier to apply engineering best practices to the massive amounts of telemetry data generated across DevOps workflows, he noted.

    No one knows for certain if all that data will require the addition of data engineers to DevOps teams, but as DevOps continues to evolve, it’s clear there is a need to cost-effectively manage data at scale. DevOps teams that embrace observability are quickly discovering that storing logs, traces and metrics for an extended amount of time can quickly become cost prohibitive. Naturally, there’s a lot more sensitivity to those costs during challenging economic times.

    In theory, of course, various forms of artificial intelligence should automate data engineering best practices, but in the meantime, there simply isn’t enough data engineering expertise available. The need for a platform to optimize the flow of telemetry data across DevOps workflows is becoming more acute.

    It’s not clear how quickly DevOps teams are embracing observability, but it’s clear that relying on predefined metrics to monitor IT environments is no longer sufficient. DevOps teams need to be able to query telemetry data to, for example, identify bottlenecks created by dependencies between microservices. Resolving those issues can’t be slowed down by a lack of data to query. Most problems can usually be traced back to a recent update to an application, but there are always going to be some issues that require access to historical data. That historical data should be quickly and easily available, especially since application downtime directly equates to lost revenue for many organizations.

    Observability, regardless of cost, is rapidly becoming a requirement for DevOps teams’ success as application environments become more complex in the cloud-native era. Dependencies between the microservices that make up those applications are too difficult to track and analyze without the aid of an observability platform. The challenge is not every observability platform provider is equally concerned about how much it costs to store all that data. Striking the right balance between the amount of telemetry data collected and the cost of storing it has become a much larger concern.

  • Google’s Shiny New AI Dev Environment — the ‘Experimental’ Project IDX

    Google’s Shiny New AI Dev Environment — the ‘Experimental’ Project IDX

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Google’s full-stack, browser based development environment in the cloud. It’s not just a Copilot clone, but aims to help you “get an app from zero to production.” (more…)

  • DevOps Download: What to Know About Life in the Cloud(s)

    DevOps Download: What to Know About Life in the Cloud(s)

    According to the 2022 Flexera State of the Cloud Report, 89% of cloud customers now use multiple cloud hosting providers. Nearly nine out of 10 might sound high, but it tracks with the anecdotal evidence we’ve seen. For varied and good reasons, companies are spreading their usage across multiple hyperscalers rather than putting all their eggs in one basket.

    In this article, I’ll take a look at what life in the cloud(s) looks like today. Along the way, I’ll examine the value that companies and their DevOps teams can derive from this approach, the challenges to be aware of, use cases for making the multi-cloud move and more.

    What is Multi-Cloud?

    Multi-cloud is a type of cloud strategy approach where an organization uses computing services from at least two cloud providers to run their applications. Instead of a single-cloud stack, these environments include two or more public or private clouds, or a combination of both. With multi-cloud, you can have an application running on Google Cloud Platform (GCP), another in Amazon Web Services (AWS), and maybe one on premise for hard-to-move applications.

    There are three kinds of cloud setups. Single cloud is when workloads run solely on one environment. Hybrid cloud entails overlapping workloads across multiple clouds, such as operating the same programs on different providers for business continuity purposes. The final setup is planned multi-cloud, which disperses applications and usage across providers along specific lines, like having compute intensive applications with one provider and data analysis applications with another.

    Got a Preference?

    Make no mistake, using multiple cloud providers is more complex than a single host, but as its popularity shows, the benefits can carry more weight. Foremost, that’s because an organization gains the flexibility to create a best of all worlds situation, based on factors like the following:

    • Operational preference: Users may like different features from different providers, and splitting types of use can yield the ideal setup. For instance, a customer may run compute on AWS but want their business intelligence and machine learning on GCP. In this deployment, the front-end app could gather the needed information, then send the data to the other provider for analysis and ML.
    • Regulatory requirements: Certain workloads may require data storage in specific regions, but not all cloud vendors have multiple “regions” in each country. Companies that want to use the cloud can set up a “multi-region” environment by having one region on AWS and a second region on GCP within the same country.
    • Cost savings: Cloud providers offer different savings plans. For a savvy user, structuring a flexible program can take advantage of each to significantly reduce costs. It’s also a great way to avoid the pitfalls of lock-in and can make it easier to go elsewhere should a provider lose your trust.
    • Failsafe Options: Outages happen, so companies needing 24/7 access to apps will sometimes run portions redundantly on two hosts to ensure smooth operations. For example, a retail app may run on GCP and AWS with live failover between them. That way, if one goes down the other can kick in without noticeable downtime.

    Not Without its Challenges

    The benefits to running a multi-cloud setup are clear, but there are some challenges as well. A carefully planned multi-cloud setup can save money, but a careless approach will duplicate operations and cost. It also leaves companies vulnerable to mistakes that can lead to significant overrun, especially for hybrid cloud operations, which can be extremely complex. Specialists must be well versed in FinOps principles and experienced at writing modern applications that can scale and work across regions and clouds, to ensure success in a multicloud approach.

    Coming to Terms

    You’ll undoubtedly come across a lot of new ideas and terminology when researching cloud setups. Here’s a few of the more crucial ones to become familiar with as you plan strategy.

    • Lift and shift: This refers to cloud-to-cloud migration, in other words, taking what you have in one cloud and putting it in another, whether it is private to public or public to public.
    • Move and improve: This usually refers to migrating while making a small change to take advantage of a public cloud feature. For instance, you could be using open source Kubernetes on premise but want to move to a managed version. Eliminating management of Kubernetes would require a minor change but it improves operations greatly.
    • Cloudbursting: On premise offers hard limits to capacity. For companies able to anticipate business peaks, cloudbursting can scale workloads to an on-demand public cloud for a short period. For example, a retailer who operates on premise may cloudburst on Black Friday in order to handle a volume increase.
    • High Availability: The goal of this is to eliminate single points of failure by enabling users to continue working even if one of the IT components that an applications depends upon fails.
    • Disaster Recovery: Clouds offer solutions to help an organization restore critical systems and data after a disaster strikes. Recovery comes in three “heat levels”: Cold is for data backup only, warm restores operations in several hours, hot is for near-instant resumption of cloud operations.

    People and Processes

    The most important considerations for operationalizing multi-cloud are people and processes. Providers pump out new services fast, and not only can it be difficult for staff to keep up, it can be demoralizing when they’re unable to. Allowing time to succeed, fail and learn is key, and larger companies might consider designated specialists for each platform.

    When it comes to processes, management and security are highly customizable, but it requires deep knowledge to get them right. Operating via infrastructure as code rather than ClickOps is a must. The latter is a time consuming approach that requires selecting options in a provider menu and introduces both operational and security risks in the form of configuration drift.

    Life in the Clouds

    Multi-cloud brings strong benefits to DevOps. Organizations can get the right infrastructure balance based on their needs. Distributing workloads can reduce the impact of outages and bolster resiliency. There’s greater opportunity for cost savings, and scaling is simple. Ultimately, multi-cloud operational enhancements can result in more reliable and innovative solutions, which delivers on the mantra of DevOps itself.

  • Most Organizations Benefit From a Mature Cloud Strategy

    Most Organizations Benefit From a Mature Cloud Strategy

    Today’s cloud adoptions are wide and varied. Large enterprises are composed of vast, hybrid multi-cloud strata, often incorporating legacy technology acquired at various times and haphazardly equipped from department to department. But a fractured cloud approach can leave IT wanting in terms of shared automation and standard security policies.

    Not all organizations have addressed best practices around common areas like CI/CD, resource optimizations, or common policies for multiple clouds. And many teams are just getting the hang of the new cloud-native stack. But studies show that a mature approach to cloud operations can pay dividends in the long run—and this approach hinges on cloud platform teams.

    HashiCorp recently released the findings from its third annual State of Cloud Strategy Survey, which surveyed 1,000 technology practitioners on their cloud adoptions. In short, the report discovered that increased cloud maturity equates to increased overall value. Below, I’ll distill the major findings from the report to better understand how cloud-native operations are maturing.

    State of Cloud Maturity

    Most organizations are moderately increasing their cloud investments. Overall, 56% increased their cloud spending in the last year. Increases were much higher for higher cloud-maturity organizations and stilted for lower-maturity organizations. The findings demonstrate general confidence in cloud adoption, even amid economic uncertainties.

    There is also persistent optimism in multi-cloud strategies. In fact, 73% of companies said a multi-cloud strategy helped advance or achieve their business goals. And to successfully accomplish a multi-cloud strategy, respondents ranked security, uptime and availability, infrastructure scalability and regulatory compliance as top factors.

    However, there is still a lot of waste in the cloud. Nearly all organizations (94%) said they have avoidable cloud spending. These expenses most commonly stem from areas like over-provisioning resources, idle or unused resources, cost premiums or a lack of expiration date on cloud resources.

    Benefits of Maturing Cloud-Native Operations

    Overall, there are many reasons for adopting a multi-cloud approach—the report found reliability, cost reduction, security and governance and digital transformation as top motivators. And on average, high-maturity cloud adopters tend to reap the rewards more often, the report found. Some of these top rewards ranged from a stronger security posture to better visibility and increased automation. Furthermore, half of high cloud-maturity organizations reported successfully saving money using multi-cloud.

    Most interestingly, perhaps, is that high-maturity cloud adopters said they are better equipped to attract, motivate and retain talent than lower-maturity cloud adopters (74% compared with 48%). It’s generally easier to pull in talent when operating with attractive cutting-edge technologies that look good on a resume.

    Spotlight on Platform Teams

    Even then, skills shortages remained the top bottleneck in most organizations’ ability to operationalize multi-cloud and set up platform teams. This can be side-stepped by reskilling or upskilling staff, incorporating more consultants or increasing the budget. But the best strategy is to standardize on a common operating model.

    In that vein, investments into the platform engineering concept are rising. In fact, 92% of organizations are adopting, standardizing or scaling platform teams. Low cloud maturity firms tend to lag on platformification, most likely due to lacking the necessary skills to build and maintain internal developer-facing infrastructure, like the tools used to measure and improve developer productivity and satisfaction at LinkedIn. Others are still on the cusp of exploring how to implement it.

    The idea of platform engineering is still taking root, so it was interesting to see how organizations described the platform teams’ roles and responsibilities. The results showed platform engineers commonly take the reins on everything from architecting solutions to tooling acquisition and internal evangelism. Here are the five top-cited roles for platform teams:

    • Develop and standardize a cloud infrastructure strategy (95%)
    • Architect cloud solutions (95%)
    • Create and distribute cloud management, operational policies and best practices (94%)
    • Determine which cloud vendors and/or technologies are used in production (93%)
    • Define and measure site reliability (91%)

    The Path to Cloud Maturity

    In general, companies are still confident about their cloud trajectories. But, getting to a mature multi-cloud state will take work. One area that needs further scrutiny is cybersecurity, as password or credential leakage is now seen as the most prevalent cloud security threat. Organizations will require better DevSecOps best practices to avoid such leaks. That, and certain open source cloud-native tools could also aid security and compliance efforts.

    A mature cloud strategy will also require localizing development efforts through shared platforms and increasing infrastructure automation. On that note, 83% of organizations say automation tools are important or extremely important to operationalize their multi-cloud infrastructure. More automation for cloud-native areas like CI/CD, observability and Kubernetes management could also abstract complexity and fill the skills gap.

    As the HashiCorp study clearly showed, those organizations that have matured their cloud strategy are far better off because of it. They tend to understand their workflows better and can more seamlessly introduce new automation. Now, it’s only a matter of getting there …

  • Data Management: Cloud Technology or Data Centers?

    Data Management: Cloud Technology or Data Centers?

    Traditionally, larger enterprises have relied on in-house server hardware equipment for their data management needs, optimized to run high-performing applications with a certain degree of assured security and reliability. 

    However, the high costs and technical expertise required to manage these solutions effectively often render them inaccessible to startups and emerging businesses. But, as the old saying goes, necessity is the mother of invention. 

    With the growing demand for cost-effective, efficient data management solutions, the tech industry has witnessed the advent of cloud technology and data center colocation rental services. These alternatives pave the way for startups and small businesses to handle large volumes of data without straining their budgets.

    This article will compare and contrast cloud technology and data center services to help business leaders make well-informed decisions about their data management needs and the solutions that can help them.

    Understanding Data Center Colocation Services

    Data center colocation services offer an enticing solution for companies that require robust data management capabilities without incurring high capital expenditure for securing storage hardware, along with storing and powering them. 

    Essentially, this solution is like renting a space for your data in a specialized facility. These data centers provide a secure environment with the necessary power, cooling, and physical security for your server, storage, and networking equipment.

    In opting for colocation services, companies can leverage state-of-the-art data center infrastructure without the burden of high upfront costs and ongoing maintenance responsibilities associated with owning a private data center. This also allows businesses to focus more on their core competencies rather than the intricacies of data center management.

    However, data centers come with their own set of unique challenges. For instance, they often require a certain degree of technical expertise on the part of the business to manage their servers. This means hiring or training staff to handle server management, increasing the overall cost of this solution.

    Additionally, data centers’ effectiveness can sometimes be influenced by physical proximity. The closer your operations are to the data center, the better the performance can be. This geographical factor can pose problems for businesses operating in multiple locations or managing data across different data centers.

    The Power of Cloud Technology

    In contrast to relying on data centers, cloud technology offers businesses of all kinds and sizes immense scalability and flexibility in terms of data management solutions. 

    As a service that delivers computing infrastructure, software, or storage as a service via a network connection, typically the Internet, cloud technology eliminates the need for physical proximity to servers. It is quickly scalable, both upwards and downwards, as per business needs, ensuring that you only pay for what you use.

    Moreover, the ability of cloud technology to integrate various services can simplify business operations and enhance customer engagement. For instance, many businesses are now offering cloud-based text-to-pay systems that allow customers to pay from any device, providing ease of transaction while simultaneously gathering customer data in the cloud.

    Cloud services can also provide robust disaster recovery and business continuity solutions. However, businesses often cite concerns about data security and privacy in the cloud. Although most cloud service providers implement best-in-class security protocols, the very nature of the cloud—being an off-premises solution—necessitates a high level of trust in the provider.

    AI and Data Management Solutions

    If your business relies heavily on AI applications, you must understand how your data management solution supports this need. AI requires high-quality, well-managed data to function effectively. Here, cloud technology often presents an edge over traditional data centers.

    Many cloud service providers offer built-in tools and services designed to better facilitate AI implementation, including pre-trained machine learning models and AI platforms as a service. These resources can dramatically reduce the time, costs, and complexity that come with implementing AI, particularly for small to mid-sized businesses without extensive in-house AI expertise.

    On the other hand, while data centers might not have these built-in tools, they offer robust performance capabilities for AI workloads and often provide more control over the data, which is vital for sensitive AI applications.

    Keep in mind you can turn to other server-based solutions for storing data as well. For example, if your application relies on a significant amount of PDF content to store data and function effectively, hosting those files on a server such as a server-side PDF SDK will allow you to process those documents on the server via an API. This will then allow you to autoamte your document processing and convert large volumes of your PDF files. 

    Web Hosting Requirements

    Your current web hosting requirements can also significantly impact the choice between data center services and cloud technology. Shared hosting services may be sufficient for businesses with low-traffic websites or applications, and if you don’t need to store a lot of data, but as your traffic increases and the website grows, so do your hosting requirements. 

    Cloud hosting solutions offer excellent scalability, allowing your website to grow with your business. As traffic surges, cloud servers can easily handle the increase, maintaining website performance and user experience.

    Conversely, with a dedicated data center, you’re generally limited to the capacity of the server you rent. To increase your capacity, you may need to physically upgrade the hardware, a process that can result in downtime and potential loss of revenue.

    Considering Cost Implications

    The overall cost of your chosen solution will be a key factor when deciding between data center colocation services and cloud technology. Each service has unique cost structures that can vary between providers, and understanding these can significantly impact your company’s financial health and resource allocation.

    With data center colocation services, the costs are generally more predictable. You pay a fixed amount based on your rented server space and the bandwidth you use. This stability can be advantageous for budgeting purposes, especially for businesses with steady data usage and processing needs. However, you may also incur additional expenses for any necessary hardware upgrades, maintenance, or increased bandwidth usage.

    What’s more, there’s the indirect cost of employing or training staff to manage the servers, plus the potential costs of transportation to and from the data center, particularly for businesses not located near their rented server space. These costs can add up and should be factored into the total cost of operating within a data center.

    Cloud services, conversely, operate on a pay-as-you-go model. This solution can be much more cost-effective for businesses with shifting data needs since you can readily scale up or down based on demand, ultimately only paying for the resources you actually use.

    Choosing the Right Path for Your Organization

    Unfortunately, there’s no one-size-fits-all solution for effective data management. Each business has unique needs, requirements, and considerations that necessitate a carefully tailored approach. When choosing between cloud technology and data center colocation services, business leaders must carefully consider several key factors to determine the right solution for them.

    If scalability, flexibility, and AI integration rank high in your data management strategy, cloud technology presents an excellent solution with its on-demand resources and array of built-in tools. On the other hand, data center colocation services provide robust performance capabilities, greater control over data, and predictable costs—features that may appeal to businesses with more consistent data management needs.

    Understanding your current web hosting requirements and the potential cost implications of each solution are also crucial elements to consider in your decision-making process. The demand for efficient, secure, and cost-effective data management solutions will only continue to grow as data storage needs expand rapidly.

  • Low-Hanging Fruit, 2023 Edition: Part One

    Low-Hanging Fruit, 2023 Edition: Part One

    We’ve been cranking it out, haven’t we? Generating cool new toys for the business, creating new infrastructure to host those new toys and trying our best to secure all this new stuff while maintaining both our sanity and our cadence. And we’re doing it. Kind of.

    One thing we’re naturally lagging behind on is security. While we don’t purposely shortchange security in our DevOps processes these days, it always takes a bit to figure out the best way to secure new technologies after they’re implemented. And we’re implementing a ton of new technologies. So our security staff (and increasingly, DevOps/SRE teams) is rushing to play catch-up. One area where this is screamingly apparent is APIs. We’re spewing them across the organization, often without cataloging or responsibility. This makes sense, as in many cases, the API is just an evolution of the function call. “I need a service, it is a good generic service, let me throw it up and we can use it from wherever the client ends up deployed,” type of thing. The problem is that leaves a ton of attack surface not truly protected. Because without cataloging or responsibility, there is no way for security to know about many of these APIs.

    API Scanning Technology

    That brings us to today’s topic. If you haven’t done so yet, it is past time to look at API scanning technology. It comes standalone and bundled with pretty much any API security product on the market and is increasingly bundled with other development security or even development tools. And the whole point is to get a list of APIs. Oh, most commercial implementations do something with that list–include it in static testing, drive it into WAFs, whatever—but just having the ability to inventory what is running on your network is reason enough to stop doing less important stuff and figure out what your exposure is. And the tools make it easy. They come in two flavors, and we should consider both.

    Static API Scanning

    The first type digs through source code and API specs to come up with the list of what is supposed to be running on the network. The big weakness of this method is that it only knows what it is told. This is static API scanning.

    Dynamic Scanning

    The second watches network traffic for API calls and then documents them, possibly even generating API specs for you. The big weakness of this method is that it only discovers what is being accessed on the network while it is watching traffic. Currently, unaccessed APIs are not caught, nor are ones on unreachable networks. Unsurprisingly, this is dynamic scanning.

    As usual, the two together are more powerful than apart. By implementing both technologies, an organization can work toward a hardcore comparison of what should be running on the network versus what is running on the network. This goes beyond detecting unreported APIs, it also means that zombie APIs or runtime drift can be discovered by comparing what is there to what should be there.

    If you’re not doing this, consider adding it to the toolbox. Setup is not onerous, and the reports are lists of APIs and parameters that can be reviewed at leisure. Some commercial products that offer these two functionalities take it quite a way beyond list generation, and if comparisons are made, work hard to understand just how much each toolset can do.

    And keep rocking it. We need more tools to lock down the new perimeter precisely because you and your team are tackling new technologies and platforms to better serve the business. We are increasingly seeing security tooling catch up with the broad selection now available, so go implement them.