devopsy.kinsta.cloud

Category: AI

Three Trends That Will Transform DevOps in 2023

The onset of COVID-19 accelerated digital transformation-related initiatives like cloud migration, microservices, observability and CI/CD, giving long-delayed projects the budget and staffing they needed to build momentum. After almost three years of increased investment and accelerating progress, CTOs and software development teams are navigating how to enable the improvements in time-to-market and customer experience that they set out to achieve. In 2023, we’ll see enterprise DevOps mature and expand its business impact by placing a central focus on quality, delivering improved time-to-market, customer satisfaction and productivity.

DevOps Matures as a DX Enabler

The past three years have seen massive growth in cloud computing, observability and pipeline automation as enterprises raced to deliver new, high-quality digital experiences to consumers. Cloud spending alone increased from $270 billion in 2020 to 397.5 billion in 2022, according to Gartner. With new technologies in place, the focus shifts to processes that deliver business agility and efficiency.

Under increased pressure to deliver faster, businesses need to revisit, evolve—and, in some cases, undo—decades of risk and change management practices built around brick-and-mortar experiences and long software release cycles.

As digital engagement increases, so will expectations for software quality and scalability. We’ll need to invest in new tools and practices—from testing to observability and beyond—to ensure that we’re meeting or exceeding those expectations.

The economic uncertainty will compel us to look for ways to improve productivity and decrease waste. Quality practices are central to both. We’ll continue to look for ways to shift left and catch defects early in our pipeline when they are the least expensive to fix. We’ll want to optimize our build, integration, testing and deployment processes to identify bottlenecks and causes of rework that drain the productivity of so many teams daily.

DevOps, with its emphasis on automation and collaboration, will connect the dots between operational goals, technological investments and improved software. Established frameworks like DORA give software development organizations concrete ways to measure the efficiency and effectiveness of delivery pipelines. The key metrics outlined by DORA—deployment frequency, lead time for changes, change failure rate and time to restore service—enable teams to align on where they can improve development practices and quality practices.

DevOps Will be Measured by Business Success

The value of digital transformation and DevOps will also extend even further across the enterprise as CTOs evaluate DevOps practices with structured assessments such as DORA. With several years of accelerated digital transformation under our belts, the software development industry can build out a broader definition of DevOps performance over time. The DORA framework sets the stage for understanding how engineering practices impact the customer experience and, in turn, company growth. In one example, Liberty Mutual, a company prides itself on exemplary customer service, is already seeing how DevOps performance connects to business success.

Architect Justin Robinson notes in their roundtable on data-driven decision-making and DORA metrics: “For organizations, this is improved organizational performance. If we want to keep investing in great tech and serving our customers as best as possible, [we need] behaviors and practices that we can adopt to continuously get better and hit on every front here. Happier engineers, a better culture and a better bottom line for our company.”

In 2023, more enterprises will harness DORA to measure DevOps performance and its impact on company growth.

Customers Will Reap DevOps Benefits

We’re already seeing how metrics like deployment frequency can support an organization’s ability to attract and retain users. According to the 2022 Testing in DevOps Report, teams that increased deployments by 50%-100% were two-and-a-half times more likely to report high user satisfaction. The ability to innovate—without introducing defects to users—is the next frontier in DevOps success. Quality engineering will serve as a safeguard for DevOps teams so they can deliver new features faster and with confidence.

The Testing in DevOps Report also illustrated the growing connection between digital transformation, DevOps and quality engineering. A full 88% of teams with highly automated pipelines and mature DevOps practices reported high test coverage, which in turn is connected to high customer satisfaction. Test coverage, a core metric for quality assurance teams, is becoming the connection between code and customers throughout development. Quality teams will expand their testing strategies to cover more aspects of the customer experience, including non-functional attributes like performance and accessibility. Automation, particularly automated testing embedded in CI/CD pipelines, will allow teams to adopt these additional quality factors without slowing deployments.

Connecting Engineering Transformation to Business Success

2023 is shaping up to be an unpredictable year in technology. But with the momentum propelling digital transformation to an enterprise-level priority, engineering leaders have the data and the tools to shift focus from DevOps processes to transformative outcomes. DevOps maturity, measured and improved with DORA, will help organizations extract more value from their digital transformations by supporting greater innovation and better customer experiences. Quality engineering, augmented with new testing strategies that measure more of the customer experience, will play a critical role in this evolution by enabling development teams to move faster with confidence.

February 2, 2023
3 Performance Challenges as Chatbot Adoption Grows

Today, many organizations are starting to use voice or text-enabled chatbots for the first time or already have chatbot systems in place. In fact, Gartner said 25% of organizations are predicted to make chatbots their primary customer communication channel by 2027. This is because consumer behavior has shifted to primarily digital interactions and away from traditional customer support channels.

As a result of today’s digital landscape, consumers are now accustomed to immediate gratification and expect instant responses and resolution from customer support, which makes chatbots an ideal solution for brands to rapidly serve customers. Due to this rising demand for instant customer service, chatbot systems have become more sophisticated and consumers’ reliance on them is increasing. This increase in adoption has led to higher performance expectations for bots and if those expectations are not met, customer satisfaction and brand loyalty will suffer.

Below, we’ll look at three major challenges tied to the growing popularity of chatbots as they handle larger and more diverse groups of customers, and how business leaders developers and DevOps and customer experience (CX) teams can tackle these common challenges with a continuous testing approach.

The Pain Points of Adoption: 3 Key Challenges as Chatbot Usage Grows

As the popularity of chatbots explodes for customers around the world, the underlying systems that handle natural language processing (NLP), latency, data security and other functions need to be stronger. This is due to a growing and increasingly diverse demographic of users, many of whom are sharing more of their personal information to address a wider range of customer needs. Common chatbot pain points can be summarized by three overarching challenges.

Challenge #1: Chatbots must contend with a larger and more diverse base of customers

Chatbots that are voice or text-enabled must be smart enough to operate securely while navigating countless variations in spelling, a wide range of spoken accents and vernacular, background noise, static from bad connections and more. Even in a single language, there may typically be only four or five “intents” from most customers in engaging a chatbot, but those intents may be phrased in hundreds of different ways. Now, the global growth in adoption raises the chance of poor connections and exponentially increases the number of languages and variations in phrasing that a chatbot must understand and interpret.

Challenge #2: Customers are trusting chatbots with more personal and sensitive data

As chatbots become smarter and more trusted by customers to meet their needs, people are getting more comfortable giving their personal data to chatbots. That increases the number of situations where chatbots are working with personally identifiable information (PII) or other highly sensitive data. Especially given GDPR and other data privacy regulations being enacted worldwide, there’s added pressure to ensure a chatbot system’s handling of this data remains secure and compliant across all use cases and geographic regions.

Challenge #3: Modern chatbots require 24/7 availability and reliability

In an era of flexible work and millennial and Gen Z audiences interacting with bots after business hours and on weekends, chatbots can’t afford to work in shifts. Users may be spread across many time zones or around the world, meaning there’s always a sizable portion of customers awake, online and looking for fast and efficient service. Because of this, chatbots must be able to handle spikes and heavy traffic 24/7.

The executive takeaway from these challenges is that modern chatbot investments must be built around dynamic systems capable of adjusting to shifting customer usage and performance requirements.

A Continuous Testing Mindset is Required for Top Chatbot Performance

Fortunately, by honoring a few best practices and making the right strategic technology investments in automated artificial intelligence (AI) and machine learning (ML)-driven systems, C-suite leaders responsible for CX and the teams they manage can deploy chatbots capable of keeping up with the above challenges. Perhaps the most central element for success is to embrace a continuous testing approach.

For example, when a company is undergoing digital transformation and shifting its contact centers and supporting systems to the cloud, they are moving a significant amount of data, reimagining complex integrations and rearchitecting processes. In order to maintain application stability and existing integrations in the contact center, continuous testing is critical to ensure that an organization’s systems still operate effectively throughout the cloud migration process. With continuous testing, development and testing aren’t separate processes. The testing is automated and quality assurance is involved as developers submit code.

The most effective approach to continuous testing includes end-to-end analysis across all channels. Testing chatbot behaviors should mimic real-time customer interactions. This should include challenging scenarios such as unexpected user inputs and usage spikes at random times of the day and night. Automation is critical, especially in peak demand scenarios such as Black Friday, Cyber Monday, open enrollment, etc. Load testing up to 1,000-100,000 bot requests per second, or more, can help ensure companies will be able to handle the increased traffic. Ideally, the testing approach should be comprehensive to include automated NLP score testing, conversational flow testing and security testing and performance testing and monitoring.

By promoting the continuous testing approach across contact center operations, executives can empower their teams to build and maintain chatbots that remain optimized on an ongoing basis and that form an essential part of providing exceptional CX. This will make a huge difference in performance and reliability as systems scale and adapt to the evolving demands of a growing customer base.

Continuous Testing Elevates Chatbot Reliability and Value

High-performing chatbots and the continuous testing that keeps them optimized at those performance levels can benefit any company by lowering risks and enhancing revenue. Organizations that become adept at applying continuous testing avoid 90% of the downtime from high-severity (SEV 1) errors, which can result in saving $2.2 million in costs over a three-year period. When continuous testing touches every stage of the development process, it identifies issues earlier and helps strengthen an organization’s ability to innovate faster and service its customers effectively and profitably.

January 31, 2023
Let the Machines Do It: AI-Directed Mobile App Testing
AI has been in the news a lot lately. We’ve all heard a lot about ChatGPT and you’ve no doubt encountered the end result of a variety of other advances in AI technology of late, such as Lensa. But there’s also a much quieter and less visible shift happening behind the scenes during the development of your favorite apps—AI-directed mobile app testing. Let’s take a look.

Behind the Scenes

It’s no secret that every product, no matter the industry, goes through a gauntlet of tests before it’s unleashed on the general public. This goes for everything from a simple object like a cardboard box, for example, to a far more complex product, such as the latest electric vehicle.

The same goes for the mobile apps on your phone. And if you haven’t given that much thought, then good! It means that the QA team behind the app’s functionality did its job, producing a seamless experience that functioned without a hitch, exactly as you wanted and expected.

But that’s not always the way it goes, and mobile app testing is only growing more complex. That’s a growing problem that AI will eventually resolve.

If your app didn’t perform as expected, you—and many other users like you—may delete it and move on to the next app that promises to do the same thing. While this may not seem like a big deal to you as the user, it’s a total disaster for the tireless teams working behind the scenes to ensure a successful app launch or update.

The adage ‘You only get one chance to make a first impression’ is very true in the mobile app market. It’s a big, crowded and cutthroat mobile app world out there, and users simply have no incentive to put up with an app that doesn’t do exactly what it’s supposed to do. After all, for every single app, there are often several others standing by to scoop up your unhappy (former) users.

This issue is by no means restricted to small development teams strapped for resources. I’m sure you can think of some high-profile software releases from major companies that have launched in a disastrous state over the past few years (particularly in the tremendously profitable arena of video games).

So why is it that some teams cut corners when it’s time for testing, only to dearly wish they hadn’t?

Well, the answer is pretty simple: Up until very recently, mobile app testing has traditionally been time-consuming, complex and bewildering, all too frequently delaying releases and flabbergasting QA teams.

To understand this ongoing problem, let’s take a look at what this kind of mobile app testing entails and how this landscape has recently started to shift, as well as what’s on the horizon to make it all more efficient.

Cutting Corners = Consequences

Once upon a time—in an era that seems like it was really just yesterday—software testing was performed manually, entirely by hand. This was possible because quality assurance personnel only needed to consider a very limited set of variables: There were only so many use cases and form factors to consider, and programs were only expected to do so much.

It’s hard to imagine testing a modern app manually and having the result be effective. The approach is only reasonable under very limited circumstances. Nowadays, teams use automated tests, which save them a tremendous amount of time, effort and resources.

Yet these automated tests come with their own problems. Most of these automated tests are hand-coded, meaning that they must be set up and adjusted by specialists with a background in computer programming. Such specialists are not only costly and in demand, but they’re also needed for later aspects of the software development cycle, such as developing new product features and overseeing other aspects of production.

And this is where companies start to cut corners and miss problems and issues, both large and small. Instead of what is known as shifting left (that is, testing as early in the cycle as possible), many teams, unfortunately, view testing as a burden or afterthought. Only when things go haywire and they have a full-blown crisis on their hands does the need for rigorous app testing become apparent.

Testing, Machine-Made

This is where machine learning and artificial intelligence enter the picture. For several years, we’ve known that testing performed by machines will be more and more necessary to handle increasingly complex apps, and it won’t be long before the cost associated with the complex testing scenarios will simply become too much for many developers.

Fortunately, advances continue to appear on the horizon. These include the much-discussed no-code revolution, which abstracts away much of the noise associated with software development and opens the door to a much wider testing base.

AI- and machine-learning-based testing is also on the horizon, yet there’s a significant challenge that must be overcome before this can happen at speed and scale. To develop mobile app tests with true intelligence and maximum efficiency, machines must be capable of autonomously completing the following steps:
1. Comprehension: Draw from the ability to understand what’s changing in an app. For example, the machine must be able to detect differences and issues in a signup sequence or a checkout screen change.
2. Impact analysis: The machine must know what test cases can cover that change. This sort of identification can be quite tricky.
3. Test case generation: Next, the machine must possess the ability to generate test cases related to steps one and two. To do this, the machine must draw from several data sets. These include:
1. 1. Extent test collateral/test cases with product code
  2. Data from actual customer usage so that numbers one and two above may be identified
  3. What user environments look like, including information like how many users are active in the United States and in Europe
  4. Automatic execution: Perhaps the most crucial need is that the machine can perform testing entirely by itself. With the ability to self-execute tests, we’ll have reached a new era in testing.
  5. Reports: Before the testing cycle can begin again, the machine must be able to produce detailed reports that make sense to its human handlers. When it comes to test results, the more detail the machine can provide in as approachable a manner as possible, the better.
Problems and Solutions

Some testing platforms are now capable of many of the tasks I highlight above, and quality assurance teams that move away from self-coding tests and toward the future of testing stand to benefit from these solutions (particularly when their competitors do not).

Partially enabled by AI and ML, no-code solutions provide a stopgap solution until machines can really get a handle on the above five steps. At this point, it’s really just a matter of time: We can already see glimpses of a wholly automated testing cycle where human input is limited and machines do the heavy lifting.

It’s easier than ever to imagine a world where there’s no need for testing to be an afterthought and software disasters have become a thing of the past.
January 30, 2023
What DevOps Needs to Know About ChatGPT

OpenAI’s generative pre-trained transformers (GPTs) bring powerful AI language models to every IT community. Within the past few months, ChatGPT has become an incredibly popular derivative of GPT. GPT also has the potential to shake up the DevOps and DevSecOps communities. This article will provide an overview of GPT, its history, current status and future effects on IT, especially DevOps.

Learning Overview

Understanding the difference between supervised and unsupervised learning is essential before understanding GPT. Machine learning and artificial intelligence (AI) use unsupervised and supervised learning approaches. A supervised model uses labeled data to predict outcomes, while an unsupervised model does not. In general, supervised models have some limitations. They need a large amount of annotated data to learn a particular task, which is often unavailable, and they also need to generalize beyond the task for which they were trained.

A generative language model is a class of statistical models that differ from discriminative models. You can, for example, generate photos of animals that look like real ones using a generative model. Whereas with a discriminative model, you can determine whether an animal is a dog or a cat. Generative models can assign probabilities to a series of words, such as those that predict the next word or sentence. A generative pre-trained transformer is unique because it uses both unsupervised and supervised learning from unsupervised unlabeled text and follows up with discriminative supervised fine-tuning.

GPT-based models—ChatGPT, for example—always return the text “I am” when asked, “I think, therefore …” On the other hand, if you ask ChatGPT, “Why did the rabbit cross the road?” it may respond thusly:

“The age-old riddle still awaits an answer: Why did the rabbit cross the road? It’s a mystery that has stumped people for centuries and will likely remain an enigma for years to come.”

In this way, we can better understand how GPT-3 works. When asked, “Why did the rabbit cross the road?” most humans will answer, “To get to the other side.” A Google search for “Why did the rabbit cross the road?” mainly yields joke punchlines on the first page of the results. GPT doesn’t differentiate between humorous or cynical answers or specific matter-of-fact answers. The model makes only guesses based on the data. Since, in this example, most of the highest-ranked answers are jokes, GPT’s answer is enigmatic at best. GPT is not pixie dust; it doesn’t have a sense of humor, is not sentient, doesn’t understand juxtapositions and, most importantly, doesn’t evaluate context. As another example, I asked a GPT-3 tool to provide biographical information about two individuals. Abraham Lincon was the first. Approximately ten paragraphs of accurate facts were generated by the GPT-3 tool. Second, I used Gene Kim, the author of The Phoenix Project and The DevOps Handbook. GPT-3 began inventing facts about the companies he worked for by the second paragraph. Additionally, it suggested that he had spoken at TEDx. While a Dr. Gene Kim has spoken at TEDx, that Dr. Kim is an opthamologist.

History of GPT

GPT was released for the first time on June 11, 2018. Sometimes it is referred to as the GPT-1 model. A large dataset of online books was used to train it. A database called BookCorpus also was used for training. There were 7,000 unpublished books in the original BookCorpus dataset at the time of training.

OpenAI’s GPT-2 model was created in 2019 as a direct scale-up of the original GPT model. There were ten times as many parameters and ten times as much data in that training set. The GPT-2 dataset was based on an internal OpenAI dataset called WebText. A focus was placed on the quality of the documents when scraping web pages for this tool. The authors scraped all Reddit outbound links receiving at least three upvotes. GPT-2 used about 40GB of Internet text, 8 million web pages, and 1.5 billion parameters. In neural networks, parameters are often used as synonyms for weights (found more often in statistics literature). Parameters in GPT are the weights and biases of the dataset and generally indicate the power.

As the third-generation language model, GPT-3 was introduced in May 2020. It has 175 billion parameters, making it more accurate than its predecessors—an increase of tenfold in capacity over the next largest NLP model available. Sixty percent of GPT-3’s weighted pre-training dataset is filtered Common Crawl. Common Crawl is a nonprofit organization that crawls the web and makes its archives and datasets freely available. Since 2011, Common Crawl has collected petabytes of data. About 37% of the GPT-3 sources were WebText2 and other sources, while about 3% came from Wikipedia. OpenAI Codex was also included with GPT-3, which generated code and served as the basis for GitHub’s Copilot.

As part of its training, GPT-3 used data available until November 2019. Thus, GPT-3 was unaware of events beyond January 2020. COVID-19, for instance, was unknown to it. Even so, it did have extensive knowledge of coronaviruses, pandemics, Wuhan and SARS. As of September 22, 2020, Microsoft, through a considerable investment, received exclusive permission to use GPT-3; others can still access GPT-3’s public API, but Microsoft is the only one with access to the underlying model. OpenAI developed DALL-E in January 2021, which generated images using GPT-3. OpenAI introduced models in the “GPT-3.5” series on November 30, 2022, including the now-famous ChatGPT, derived from a model from the GPT-3.5 series. Unlike GPT-3, GPT-3.5 includes data up until November 2022.

Current Status

I have used several AI language model tools in the past few years based on GPT-3. It is used by Grammarly and by some useful writing tools like Jasper.ai and Wordtune. I also have had great success with GPT-3 research tools like Frase.io and Genei.io. One of my more recent go-to tools is called QuillBot. In my experience, these tools are aids, not replacements for writing. When I first started using these tools, I was writing a book about Dr. W. Edwards Deming. I would get inaccurate information about Dr. Deming’s childhood from a GPT-3-based tool. One time, it said he had six siblings. In fact-checking, GPT-3 confused Dr. Edwards Deming with Dr. Richard Deming. With surgically precise input, the results could be more accurate. One of my findings was that if I used a GPT-3 on a well-known subject, the information was primarily precise; however, lesser-known topics produce fewer facts. Furthermore, I learned from the documentation that when people talk about training GPT-3 models, what they really mean is that you can input a maximum of 2,048 linguistic tokens, which is about 1,500 words. In GPT-3, content is predicted by analyzing past context, one word at a time. The content you input to GPT-3 significantly impacts its output. As a result, it pulls words from a shortlist and calculates their probability of being used. A great set of tutorials on using GPT-3 can be found on Jasper.ai. This figure shows how Jasper (which is based on GPT-3) word prediction works.

Image source

People have accomplished some remarkable things by using the 2,048 tokens creatively, but complex training still needs to be improved. Due to the Microsoft deal, the original GPT models were close sourced, and robust training of GPT-3 models is unlikely to happen any time soon.

Future Effects on IT

Where does all the hype come from? Despite its limitations, GPT-3 has created some unique industry artifacts. DALL-E and Copilot by Github are excellent tools. On September 8, 2020, the Guardian published an article titled, “A robot wrote this entire article. Are you scared yet, human?” This article was written by GPT-3. Using GPT-3, some full-blown novels have also been written; they’re not horrible, but they could have been better. According to a study conducted by Drexel University in 2022, systems based on GPT-3 could be used to screen for Alzheimer’s disease at an early stage. Just five days after its launch, ChatGPT surpassed one million users. It took 41 months for Netflix to reach one million users and 10 months for Facebook to reach one million users. By the end of the first quarter of 2023, ChatGPT will have over one billion users. Based on GPT-3.5, ChatGPT is more robust and up-to-date.

How smart is ChatGPT? Using a verbal-linguistic IQ test and Raven’s ability test, ChatGPT has an IQ of 147. GPT-3.5 has passed the United States bar exam, CPA exam and U.S. medical licensing exam. People often ask if it is plagiarism to use GPT-3. Technically, the answer is no. The GPT datasets are estimated to represent 300 years of pre-training with a connection of trillions of words. However, using these tools for rewriting other people’s work is, from an ethical viewpoint, plain and straightforward plagiarism. A recent set of tweets by Greg Brockman, president and co-founder of OpenAI and Sam Altman, CEO of OpenAI, sums up the current status of ChatGPT.

What Does This Mean for DevOps Practitioners?

AI learning models based on GPT-3 are here to stay. From grammar correction to proposal writing to code generation, GPT-3 has revolutionized everything. In my opinion, GPT-3 still needs to solve some challenges before it is of actual value to DevOps and DevSecOps patterns and practices. For example, a great use case for GPT-3 is training it to deal with organizational incidents. If a GPT model could be trained this way, it would become an excellent tool for providing adaptive diagnostics. Based on the current GPT implementation, this is very difficult or impossible. Another example would be to train the model with the organization’s IT risk policies and enable predictive solutions. The opportunities are endless.

An alternative to GPT-3 is InstructGPT, which OpenAI describes as more truthful and less toxic. It has also been noted that GPT-3 can generate untruthful outputs. GPT models aren’t generally aligned with their users. InstructGPT models follow instructions much better than GPT-3. Furthermore, InstructGPT produces more truthful and less toxic outputs. Therefore, they generate fewer harmful outputs and more facts. The InstructGPT models use human feedback as part of the training model. Some of the real power behind ChatGPT is that it uses InstructGPT along with GPT-3.5. Some of my upcoming research will focus on prototyping some of the complex opportunities DevOps and DevSecOps practitioners currently face using InstructGPT.

January 24, 2023
Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration
Kubernetes continues to grow as the leading container orchestrator for high-performance, cloud-native DevOps, DevSecOps and site reliability engineering (SRE) use cases. While Kubernetes offers excellent orchestration capabilities, it does not, by itself, assure cloud computing costs are optimized.

The current state of cloud expenditures for many organizations indicates a lot of waste. So much so that there is a trend to replatform from the cloud to private infrastructures to save money—escalating cloud costs are too high and the deep level of real-time optimizations are too hard and too slow.

According to an article from VentureBeat, Cloud computing: Cost at scale, “Large enterprises have started exploring cloud repatriation. What is worse, the forecasted trends are that “cloud spending could go up as high as 80% of the cost of the revenue for software companies. Moreover, the $100 billion market value among 50 software companies is lost due to the impact of the cloud on their margins.”

An article by CAST AI indicated 60% of cloud costs could be eliminated by being smarter about the use of cloud resources and augment Kubernetes with AI tooling, no matter the size of the deployments. Given the large number of human and automated users associated with DevOps, DevSecOps and SRE use cases for enterprise cloud resources, the savings potential can equate to tens of thousands of dollars per month.

However, typical optimization methods are not fast enough nor smart enough to process the resource changes quickly enough to achieve these optimizations at speed. An AI-assisted Kubernetes orchestration is needed to achieve this.

Kubernetes Landscape

According to 2021 State of Cloud Native Development Report, Kubernetes is used by 31% of all backend developers. Currently, 5.6 million developers use Kubernetes. The edge computing category has experienced rapid growth in the adoption of Kubernetes and has the highest usage rates of both containers and Kubernetes at 63%.

A report from the Cloud Native Computing Foundation (CNCF) indicated that spending on Kubernetes, the primary means of orchestrating containerized applications, was up year-over-year from 2020 to 2021. Of those surveyed, 67% reported an increase of 20% or more over the last 12 months, and 10% spent more than $1 million per month on their deployments. Fewer than 25% accurately predicted how much they’d spend to within 5% of the actual cost. And over 20% could not predict their cloud bill. That’s no way to run an operation.

Cloud Cost Use Cases for DevOps, DevSecOps and SRE

DevOps, DevSecOps and SRE have many use cases for orchestrating cloud computing. The list of computing tasks below is only a partial set of computing resources that need to be orchestrated for every value stream.

• Communication—emails, chats
• Experimentation, planning and research-related computing tasks
• Configuration tasks associated with infrastructure-as-a-service, platform-as-a-service, software-as-a-service and serverless.
• Version and artifact repositories transactions
• Application and microservice transactions
• API transactions
• DevOps administration tasks
• Developer pre-commit builds and testing
• Continuous integration commits, builds, scans, tests, and tools
• Continuous delivery—pre-prod tests and deployment development
• Continuous deployment automation and deployment strategies
• Production operation, backups, rollbacks, production testing
• Replication of hybrid cloud and multi-cloud instances
• Software repositories, data lake transactions
• Monitoring, logging, synthetic monitoring
• Tracing, metrics, collectors, analytics and dashboards
• Release and value steam management
• Anti-fragility—Chaos engineering, disaster recovery, fire drills, security red/blue/purple teams
• Diagnostics, issue reporting and tracking
• On-call support and retrospectives
• Documentation and training (computer-based) transactions

Sources of Waste and Potential Savings

According to the State of Kubernetes Report: Overprovisioning in Real-Life Containerized Applications, 37% of cloud compute capacity is not used. The primary reasons behind cloud waste are:
- Lack of visibility into cloud usage and costs
- Overprovisioning
- Leaving cloud resources idle
- Fragmentation of usage between teams and departments
One of the major drivers of wasted cloud spend is the inability to scale back once application demand drops. By eliminating this sort of overprovisioning, organizations stand to reduce their monthly cloud spend by almost 50%. By adding spot instances to the mix, organizations could cut their cloud spend by 60% on average.

Complexities Impacting Ability to Optimize

According to the article Cloud Cost Management for DevOps, “When properly done, cloud cost management can truly save organizations significant amounts of money. But, there is a learning curve and traditional approaches cannot keep pace with the rapidly changing requirements for DevOps, DevSecOps and SRE use cases.”

Trial and error are needed to identify optimization challenges that DevOps team members can anticipate. Before you can even begin planning around cost optimization, you need an accurate picture of your current infrastructure and spending landscape. Cost attribution cannot be done by simply opening your AWS bill. Cost data from monthly bills is not granular and cannot attribute costs to projects and teams. Furthermore, there are additional expenses that are included in your bill that are not tied to your cost of goods sold, such as research and development expenses.

Accurately allocating costs requires accurate and disciplined tagging infrastructure. The act of tagging infrastructure requires a fair amount of manual work for each piece of infrastructure in each sub-account. Enforcing tag hygiene can be a challenge of its own.

Opportunity for an AI-Assisted Kubernetes Orchestrator

Overcoming the complexity of optimizing cloud compute resource orchestration for the many uses cases for DevOps, DevSecOps and SRE requires a smarter, more responsive approach than traditional approaches that require manual planning and implementation. There is an opportunity to save an average of 60% of cloud spend by using an AI-assisted Kubernetes orchestrator.

What This Means

While Kubernetes offers excellent orchestration capabilities, it does not, by itself, assure cloud computing costs are optimized for the many use cases required for DevOps, DevSecOps and SRE. An article by CAST AI indicated 60% of cloud costs could be eliminated by being smarter about the use of cloud resources, no matter the size of the deployments. Given the large number of human and automated users associated with DevOps, DevSecOps and SRE use cases for cloud resources in enterprises, the savings potential can equate to tens of thousands of dollars per month.

An AI-assisted Kubernetes orchestration capability is needed to process cloud resource changes quickly enough to achieve optimizations at the speed needed for DevOps, DevSecOps and SRE use cases.

This article explained the landscape of requirements for an AI-assisted Kubernetes orchestrator to optimize cloud costs for DevOps, DevSecOps and SRE use cases. Future articles will discuss a solution for such an orchestrator and detail specific applications of the AI-assisted Kubernetes orchestrator for DevOps, DevSecOps and SRE use cases.
January 24, 2023
8-Bit Floating Point for AI/ML? | Amazon and Microsoft Shed Tech Jobs

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: New ideas bring low-power ML inference, and more big-tech jobs are going. (more…)

January 19, 2023
Why MLOps Hurts: 3 Pain Points for DevOps Teams
Machine learning operations (MLOps) is a practice that aims to improve the collaboration and communication between data scientists and IT professionals in the development, deployment and maintenance of machine learning models.

MLOps aims to streamline the process of building, testing and deploying machine learning models by automating and standardizing the various tasks involved. This includes automating the training and testing of models and the deployment and management of models in production.

MLOps involves using tools and processes to automate the various steps involved in machine learning model development and deployment. This includes tools for version control, continuous integration and continuous deployment (CI/CD), and tools for monitoring and managing the performance of machine learning models in production.

By adopting MLOps practices, organizations can improve the speed and efficiency of their machine learning model development and deployment and reduce the risk of errors and inconsistencies in the process.

What Role do DevOps Engineers Play in MLOps?

In MLOps, DevOps engineers play a crucial role in supporting the development and deployment of machine learning models. Their responsibilities may include:
- Setting up and maintaining the infrastructure needed to support machine learning models, including hardware and software environments, data pipelines, and monitoring systems.
- Automating the various tasks involved in machine learning model development and deployment, including training, testing, and deployment, using tools such as continuous integration and continuous deployment (CI/CD).
- Monitoring the performance of machine learning models in production and troubleshooting any issues that may arise.
- Collaborating with data scientists to understand the requirements and constraints of machine learning models and to help them implement and deploy their models effectively.
- Managing the release process for machine learning models, including coordinating with different teams, conducting testing and validation, and rolling out updates and improvements.
Challenges to Implementing MLOps

MLOps is not without its challenges. Here’s what to consider when developing and deploying your machine learning pipelines.

Data Security

Data security is a key challenge of MLOps for DevOps teams because machine learning models rely on large amounts of data to function effectively. Ensuring the security and integrity of this data is critical to the success of the model and the overall MLOps process.

There are several risks to data security that DevOps teams need to consider in MLOps:
- Data breaches: Machine learning models often work with sensitive data, such as personal information or financial data. If this data is not secured properly, it could be accessed by unauthorized parties, leading to data breaches and potential legal and financial consequences.
- Data tampering: Machine learning models rely on the accuracy and integrity of the data they are trained on. If the data is tampered with or manipulated, it could affect the model’s performance and lead to incorrect or unreliable predictions.
- Data privacy: Machine learning models may work with data that is subject to privacy regulations, such as the General Data Protection Regulation (GDPR) in the European Union. Ensuring compliance with these regulations is critical to avoid fines and other consequences.
Model Deployment

Model deployment is a key challenge of MLOps for DevOps teams because it involves integrating the machine learning model into an existing application or system and setting up the infrastructure to support it. This can be complex, especially if the model is part of a larger application or system, and requires careful planning and coordination.

There are several challenges that DevOps teams may face when deploying machine learning models:
- Integration: Machine learning models need to be integrated into an existing application or system in a way that is seamless and does not disrupt the overall functionality. This can be a complex task that requires careful planning and testing.
- Testing: It is important to thoroughly test the machine learning model before it is deployed to production to ensure that it functions correctly and meets the required performance standards. This may involve testing the model on a variety of data and in different environments to ensure that it is robust and reliable.
- Performance: Machine learning models may require specialized hardware and software environments to support their performance. Setting up and maintaining these environments can be a challenge for DevOps teams.
- Scalability: Machine learning models may need to scale up or down depending on the volume of data and the demands placed on them. Ensuring that the model is able to handle different workloads and scale appropriately is a key challenge for DevOps teams.
Model Performance and Maintenance

Model performance and maintenance is a key challenge of MLOps for DevOps teams because machine learning models require ongoing monitoring and maintenance to ensure that they continue to perform well in production. This can be a complex and time-consuming task that requires close collaboration between data scientists and IT professionals.

There are several challenges that DevOps teams may face when it comes to model performance and maintenance:
- Model drift: Machine learning models can suffer from model drift over time, where the model’s performance begins to degrade due to changes in the data or the business requirements. This can be difficult to detect and can significantly impact the model’s accuracy.
- Data changes: Machine learning models rely on the accuracy and integrity of the data they are trained on. If the data changes over time, the model may need to be retrained or updated to ensure that it continues to perform well.
- Hyperparameter tuning: Machine learning models often have several hyperparameters that need to be set to optimize their performance. Identifying the optimal values for these hyperparameters can be challenging and the model may need to be retuned over time to maintain its performance.
- Model versioning: Machine learning models may need to be updated or replaced over time to improve their performance or to address issues. Managing different versions of the model and coordinating the rollout of updates can be a challenge for DevOps teams.
5 MLOps Best Practices to Help DevOps Teams

There are several best practices that DevOps teams can adopt to help overcome the challenges of MLOps:
1. Automate as much as possible: Automating tasks such as model training, testing and deployment can help streamline the MLOps process and reduce the risk of errors and inconsistencies.
2. Use version control: Version control systems such as Git can help track changes to the machine learning model and its supporting infrastructure, making it easier to roll back changes or deploy updates.
3. Use CI/CD: CI/CD pipelines can help automate the process of building, testing, and deploying machine learning models, improving the speed and efficiency of the MLOps process.
4. Monitor and log model performance: Setting up monitoring and logging systems can help DevOps teams track the performance of machine learning models in production and identify any issues that may arise.
5. Collaborate and communicate effectively: MLOps requires close collaboration between data scientists and IT professionals. Establishing clear communication channels and effective collaboration processes can help ensure that all parties are working towards a common goal.
Conclusion

In conclusion, MLOps can be a challenging process for DevOps teams due to the complexity and specialized nature of machine learning models. There are several pain points that DevOps teams may face when implementing MLOps, including the complexity of setting up and maintaining the necessary infrastructure, the difficulty of collaborating and communicating effectively with data scientists and the challenges of ensuring the security and integrity of data used to train and test machine learning models.

By adopting best practices such as automation, version control, CI/CD pipelines and effective collaboration and communication, DevOps teams can help overcome these challenges and improve the speed and efficiency of machine learning model development and deployment.
January 13, 2023
The Time for Automated Testing is Now

Skilled software testers continue to be in high demand, and it appears that trend won’t change any time soon. The current labor shortage and a looming recession is giving software development leaders good cause to think about how to attract and keep qualified personnel and, more importantly, enable them to work as efficiently as possible. Added to this scenario is a trend in the software development cycle toward shift left testing, which has put increased pressure on developers to take on testing responsibilities that often require taking on new perspectives and using new tools. Automated testing solves these problems by making it easier to learn how to build a consistent, reliable testing program in less time, requiring less maintenance.

Add to this the future promise of artificial intelligence or AI-augmented testing platforms and current machine learning (ML) capabilities, such as self-repairing of test scripts, and you have a platform that both levels the learning curve and accelerates efficiency. The sooner organizations can transition from manual to automated testing, the better they can withstand internal and external market fluctuations.

A Developer and a Tester Walk Into a Bar …

Traditional software testing is a manual process in which testers are employed to replicate users and are expected to bring that perspective to their work. As such, software testers have a very different mindset than software developers. Where developers are protective of their creative output, software testers are driven by a curiosity to probe the limitations of a system. It stands to reason that the qualities that make someone excel at either of these roles is, likewise, not the same. And yet, with the trend toward shift left testing and continuous integration/continuous development (CI/CD), the industry has blurred the distinction between coder and tester to the dismay of both sides.

Automated testing removes much of the monotonous work associated with repetitious test plans. It allows testers to focus on writing better scripts, extending the scope of their testing plans and scheduling more powerful load tests. From the organization’s perspective, once you have established a library of reusable test scripts, your testing program will be protected from future turbulence in the employment market, including recession and attrition.

Low-Code/No-Code

Regardless of the tool you use, code will be created if you are creating software — even test scripts. Ideally, however, a low-code/no-code tool can hide some of the complexity and promote best practices. Using a low-code/no-code tool can relieve a tester from creating test scripts from scratch by recording manual interactions and offering suggestions. In the case of shift left testing, a low-code tool can eliminate the need to learn entire testing frameworks while still enabling a developer to access code when desired. Automating testing also eliminates the human error inherent in repetitive manual tests. In short, test automation does not diminish the value of skilled software testers and developers; it increases it.

Placed in the hands of skilled testers, automated testing will only become more powerful, more effective and more critical to your development cycle. It elevates the skills of testers and makes them more productive, accurate and thorough. Test automation also makes for a smoother transition to shift left testing, allowing developers to spend less time testing their software and more time doing the creative work that makes them thrive. In a market where attracting and keeping top talent is only going to get more difficult, anything you can do to lighten the burden on your employees and make them more productive will be a big win for your business. So, the question is not whether you should transition from manual to automated testing but when.

January 9, 2023
Best of 2022: 6 Top Technology Trends to Watch

As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

Technology trends simply cannot be ignored in today’s dynamic environment. As the world enters a post-digital era, organizations are exploring disruptive technology trends. Artificial intelligence, machine learning, intelligent process automation, advanced analytics, and robotic process automation (RPA) are among the top DevOps technology trends to watch. According to a survey by Accenture, 94% of IT and business experts reported that emerging technologies have accelerated firms’ innovation pace in the last three years.

Winning This Decade in the Context of the Next Normal

This decade continues to witness further development in technology, as innovation transforms industries such as insurance, banking, and healthcare. It is crucial to look beyond seeking operational efficiency and decreasing cost — particularly as customer expectations, regulatory requirements and business landscapes are rapidly evolving.

Take a look at six vital technology trends that have gained momentum in today’s hyper-digital world:

Trend 1: Artificial Intelligence (AI), Machine Learning (ML) – Integrating Intelligence

Technologies like AI and ML have existed for some time, but only now have they started maturing from their perception of “interesting” to “organizationally effective.” Artificial intelligence and machine learning are digital transformation catalysts, and some of the capabilities that fall under this technology include pattern recognition, natural language processing and sentiment analysis. Artificial intelligence has become increasingly important because it provides companies with a competitive edge, is cost-effective and allows them to future-proof themselves.

Trend 2: Predictive Analytics – Harnessing New Opportunities

In this digital-savvy environment, it has become crucial for organizations to employ tools like predictive analytics. An advanced form of data analytics, it’s used to extract information and predict trends, behavior and activities with precision. It is increasingly being introduced across industries as it optimizes productivity, helps understand customers better, identifies revenue opportunities and areas of attrition. Financial institutions have already begun to deploy predictive analytics, helping them customize the product offerings, detect fraud and identify new opportunities for cross-selling.

Trend 3: Low code – Accelerating Innovation

Gartner forecasts that low code application platforms will be responsible for 65% of application development activity by 2024. Organizations are rapidly leveraging low code to cater to the growing needs of the digital-first customers, deliver mission-critical and enterprise-grade applications quickly and respond rapidly to unforeseen scenarios.

Trend 4: Intelligent Process Automation (IPA): Unifying Processes

Another technology trend that has paved its path in the digital world is intelligent process automation (IPA). IPA is a group of technologies that work together to organize and digitize the end-to-end business processes and the customer journey. The key drivers of IPA are the need for continuous improvement, retaining data consistency and bridging existing silos. The technology offers several benefits to an organization, such as reducing error rates, unifying operations, and accelerating processes, but more importantly, assisting in streamlining employee and customer-facing journeys.

Trend 5: Contextual Content Services – ECM: Leveraging Context in Content

Contextual content services, an evolution of traditional enterprise content management (ECM), play a key role in powering automation initiatives. The right tools in ECM like intelligent taxonomy and cognition can bring context to business processes. Empowering users to make smart decisions with relevant information ensures accurate data extraction, and mitigates risks by managing the overall content lifecycle.

Trend 6: Robotic Process Automation (RPA): Improving Productivity

Robotic process automation is rapidly becoming an integral cog in the wheel of the end-to-end digital journey of organizations. RPA can be the last mile of process automation — or in between — but is key to streamlining and optimizing customer-facing and back-office tasks. The use of bots helps automate repetitive tasks and liberate knowledge workers to focus on other value-added tasks. The capabilities of RPA offer several benefits, including faster turnaround times, process optimization, sustaining consistency and increasing employee productivity.

Future-proofing the Enterprise

A close eye on the different technology trends can help enterprises leverage the right tools to optimize their business processes and adapt to the changing market needs. Technologies like low code platforms, advanced analytics, AI, and ECM are already being implemented by some organizations to stay ahead of the competition. Ensure your enterprise is tapping into these technology trends to thrive in the next normal.

December 28, 2022
DevOps 2022: The Year That Was and Wasn’t
We will be wrapping up our DevOps coverage for the year by the end of this week. It’s hard to believe that another year has gone by. The gravitational lensing of COVID-19 makes it hard to see the progression of the last nine years here at devopsy.kinsta.cloud. Things seem to get a bit hazy looking back. However, 2022 was certainly one for the record books in ways both good and bad. 2022 saw DevOps, our community our countries and our world pushed and pulled in directions we would not have imagined just a few short years ago. To understand what happened in DevOps in 2022, we need to account for several macro factors that played outsized roles this year. They are:
1. COVID-19: It’s still here; still killing hundreds of people every day. While some in the U.S. and Western Europe have “moved beyond” it, COVID-19 is still draining our resources, sucking the air out of the room and impacting not just the economy but how we interact with each other individually. In the next few months, I am afraid we will see the 1.4 billion people of China experience a backlash as they move away from zero-COVID policies. Not to mention the other viruses and diseases that are circulating.
2. War: When I was a young boy, I remember a popular poster/sticker that said something like, “War is not healthy for children and other living things.” That’s still true; it’s not healthy for the economy, either (unless you sell ammunition). The human toll of the war in Ukraine is heartbreaking in the present moment, and the toll that this war will take on the once-healthy tech sectors in Ukraine, Russia and the rest of Eastern Europe also will be felt for a very long time.
3. Inflation: Again, I am transported back to my youth as inflation rears its ugly head and puts its foot on the throat of our economy. Sometimes I wonder if the Fed and other central banks won’t stop raising interest rates until the economy is good and dead, but their fear of inflation is almost genetic. While recent signs point to a potential peak, we still have a long way to go and the possibility of more interest rate hikes ahead of us. These will continue to make it harder to raise money and put a crimp in companies’ budgets. That means layoffs and budget cuts; projects cut or pushed out. We haven’t seen disruption like this in tech for more than 10 years. But for those of my vintage, we have seen cycles like this before. This too shall pass.
4. Hate: Is it me or has the whole world gone crazy with tribalism, nationalism, racism, antisemitism and a bunch of other “-isms”? We seem to be ripping ourselves apart from the inside out. I don’t have a clue how to combat this on a macro level, but I pray every day that things get better.
With these four horsemen of the apocalypse riding among us, you may think that I am not very bullish on DevOps now and in the future. But you’d be wrong. Like the little boy in a room full of horse manure, I remain very excited because I know there must be a pony around here somewhere.

DevOps has continued to evolve, continued to mature and has become an indispensable part of today’s modern software factories. Like their own version of Marvel’s Avengers or DC’s Justice League, DevOps, along with cloud-native, SRE, Agile, CI/CD, DevSecOps and dare I say it, platform engineering, are wielded by IT teams around the globe to continue to push forward as we digitally transform our businesses and the very fabric of our lives. That is what gets me up and out of bed every day and makes me excited to come to work.

While some marketers are busy running around printing stickers and shirts proclaiming that DevOps is dead, nothing could be further from the truth. Look around—as far as the eye can see, DevOps is everywhere. As we explored in our DevOps Experience virtual event in October (the sessions are still there for you to watch on demand), DevOps is everywhere. It is at the edge; it is in the “connected things”; it is inherent in technologies like WebAssembly (Wasm) and DevOps is distributed.

In 2023, I think DevOps at the edge, the DevOps of things and Wasm will be huge areas of growth. I realized that some of these folks claiming that DevOps is dead are trying to say platform engineers will replace DevOps engineers. But like Darth Vader trying to strike down Obi-Wan, all they are doing is making DevOps stronger than ever. You see, DevOps was never about DevOps engineers; it’s not about one role or one individual. In fact, some of my best friends helped kick off the DevOps movement and they’d react quite loudly when they heard the term DevOps engineers. DevOps is about building, maintaining and managing our software and applications. It’s a way of thinking about how we can go faster, be smarter and achieve better quality. It’s not about a particular tool you use or the role you play on the team, it’s about the fact that you have a team.

So, what does 2023 have in store for DevOps? More of the same, I think. DevOps, along with all of the associated frameworks and technologies, will continue to allow us to do more and do it faster and better. As I think about the coming year, I am very excited about the potential for ML and AI and I think they are going to have an outsized role in allowing us to be more successful. I think the novelty of these technologies has worn off and we will get to the productivity phase. I remember interviewing Luke Kanies years ago; he told me that one day software would write software. I asked him when he thought this might happen, and he said maybe 2035 or so. I think we are on track. Much like ChatGPT and Jasper’s impact on copywriting, AI won’t take the place of developers. But it will free them of some of the mundane, repetitive chores they do now.

I think we’ll also see an emphasis on feedback loops and iteration. I think automation, ML/AI will contribute to this in a big way. We have put so much of our DevOps emphasis on the development, integration and deployment of software. But observability allows us to see patterns and use cases as never before. We need to incorporate those findings into the next revision of our software. This is going to be a big story in 2023. That old infinity loop model for DevOps will rise to the top once again.

Thank you for all of the support for devopsy.kinsta.cloud this year and over the past few years. I hope this holiday season finds you and yours healthy and happy. Merry, merry. Happy, happy. See you in 2023.
December 20, 2022