Category: Social – Facebook
-
Latest DORA Report Highlights Importance of DevOps Culture
A 2023 Accelerate State of DevOps Report published today by the DevOps Research and Assessment (DORA) team at Google Cloud found teams with generative cultures composed of people who felt included on their team have 30% higher organizational performance than organizations without a generative culture.
The report was based on a global survey of 36,000 IT professionals and tracked deployment frequency, change lead time, culture, change failure rates and failed deployment recovery times.
Overall, the survey found that while just under a third of respondents were successful in terms of their ability to deploy on demand (31%) and keep remediation efforts after deployments to 5% (33%), only 18% had lead times for changes of less than one day, while only 17% had a failed deployment recovery time of less than an hour.
This is an area where DevOps teams appear to need the most amount of help; nearly two-thirds (64%) of software releases or updates required some level of remediation after deployment, according to the report.
Nathen Harvey, developer advocate for DORA and Google Cloud, said the one area most underappreciated when it comes to improving performance is documentation. High-quality documentation leads to 25% higher team performance, the survey found.
DevOps teams that rely on the public cloud also benefited from a 22% increase in infrastructure flexibility, resulting in 30% higher organizational performance, noted Harvey.
The survey also noted that, as far as reliance on artificial intelligence (AI) to automate DevOps workflows is concerned, it’s still early. But most respondents are, at the very least, experimenting with one or more use cases.
In general, the management of DevOps will soon be very different as more AI capabilities are leveraged to eliminate bottlenecks and reduce toil. The immediate challenge is developers are benefiting more from AI than software engineers, which is resulting in more code starting to simultaneously move through DevOps pipelines. There is little doubt that in the months ahead there will be increased reliance on AI to manage pipelines at greater scale. In the meantime, it will be increasingly difficult to manage codebases that are steadily getting larger as developers become more productive.
At the same time, more organizations are embracing platform engineering as a methodology for centralizing the management of DevOps workflows at levels of scale that previously were difficult to achieve.
Hopefully, the combination of platform engineering and AI will result in less overall burnout among DevOps teams. In the short term, the burnout issue may become more challenging to address as the amount of code being created increases.
In the longer term, the results of the DORA survey may be substantially different next year as AI and platform engineering are employed more widely. The current survey results are not substantially different than in previous years, in the sense that DevOps teams are still challenged by many of the same issues.
Of course, as important as it might be to improve those metrics, the most important thing any DevOps team should focus on is developer productivity which, despite the existence of DevOps pipelines, remains relatively low.
-
Something is Amiss in the Open Source World
You don’t have to be a psychic to grok that the winds of change are blowing hard in the open source world. For the last few months, I’ve seen hints and heard whispers about a new direction for what I believe is the most passionate and enthusiastic community in tech. I think we are about to sail off into unchartered territory—and it will be interesting to see what the repercussions are.
Of course, this will all play out against the backdrop of continued layoffs and belt-tightening in the broader tech sector. Before I detail what I think are the telltale signals of change, let me state that I am and have always been a big supporter of open source software. I firmly believe we live in a golden age for open source software, which I call the “Foundation era of open source.” At a time when 90% plus of organizations use open source software, many of the leading open source projects are under the domain of not-for-profit foundations (NFPs) such as the Linux Foundation, the Apache Foundation, etc. This allows multiple commercial companies who might otherwise compete with each other to work in coop-etition. This has, in many ways, enabled this golden age of open source. Prior to the Foundation era of open source, there were two prior dominant models. Initially, there was what I call the “Cathedral and Bazaar era,” named after the great book by Eric Raymond. This was a time of idealistic notions of “pure” open source, unsullied by commercial aspirations. To me, Dr. Richard Stallman embodied that time; if you were to make an analogy to history, it was akin to the 1960s era in the U.S.
From that era, let’s zoom to sometime in the early 2000s when we moved to what I call the “Big Brother era” of open source. In this era, open source projects were owned and managed by one vendor, for all intents and purposes. The project could have a vibrant community, but ultimately, it was the managing vendor or “Big Brother” who reaped most of the rewards. This made it difficult for multiple tool vendors to support the same project and stifled progress.
The dawn of the foundation era of open source changed all that, and we have seen transformational growth as a result. But nothing lasts forever—and I think the king of the hill may be on the way down. Why do I say this? Here are several events/trends I’ve noticed recently that lead me to believe this. Some of these things I was told in confidence, so I cannot reveal names to protect the innocent.
- The move away from “pure” open source licenses. Over the last few months, several pillars of the open source world have moved away from accepted open source licensing. Red Hat has done so with Red Hat Enterprise Linux (RHEL) and now Hashi has, as well. Now, to some, this change is subtle and it’s not a big deal. Only paying customers get access to source code. The overwhelming majority of users never look at the source code anyway. Nevertheless, this is a huge red flag for the community—the accepted licenses and a core tenant of open source, that source code is available to everyone, are no longer being used.
- Diminishing corporate financial support for open source communities and foundations. I guess this was to be expected in hard economic times. The path to profits from supporting open source foundations, projects and communities can sometimes be more of a dotted line than direct. And a dotted line doesn’t usually work when the bean counters are involved. In looking to cut expenses, these are low-hanging fruit.
- Closing foundations. This is a natural result of the economic strife. Foundations may operate as not-for-profits, but without operating funds, foundations can’t operate. Generally, foundations get a lot of mileage from volunteers. However, volunteers can only go so far before hard costs are involved. Also, volunteers are able to volunteer because their corporate employers empower them to do so. When their employers pull back the funding for those activities, the volunteers have to do what they can to earn a living. I have heard of at least one NFP foundation forced to cease or curtail operations due to corporate sponsor funds drying up.
- Orphaned projects. Again, this is a direct result of number two, above. With the foundations closing and supporting funds drying up, I know of at least two bellwether open source projects that are in danger of becoming orphaned. Both of these projects boast huge community numbers and dominate their markets. The fact is though, that supporting a major project costs hundreds of thousands of dollars. While millions of community members may use a project and support it, they don’t contribute dollars. Dollars that pay for hosting, infrastructure and other hard costs. Coughing up a few hundred grand to support a project is a bridge too far for some companies in the current economic climate.
What does this mean for open source software and you? Well, in the near term, if you are just a consumer of open source software, probably not much will change day-to-day. You may have to cough up some money if you want access to source code or support from the community. Bigger picture, though, is the winds of change are blowing. I think we are going to see more companies taking a Hashi-type approach to protecting their source code. This will result in more forks of open source projects. However, my experience is that most open source forks don’t succeed (with some notable exceptions—like Jenkins forked from Hudson). Companies who are building businesses around open source models will be the most at risk. I think an open core business model may be the best bet for an open source vendor. I think the future of NFPs as the stewards of open source may become a bit cloudy (no pun intended). What will take its place or fill the void? That remains to be seen.
For the record I was never a fan of that old Irish proverb, “May you live in interesting times.” This kind of upheaval is a little too interesting for me. But one thing I have learned in 30 years in tech (and more than that in life) is that change is constant. You can embrace it and thrive or shrivel and die.
-
Moving Beyond Legacy API Management Tools and Strategies
What is legacy API management? That’s a question that many organizations haven’t even thought to ask because they don’t realize that the way they manage their APIs is outdated. On the contrary, they assume that if they have API gateways in place to provide security and load balancing for their APIs, they’re already ahead of the curve when it comes to managing and protecting their APIs.
But the fact is that traditional approaches to API management are fast becoming outdated. In a world where the ability to communicate in real-time is more critical than ever, legacy API management solutions can’t keep up, because they’re not designed for APIs that need to share large volumes of data using asynchronous paradigms.
That’s why forward-thinking organizations should be looking for ways to overhaul their API management tools and strategies. They need to embrace next-generation API management solutions, which can do much more than manage traditional, synchronous APIs.
Allow me to explain by walking through the shortcomings of traditional API management tools and explaining what next-generation API management entails.
Legacy API Management
Legacy API management solutions are tools designed to manage and secure synchronous APIs. Typically, these tools come in the form of gateways that sit between backend REST APIs and applications that connect to those APIs using the HTTP protocol. The gateways manage traffic flows and secure communications between the applications and the API backend.
That approach works well if two conditions are true:
1. All of your APIs are synchronous, which doesn’t always work well with continuous data streams.
2. All of your clients can consume data from APIs using HTTP requests.Traditionally, these conditions were true for most API use cases at most organizations. That’s why the standard approach to API management is to deploy a gateway, configure some load balancing and security policies and call it a day.
Next-Generation API Management
Increasingly, however, not all data is shared synchronously, and not all applications can consume data using HTTP requests. Instead, businesses want to be able to stream data in real-time to multiple types of applications. They might use an open source event broker like Kafka to stream monitoring data from across their IT assets to the tools that IT engineers use to observe and analyze systems, for example. Or, they might need to stream sales transaction data to a fraud detection engine so it can catch fraudulent transactions in real-time.
For use cases like these, legacy API management doesn’t work well at all. Traditional API gateways aren’t very compatible with event brokers because the gateways are not designed to manage real-time data streams. Nor do they typically support protocols other than HTTP, which is a problem because event brokers like Kafka use their own protocols, not HTTP, to enable communication between servers and clients.
Next-generation API management solutions fill this gap. They operate as gateways that can not only mediate between traditional REST-based client and API endpoints, but also support event brokers, asynchronous APIs, and even apply policies at the message level to support message-based communication.
In addition, next-gen API management tools offer the ability to translate between different front and backend protocols. That means that you could have, for example, a Kafka stream on the backend that is exposed to clients using HTTP – effectively allowing apps that can only make requests via HTTP to connect to Kafka data streams, without having to build support for protocol translation into the applications themselves.
Thinking Beyond Internal APIs
At this point, you may be thinking, “We don’t expose data streams externally, so why would we need a next-gen API management solution to support them?”
That’s a fair point. The most basic use cases for data streaming platforms like Kafka involve streaming data internally – from one set of applications to another within the same IT environment, for example.
But this is certainly not always the case. There are plenty of circumstances where you might want to expose a data stream externally and require the security and reliability protections that next-gen API management tools deliver. Maybe you’re a manufacturer who wants to stream data to regulatory authorities so they can monitor your operations remotely, for example. Perhaps you use third-party monitoring or cybersecurity tools and need to stream data from your own systems to that software. Or, even if you are only exposing APIs and data to internal consumers, perhaps that data contains sensitive PII and additional security mechanisms are still necessary. And, for almost all use cases, being able to centrally expose, catalog, and make APIs discoverable is a major leg up.
Even if your business doesn’t currently expose data streams externally, by the way, there’s a good chance it will in the future. As Gartner notes, “Organizations are improving their decision intelligence and real-time applications by tapping the growing availability of streaming data.” The future for most organizations is likely to include more data streams and more complex use cases for those streams, making the functionality of next-gen API management increasingly critical to business success.
Conclusion
In short, it’s time to move beyond the confines of legacy API management – which means no longer being limited to supporting only REST API management. To capitalize fully on the opportunities enabled by streaming data, businesses need a more flexible and dynamic approach to API management, which requires a new breed of API management software.
-
Tips For Managing Internal Developer Platforms (IDPs)
Platform engineering has emerged as a discipline to centralize configurations and standardize the DevOps process. As part of this practice, internal developer platforms (IDPs) are a key ingredient to establishing a paved road or “golden path” for developer users. But how do we ensure these IDPs are designed and rolled out correctly?
Well, the IDP concept is still relatively new, but some best practices are beginning to emerge. For example, treating the platform as a product helps encourage proper feedback. Reference architectures can guide initial designs. And testing first iterations with a single team can encourage buy-in around a proof of concept.
I recently chatted with internal developer platform experts, including Luca Galante, product and growth, Humanitec, and Markus Eisele, senior principal product marketing manager, Red Hat, to gather insights on how to best manage IDPs. Below, we’ll consider how to create golden paths that help optimize developer productivity and happiness and consider how these techniques affect the bottom line.
Start With a Reference Architecture
First, it’s a good idea to begin with a reference architecture when creating an IDP. This can help frame the initial infrastructure and its constituent components. For example, McKinsey recently debuted a sample IDP architecture, which has been reinterpreted for Azure and GCP. Platform engineering also maintains a tooling landscape that lists tool options for the various planes.
It’s good to note that an IDP is not a single product, like a platform-as-a-service. “An IDP is not a turnkey solution,” said Galante. “You can’t buy an IDP.” Therefore, there is not a one-size-fits-all enterprise-grade option that covers the entire stack. An IDP will most likely be a combination of open source and proprietary components.
According to Eisele, these new reference architectures are a good step in the right direction. “These IDP reference architectures are developer-centered views about modern application development platforms,” he said. “They shift the perspective from a sheer infrastructure perspective into what matters most for the developer.” And while a one-size-fits-all platform may not exist, the tools to build them do, such as Spotify’s Backstage or Red Hat’s Developer Hub, a platform for building developer portals.
Treat the Platform as a Product
Treating the platform as a product is another commonly shared best practice in platform engineering circles. “This is something that will make or break an IDP initiative,” said Galante. According to Galante, this platform shouldn’t be architected and then taught to developers in a top-down fashion—this can result in lengthy onboarding, broken functionality and ShadowOps.
Instead, successful platform teams take an existing team’s work and restructure it around a product manager to evangelize, said Galante. Another option is to keep your infrastructure team and develop a separate product team to start from scratch. Regardless of the exact organizational makeup, the product mindset is crucial for kickstarting and maintaining IDPs.
Have a Tight Feedback Loop
Part and parcel of retaining a product mindset is conducting research and having a tight feedback loop with stakeholders. For IDPs, this means understanding what developers need and not building on assumptions alone. According to Galante, all stakeholders should be given a voice, including DevOps teams, developer senior backend developers, junior front-end developers and even executives. Listen to these groups to understand their needs to frame the IDP correctly and sell them on the value of a platform. This is where the skillset of a PM is crucial to navigate these kinds of dynamics, he added.
“IDPs are about enabling developers,” said Eisele. While these tools should make them more efficient, they should also help them love their job more despite the cognitive load, he said.
First Attempt With a Lighthouse Team
Before rolling out a standardized internal development platform company-wide, it’s good to validate it in practice with a smaller group. Galante calls this a “lighthouse team.” Ideally, this team is more advanced with their deployment pipelines and is well-versed with cloud-native technologies, he said. This should help establish the right golden path with minimum friction and stress test it under real-world scenarios. After an IDP is validated in practice, rolling it out across other teams will be much easier.
Centralize the Platform Team Itself
Another facet to consider is the organizational makeup. Who will architect and support the IDP? Galante believes the platform team should be centralized instead of having various platform teams alongside each department or division, and such a team should consult various stakeholders throughout the application lifecycle. He also references Team Toplogies as a helpful resource for implementing platform teams.
At the same time, Galante admitted that at scale, large enterprises might spin up multiple platform teams simultaneously. Sometimes, this is done intentionally and could even bring positive competition to improve operations, he said.
Plan For Flexibility
Lastly, the big takeaway is that IDPs are not a silver bullet, and a reference architecture that works great for small agile teams might not transition well into a complex enterprise with much brownfield development. Furthermore, you run the risk of over-specificity, too, which could step on people’s toes.
Therefore, the platform must be flexible enough to adapt to new situations. Galante recommends first looking at your configuration management to see where the most time is wasted and ensuring you standardize the most common configurations.
Setting Golden Paths For IDP Success
Developer productivity practices have emerged as a response to exploding complexity in the last ten years, said Eisele. The old days are gone, and a brand new execution stack is required to architect and deploy applications. “All this led to a point where developers just want to get back to coding,” he said.
The golden path, or paved road as Netflix calls it, is like an approved path for traveling through the software development lifecycle. As Eisele puts it, it’s a form of best practices and a new way to replicate development knowledge across an organization.
“IDPs seem to be natural partners to cloud/standardized developer environments (CDE/SDEs),” said Ivan Burazin, CEO and co-founder at Daytona. He pointed out how Uber’s developer satisfaction increased dramatically upon adopting a platform called DevPod, an integral part of their IDP. He also cites successful CDE implementations at AirBnb as further proof of the power of internal developer platforms.
But to create these golden paths, IDP architects have their work cut out for them. They must juggle the input from multiple stakeholders and simultaneously balance complexity and abstraction effectively. The platform itself must be specific enough to reduce developer cognitive load yet generic enough to be flexible for outliers.
Thankfully, the aforementioned tools and best practices should help efforts toward designing golden paths for your internal developers. The next step will be testing the platform and measuring the success of these initiatives.
-
Tricentis Launches Mobile Application Testing Grid
Tricentis today added a Virtual Mobile Grid service to its portfolio to make it simpler to test mobile applications at scale.
The service is based on the platform Tricentis gained via its acquisition of Waldo and is now integrated within the Tricentis Mobile application testing framework.
Tricentis CTO Mav Turner said the Virtual Mobile Grid extends an existing Tricentis Device Cloud service for accessing mobile devices to include the ability to author, execute and analyze tests that can be run across thousands of real and virtual devices.
In effect, Tricentis Mobile brings together Tricentis Testim Mobile, Tricentis Tosca Mobile and the Tricentis Device Cloud into a single platform, he noted.
DevOps teams can now test end-to-end at every layer of a mobile application using a platform that leverages neural network technology to test user interfaces. They can also build and run no-code tests to identify and resolve critical user interface and performance issues through real-time tracking of more than 130 key performance indicators (KPIs).
Organizations building mobile applications that must be deployed at scale and regularly updated are integrating testing platforms such as Tricentis within their continuous integration/continuous deployment (CI/CD) platforms. Many of those efforts are tied to digital business transformation initiatives that rely on mobile computing platforms to access a wide range of backend services.
There is a lot of debate over how far left application testing should be shifted. In theory, the more developers test applications themselves, the fewer issues there should be once an application is deployed in a production environment. However, organizations need to be wary of increasing the cognitive load on developers, especially to the point where they spend significantly less time writing code, noted Turner. There will still be a need for dedicated testing teams to conduct tests across multiple stages of the application development process to ensure quality, he added.
Testing mobile applications is more critical than ever because the cost of failure—in terms of impact on the business—is much higher as more organizations depend on software to drive business processes. The challenge is that many of these app developers don’t have a software engineering mindset, so they need testing platforms that automatically create and run tests on their behalf in a way that can be customized for specific use cases.
Over time, the quality of applications should steadily improve as many of the low-level mistakes developers make are eliminated. That should improve everything from application security to the user experience as applications are continuously updated.
Theoretically, as more artificial intelligence (AI) is applied to the testing process, the number of reasons tests are not run as frequently as they should be will steadily decline. Today, it is still too difficult for many developers to author tests, so they naturally don’t conduct as many as they should. The simpler it becomes to author and execute those tests, the more likely it becomes a test will actually be run before an application gets deployed in a production environment.
-
ForAllSecure Streamlines Application Security Testing
ForAllSecure today provided early access to dynamic software bill of materials (SBOM) generation and software composition analysis (SCA) validation capabilities within its Mayhem Security automated code and application programming interface (API) testing tool.
Josh Thorngren, vice president of product at ForAllSecure, said the dynamically created SBOM lists only the components present at runtime. That’s a critical capability because, given the rate at which most code is developed today, most SBOMs are out of date shortly after they have been created, he noted.
In addition, Mayhem Security will also now filter results from SCA, static application security testing (SAST) and other similar tools to only surface vulnerabilities that are actually present in the code at runtime.
That approach will substantially reduce the cognitive load that developers experience today when reviewing the results these tools would otherwise generate when applied to the entire code base, said Thorngren.
There’s been an ongoing effort to shift more responsibility for application security left toward developers. The challenge is many of the security tools being provided to developers present an overwhelming number of alerts that, upon investigation, don’t impact code actually running in a production environment, noted Thorngren.
As a result, developers become less inclined to run these tools and miss the chance to discover vulnerabilities earlier in the application development life cycle, he added.
Discovering those vulnerabilities is becoming a more pressing issue as more developers employ generative artificial intelligence (AI) platforms trained using general-purpose large language models (LLMs). While those platforms make developers more productive, they also generate code snippets that have the same vulnerabilities as the code that was used to train them. The result can be an even greater number of vulnerabilities finding their way into production environments than there already are.
At the same time, governments around the world are debating legislation that would make organizations around the world more accountable for application security. As that legislation becomes law, it’s only a matter of time before an exploited software vulnerability results in heavier fines and penalties. It may be a while before those proposals become law, but organizations will need to start shoring up their DevSecOps practices today to address those requirements.
It’s not clear whether developers will ever fully embrace security, but identifying and remediating vulnerabilities can become a lot less painful than it is today. At a time when there is more focus than ever on developer productivity, the less time spent remediating vulnerabilities, the more time there should be to write code. In effect, organizations should be able to adopt secure-by-design principles for application development, noted Thorngren.
Ultimately, any effort to reduce the number of vulnerabilities in applications will go a long way toward bridging the current divide between cybersecurity teams that view developers’ lack of appreciation for security as the root cause of many of the issues they face. At the same time, if it becomes easier to identify and remediate vulnerabilities, fewer developers will view cybersecurity as an obstacle to be circumvented.
-
LinearB to Provide Free DORA Metrics Dashboard
LinearB is making available a free dashboard through which they can visualize DevOps Assessment and Research (DORA) metrics from within the company’s software delivery management platform.
DORA metrics enable DevOps teams to track deployment frequency, mean lead time for changes, mean time to recover and change failure rates. The dashboard is currently available via an early access program and is scheduled to be generally available in late fall.
LinearB CEO Ori Keren said the challenge is that not enough DevOps teams track these metrics because there previously was a fee associated with running them. LinearB is now absorbing those costs to help DevOps teams that have adopted the company’s namesake software delivery platform to be more efficient.
In addition to the trailing DORA indicators of DevOps performance, LinearB’s dashboard makes it possible to track leading indicators such as merge frequency and pull request size. In effect, the ability to track DORA metrics should be table stakes for any DevOps team, noted Keren.
LinearB’s 2023 Software Engineering Benchmarks Report based on analysis of more than 3.7 million pull requests (PRs) from more than 2,000 DevOps teams found code reviews are the biggest bottleneck in a DevOps workflow. Best practices for improving code reviews include automating reviewer assignments, setting up automated alerts for changes to be made to deprecated application programming interfaces (APIs) and using automated labels to prioritize workflows. Not surprisingly, the report also noted that startups and smaller companies are much more efficient when it comes to managing DevOps workflows than the average enterprise.
In the meantime, the one issue that DevOps teams are already encountering with the rise of generative AI tools is that the amount of code that needs to be reviewed is exponentially increasing. Unfortunately, the pace at which that code is generated is faster than the pace at which AI is being applied to accelerate DevOps workflows. As a result, DevOps bottlenecks are likely to increase.
However, generative AI will soon be applied to surface recommendations for where DevOps teams can improve, for example, cycle times, noted Keren. Organizations that can effectively apply AI across the entire application life cycle will have a substantial competitive advantage as the pace at which software can be developed and deployed continues to accelerate, he added.
The most important DevOps metric, of course, remains developer productivity. While it’s important for DevOps teams to measure their own efficiency, all that effort is for naught if developers are inefficient. Unfortunately, most of them spend a lot more time managing development environments than they do writing code. The goal for many organizations is to enable DevOps workflows that result in developers being able to deploy higher-quality applications faster.
Each organization will naturally need to determine what level of DevOps maturity makes the most sense to strive for based on how critical software development is to their organization. Regardless, however, the one thing that is certain when it comes to DevOps is there is always room for improvement.
-
Why AIOps is Critical for Networks
Speaker 1: This is Techstrong TV.
Mitch Ashley: With great pleasure of being joined by Andrew Colby. Andrew is VP of AIOps at Vitria. Welcome, Andrew.
Andrew Colby: Good afternoon, Mitch. And thank you.
Mitch Ashley: It’s a great topic. I’m excited to talk with you about it. We could go down the share war stories in telco experience, which really could be about 10 episodes of a different show, but today in the telco environment, or just in the business environment in general, the economic conditions, competitive pressures, looking for areas where we can get more for less, there are a lot of different parameters that have shifted or changed or maybe tightened that we’re currently working within. I’d love to get your perspective on that.
Andrew Colby: Certainly, and thank you. Yeah, I’d say we see cautious optimism. Obviously, I’m based in the US in the DC Metro area, Maryland. And the US, the government entities and quasi-governmental entities have been tightening the economic structure in order to tame inflation. Fortunately, that has not driven our economy and had the potential recessionary effect that was feared, but people are still cautious, businesses are still cautious. That said, it’s hard to hire people and it’s really hard to hire technical people. So a lot of companies are continuing to look towards how to leverage technologies and automation to build efficiency so that they can do more with either the same number of people or re-task their people to higher value purposes, and let the technology do some of the more menial and mundane tasks.
And we can explore this a little bit, especially in these new complex service delivery and network environments. It’s very difficult for me to imagine how an engineer who’s gone through anywhere from two to eight years of college education is going to really be happy going and spending their days collecting a lot of data across network, container management VM and other infrastructure systems to figure out what’s going on. I mean, really that’s where a lot of the automation provides a significant amount of value to let the engineers do the smart, difficult things that we want humans to do.
Mitch Ashley: And a lot of pressures around meantime to recovery, even looking at resiliency, how do we stand up under a test-full situation, whether it be a security attack that might be going on or some unobserved condition that our systems and networks have never been under?
Andrew Colby: Oh, there’s so much of that. So much is changing. It’s not just a person like you or me behind a smartphone that can actually report that there’s a problem, but it’s sensors and equipment that won’t necessarily report right away, so it needs to be detected. So that’s a whole nother additional dimension that service providers, large enterprise IT organizations are under, which is to be able to have this kind of real-time awareness of what’s going on. Whether the service is real time, like the video conference that we’re on or not, there really is a desire and expectation to have real time awareness of the service delivery to be able to detect what’s going on, react to it, address it before the user, whoever that is, the customer, the employee detects a problem, identifies a problem, and actually reports it. That’s really kind of the last line of defense that you want to have.
Now, that said, when your customers are reporting problems, I think it’s really important to have a way to hear that, and that’s another thing that we are able to do with VIA AIOps in particular. We have customers that use that care event data, which is… It’s kind of messy data. It’s human generated, so it’s late in the timeline, but when the users are telling you something’s going on, you need a way to be able to hear that so that you can definitively react. I mean, I know for most people, there’s nothing more frustrating than having an interaction with your technical support teams that tells you, well, there’s really no problem. There’s nothing wrong or I don’t see a problem.
Mitch Ashley: That’s when customers feel like they know more about it than you do, right? I am experiencing this problem, help me please.
Andrew Colby: Absolutely.
Mitch Ashley: Well, I do want to hear some more about VIA AIOps. Just about can’t stumble out the door without tripping over AI in some form, maybe generative AI at the moment. So I’d love to hear some more about your approach around AIOps.
Andrew Colby: Certainly. I mean, it really starts with ingesting a wide variety of information, structured, semi-structured, unstructured information, enriching it, in some cases, classifying it and providing additional context around it. So we take this tremendous amount of data, the telemetry, the faults and others, and by enriching it, we actually grow it even larger and then need to be able to process it in a way that provides meaning for the business. Now, historically, you’d have expert engineers who would say, oh, well this value should be here and above that or below that, would be where we care about it.
There’s two things that are happening simultaneously. There’s too many measures. I was just working with a team turning up a pretty simple system on a container management system, a CMS, and they said, “Well, there’s like 2,400 metrics just in that one system,” haven’t even gotten really to the application measures yet. So tell me what’s important. So that’s one of the challenges, is just you can’t set the values for everyone anymore. And then the other challenge is to know which of them are important. Again, it depends a little bit upon how the service is consumed. Is it one way or bi-directional? Is it real time? Is it latency sensitive? Is it packet loss sensitive? What are the characteristics of it? And depending on those is going to be what’s important and then what’s normal. I mean, every service has some sort of pattern throughout the day and throughout the days of the week.
The machine learning, the unsupervised learning is able to learn what those normals are and then identify when those measures and values deviate from that normal and alert on it. And more important or equally important to just alerting on it, is to be able to bring together all of the related information. And that’s where the enrichment becomes important. It’s not just enough to say, oh, well this measure changed, but well, what is it related to? What are the services that run over that infrastructure? What are the other measures that have changed among those same infrastructure at about the same time? What were the planned changes or manual changes that went in, in the prior 15 minutes? Because maybe one of them had an impact and maybe it didn’t. But that’s really important and useful information to know. So there’s all this bringing together of the enriched information in order to get a full picture of what’s going on.
And that’s a lot of the value of the automation and the AIOps brings together. You don’t need to have your highly skilled, expensive engineers do a lot of that manual data gathering. Let the machine intelligence bring that together for you and let the engineers do the value that humans can do effectively, which is interpret that, identify new patterns, see what’s new, especially if you’re rolling out a new service or you have different behaviors going on. Now, there’s a wide variety of AI and ML that goes into this.
You mentioned generative AI, which is a very interesting topic and has a lot of both technical and popular coverage and attention right now. And there are places where that’s effective to be used for things like the data extraction and ingestion, which is learning what the data is and understanding how to interpret it, as well as potentially developing a suggested fix or likely cause or likely fix that could be generated from all of the different inputs and individual root issues that are identified. But there’s also a variety of other traditional machine learning and AI techniques that really provide a lot of value and are a part of VIA AIOps, and we leverage very extensively.
Mitch Ashley: In a way, I feel like networking, security, they’re very broad topics. In a few words, you’re talking about many, many different technologies. The same is true I think also for AI. There’s so many fields of it. It’s a bit overwhelming to folks. I know you prescribe kind of take an incremental approach. Don’t try to eat the whole elephant at one, if you will, that old adage. But what is that incremental approach? How do you work with customers to start to figure out that path that should go down?
Andrew Colby: That’s a great question. I’d say there’s a couple of steps to that. One is to focus on the business value. What is it that you want to achieve and how can you tell that you’ve achieved it? It’s not just enough to collect a lot of data and produce something that looks a little interesting, aha. Can you do something about it? Does it have an impact on the service? What is the measurable impact on the service? So being focused on business value and even directly measuring it is one important piece. The other is because this is all about network and service operations, it’s generally something that’s done across an entire service or entire company. It’s not so easy to try on maybe a departmental level. So it’s a big decision for companies. Because of the way VIA AIOps is built and structured, we enable it to be delivered or we enable delivery of what we call incremental transformation, which is the ability to augment the existing or augment the machine intelligence in AI with the institutional knowledge.
So to be able to specify policies, for example, for things that are… To differentiate what’s important for the business from what’s important in just a measure, to leverage the value of existing investments. Nobody’s starting this from scratch. So there’s always investments in application performance monitoring or other network and service monitoring tools. They’re not bad, but they just may be sort of siloed or they may have a real key piece of information in one area, and we want to leverage that across the entire service delivery. So that’s another way to provide that incremental transformation and leverage. And overall improving the efficiency of the operation staff and being able to deliver these really as individual use cases incrementally to continue to provide business value over time.
Mitch Ashley: You mentioned automation before, you talked about metrics and things to be what you should be observing. It all starts with getting the data and as you mentioned, bringing, it’s not just correlation anymore, it’s not correlation of events. Yes, we do that or we need to continue to do that, but if you have at 10 factors that are actually part of what’s going on, that’s usually bigger than what someone looking at a monitor or a screen up in an op center is really going to be able to put up together. And this seems like that complexity or maybe the speed of that happening is also a big driver for where you might consider AI. Agree? Disagree?
Andrew Colby: Absolutely. I mean, the kind of highest level metrics are things like the meantime to understand, MTTU, and meantime to restore, MTTR. Those are top level metrics and you want to build down and drive down from those, which is what goes into them. What does it take to understand a problem? In one of our customers, the challenge was not only to understand the problem, but to identify which part of the network the problem was in, so they can get to the right fix agent quickly. Because sometimes you create, in an incident management system, a ticket, but if it goes to the wrong team, they have to triage it, evaluate it, and they say, “Oh, it’s not us. It has to go to…” Maybe it goes to the firewall team instead of the network, instead of the WAN team.
So that’ll take a lot of time. So getting the likely root issue and likely fix to be in the right area is really important to be able to do that. So it’s areas like what are the components that drive that MTTU? How do you measure it? We have a variety of customers and they take a variety of different approaches. In the end, what they want to achieve is a higher percentage of incidents that are handled through automation. And you can do that by decreasing the overall number of incidents as well as by increasing the number that are handled through automation, and we’re able to take both approaches.
Mitch Ashley: It goes back to what you were originally talking about, not being able to hire enough people, the right people, they may not exist, right?
Andrew Colby: Absolutely.
Mitch Ashley: The people we’d like to have is some purple unicorns, but that’s the people don’t always have those skills we’re looking for. So it isn’t always just about downsizing people. Sometimes it’s bringing that curve of what you need to hire right-sizing that just to get… So you can handle the workload with the staff that you have.
Andrew Colby: Right. And some of that is to be able to free up the staff from things like monitoring screens or systems that are just telling you red and green or up or down because that doesn’t have enough context, but you need to understand. And often it’s not just simply red or green, it’s gray or purple, which is, well, it’s working, but it’s not working to the level that we want or need or expect in order to provide the level of service that our customers expect or that our underlying service requires. So it’s being able to provide all that nuance as well as that level of detail and insight, all that enriched information. We go back to that again so that the right action can be taken. Eventually, once the right action gets taken 10 or 50 or 100 times, my expectation is that the trust will have been built in the system so that that action can be now taken in an automated fashion.
And again, that’s an opportunity to accelerate that time to free up an engineer from doing something that they’ve done 98 times before and be able to more quickly allow the action to be taken. Obviously, you’d still maybe want to have some post analysis review to see, well, did we take the right action? Is that same action being taken every day? Maybe there’s some other problem that really we need to address, but still, as long as they can deliver the service in a way that meets the customers’ and expectation and the service expectation, they’re achieving their goal. And they can enable engineers and others’ network operations to kind of free up to think at the higher level about what they need to do and can do to continue to achieve that effectively.
Mitch Ashley: Yeah, we live in a world where it’s very easy for customers to say, your service is taking too long, I’ll jump to the next app or the next site or whatever. There is loyalty, but there’s also patients that tries that.
Andrew Colby: Absolutely.
Mitch Ashley: That trust and loyalty. [inaudible 00:16:54] few minutes left. I’d love to have you talk some more about the incremental approach. I think all the buzz about generative AI, and you hear everything from, we won’t need programmers. We won’t need these kind of people, we won’t need those kind of people. We have robotics in factories, but we still need people in factories, right?
Andrew Colby: Right.
Mitch Ashley: I’m not a subscriber to the… The pendulum doesn’t slam against the other side all of a sudden, those kind of changes don’t happen very often. So it’s probably somewhere in the middle where we end up. So knowing that, if you take that assumption, how do you get aggressive enough that you’re getting some value from AIOps, and I mean so conservative that you’re kind of missing the opportunity?
Andrew Colby: I’d say again, it goes back to aligning your actions with the business values you want to achieve. I’m working with a customer who says they want to achieve a value of running their operations with significantly fewer people, like maybe half or more, less than a traditional network operation center would have in a circuit switch world, let’s say. That’s the goal from the top down. The people that are responsible from the bottom up, they’re like, “Whoa, slow down. Hold on. Don’t do all of your automation yet. We want to look at everything first. We want to see.” Because that’s how they’ve been used to dealing with, and that’s some of the place where there’s tension in being able to do this when the tests are done and you measure response time from issue occurrence to issue detection, to issue resolution, there’s a lot of human think time in there.
And it’s not wasted. It’s people doing their due diligence. Is it really down? Can I restore it myself? Do I need to take an action with an outside team? But those all need to be aligned, and that’s how you can achieve a result. And that’s hard in a large organization. And in all of these cases, we’re talking about large, complex organizations with large, complex service delivery environments.
Mitch Ashley: And you mentioned earlier the trust that’s built in that process too. We don’t just throw AI into the mix and say, it’s in charge now. Let’s walk away and hope it does all right. No, we need to know that-
Andrew Colby: We need to build trust.
Mitch Ashley: … what it’s going to do is the right thing, right?
Andrew Colby: Right. We need to build that trust. And that’s actually one of the reasons that, again, we go back to this incremental transformation. We think that’s really important to be able to do it in steps so that you can see, all right, this is what the system shows and we think the action from that system should be B. So we can give you a button to say, all right, when you believe it’s B, click on B. And again, you do that 50 times. After a while, you’re going to get tired of just saying, well, every time it’s B, why don’t you just do that for me automatically? And that’s where we want to get to.
Mitch Ashley: It’s kind of supervised learning for AI. Very much so. Well, Andrew, it’s been fascinating to talk with you. I hope you’ll get a chance to come back and chat some more. And I really appreciate you sharing sort of on the ground experiences with customers as you’re working in a large setting. It’s one thing to adopt AI in a startup, it’s another in a large telco environment that generally doesn’t make those kind of shifts. On an hourly daily basis, it takes a bit to get that flywheel turning in the new direction that you’re headed.
Andrew Colby: Absolutely.
Mitch Ashley: So, where can folks find out more about VIA AIOps?
Andrew Colby: Thanks, Mitch. Please go to vitria.com on our homepage, and on the Resource tab you’ll find a suite of information about how you can actually realize this business value and the types of capabilities Vitria and AIOps can deliver.
Mitch Ashley: Great. There’s a lot of great resources under that Resource tab, so be sure and check it out. Thanks for joining me again, Andrew Colby, who is a VP of AIOps at Vitria.
Andrew Colby: Thank you, Mitch.
Mitch Ashley: Talk to you again soon.
Andrew Colby: Look forward to it.
-
Best Practices for DevOps in the Hybrid Cloud
In rеcеnt years, various tools, technologies and frameworks have еmеrgеd with thе goal of enhancing flеxibility, performance and scalability. Traditional monolithic approaches have been replaced by more modular methods like microservices and nano-services. Additionally, the rise of cloud computing has led to on-prеmisеs software being replaced by cloud еnvironmеnts that offer a wide array of advantages and capabilities that were previously unavailable.
Bеing able to optimize spееd and efficiency through the adaptation of cloud settings is a vital capability. To leverage cloud-bursting capabilities and also safeguard their applications and data, businеssеs can embrace a hybrid cloud strategy combined with DеvOps. In this article, we will dеlvе into bеst practices for implеmеnting DеvOps in a hybrid cloud sеtting.
What is a Hybrid Cloud Environment?
A hybrid cloud, which combines on-premises and cloud computing modеls, allows you to harness the benefits of both approaches. Hybrid cloud computing allows businеssеs to blеnd their own on-prеmisеs infrastructurе with thе powеr and flеxibility of cloud computing. This approach allows them to optimize their current infrastructure while also benefiting from the scalability and cost efficiency offered by public cloud environments.
By lеvеraging thе strеngths of both, companies can optimizе their opеrations for scalability and flеxibility and еnhancе sеcurity for thеir applications and data. One notablе advantage of hybrid cloud еnvironmеnts is their ability to adapt rеsourcе capacity to mееt еvolving workload dеmands rapidly.
A hybrid cloud offers flexibility and cost optimization to businesses. With a hybrid cloud, enterprises can scale up their resources and adapt to changing needs, much like a public cloud. At the same time, it allows them to retain control over sensitive data and critical applications stored on-site.
By adopting a hybrid cloud environment, organizations gain enhanced data control and the ability to flexibly allocate workloads based on evolving business needs. Organizations can achieve improved flexibility, enhanced security and regulatory compliance by adopting a hybrid cloud model.
What are the Benefits?
Here are the benefits of a hybrid cloud:
Cost-effectiveness: The cloud offers a cost-effective solution for deploying workloads. When using the cloud, an organization does not have to spend much money to purchasе additional infrastructure. Moreover, most cloud environments provide support for a pay-as-you-go pricing model, which implies you only have to pay for what you consume.
Avoids Vendor Lock-in: Companies often face the challenge of becoming too dependent on one cloud provider during the transition to the cloud, making it difficult to switch to another provider if it becomes necessary to do so. However, by adopting a hybrid model and specifically migrating to a platform that supports multiple providers, companies can mitigate this risk and avoid being locked into one vendor.
Improved Developer Productivity: By deploying a hybrid cloud platform, developers will be able to significantly enhance their productivity. With this approach, developers can leverage Agile and DevOps methodologies more efficiently and deploy their applications across multiple cloud infrastructures simultaneously.
Infrastructure Efficiency: A hybrid cloud offers development and IT operations teams enhanced and smooth control over resources. Organizations can also leverage cloud services and combine them with data stored in the cloud or on-premises infrastructure to create more value through the hybrid cloud.
Better compliance and security: Hybrid clouds offer significant benefits for business growth and acceleration. They enable faster product development cycles, promote innovation and expedite time to market. Additionally, they facilitate prompt response to customer feedback and allow for the delivery of applications in closer proximity to customers. Hybrid clouds also facilitate the integration of new services from third-party providers.
Cloud-bursting: One valuable feature of hybrid cloud environments is called cloud bursting, which enables organizations to easily increase their resource capacity to meet varying work demands.
An Overview of Hybrid Cloud Architecture
A typical hybrid cloud architecture comprises of the following components:
On-Premises Infrastructure: This refers to your privately owned physical infrastructure that can be used for storage, network, host servers, etc.
Private Cloud: Generally speaking, a private cloud is an environment that is dedicated solely to the use of a single organization. It can be hosted either internally within the organization’s own premises or externally, but it is securely protected and not accessible to the general public.
Public Cloud: The term public cloud refers to a computing model in which services such as compute, storage, deployment, etc. is provided by third-party providers as a fully managed service over the internet.
Cloud Management Platform: A cloud management platform is responsible for managing the operation and allocation of computing resources across various cloud platforms. It also helps businesses effectively use their hybrid cloud environments.
Orchestration and Automation Tools: Streamlining tasks such as resource provisioning and data synchronization across different environments requires the use of orchestration and automation tools. These tools are crucial in ensuring efficiency and seamless operations.
Best Practices for DevOps in the Hybrid Cloud
Albeit the benefits it provides, successfully implementing DevOps in a hybrid cloud requires you to adhere to certain best practices:
Monitoring: Ensure your on-premises and cloud environments are monitored and managed effectively by using the right tools. These tools offer a comprehensive view of your resources, regardless of their hosting location.
Infrastructure-as-Code (IaC): Use tools like CloudFormation, Terraform or ARM to automate infrastructure provisioning and management. IaC ensures that infrastructure is built consistently and reliably.
Containerization: Container technologies such as Docker and container orchestration platforms like Kubernetes abstract the underlying infrastructure and make moving applications between on-premises and the cloud easier.
Centralized logging: Make sure you have a centralized solution for metrics, logs and traces (like ELK Stack, Prometheus and/or Grafana) that can collect data from both the platform and the cloud environment.
Automation: From CI/CD pipelines to infrastructure deployment, it’s critical to ensure that processes are automated by leveraging tools like GitLab CI, GitHub Actions and/or Jenkins.
Security: You should consistently implement a robust security strategy. Consider the unique security requirements of a hybrid system, such as security of data transfer between disk and the cloud, encryption at rest and in transit and consistent identity and access management.
Performance: Understand latency and bandwidth issues when transferring data between disk and the cloud. Choose the right architecture (such as edge computing) and optimize your applications to minimize these issues.
Data management and integration: You should understand where your data comes from and pay attention to data gravity. You should also link applications and data sources on the server and in the cloud to ensure smooth data flow.
Application architecture: You should design cloud-agnostic, modular and flexible applications. A microservices architecture often works well for hybrid cloud environments.
Disaster Recovery and Backup: Take advantage of both on-premises and cloud resources to establish a robust disaster recovery and backup strategy.
Administration and policy management: You should enforce consistent policies across both environments using policy-as-code by leveraging tools such as Open Policy Agent (OPA).
Training and skills development: Note that hybrid clouds require knowledge across multiple platforms. Ensure your team is trained and has the skills to manage and operate a hybrid environment.
Budget and cost management: You should monitor costs on the server and in the cloud. Tools such as AWS Cost Explorer, Azurе Cost Management and third-party solutions can provide insight into where your money is going.
Communication and collaboration: Although there are challenges and obstacles, you should encourage open communication and team collaboration.
Regular reviews and audits: Review your architecture, tools and procedures to ensure they align with business goals and are as efficient as possible.
What are the Challenges?
Despite its many advantages, hybrid cloud also comes with some challenges. These challenges arise from the complexities of managing multiple environments and integrating them together. Here are the key challenges of hybrid cloud at a glance:
Security Concerns: Hybrid cloud models can pose security vulnerabilities, particularly if there is inconsistent implementation of security protocols across different environments. It is important to properly secure data that moves between private and public environments to mitigate potential risks.
Data Transfer Costs: When moving data between private and public clouds, it’s important to consider the potential costs involved, particularly when dealing with large volumes of data. Being mindful of these expenses is essential when designing hybrid solutions.
Data Sovereignty and Compliance: You must ensure that data complies with regional and industry-specific regulations when it is stored or processed in multiple locations, including public clouds in different regions.
Latency Issues: One of the challenges that may arise when implementing cloud-based applications is the integration with existing legacy systems. This can be a problem, especially for applications that require real-time or high-performance capabilities.
Vendor lock-in: Vendor lock-in can still be a challenge even with a hybrid approach, especially if organizations heavily depend on proprietary tools or services from a specific cloud provider, making it difficult to migrate or integrate with other platforms.
Skills gap: One challenge in managing and operating a hybrid cloud is the potential for a skills gap, since you’ll need a diverse range of skills, such as knowledge of both on-premises and cloud technologies. To address this, organizations may need to provide training or hire new talent with the necessary expertise in both areas.
Monitoring: Achieving visibility and effective monitoring can be challenging when resources are spread across multiple environments. It often requires specialized tools or platforms to achieve end-to-end visibility and monitor effectively.
Governance and Policy Management: Although it is crucial for maintaining security and compliance, it can be challenging to implement consistent governance and policy management across both private and public clouds.
Cost Management and Optimization: While a hybrid cloud can provide cost benefits, it can also be difficult to track and manage costs across multiple platforms. This requires ongoing monitoring and the implementation of optimization strategies.
Network Connectivity: To achieve secure, reliable and fast connectivity between on-premises and cloud environments, a well-designed network infrastructure is essential. This may involve additional investments in network services or infrastructure to ensure seamless connectivity.
How Hybrid Cloud Supports DevOps Culture
Here’s how hybrid cloud supports DеvOps culture:
Automation: Automation plays a crucial role in both hybrid cloud and DеvOps. By utilizing tools and services provided by cloud providers, automation еnablеs the provisioning, scaling, and management of infrastructure. This aligns with the core principles of DеvOps, which include automated installation, integration, and utilization.
Flexibility: Unlike with a single platform or tool, DevOps teams have the freedom to select the most suitable services and platforms from various cloud providers and incorporate them into their own solutions.
Cost Efficiency: DevOps teams can achieve cost efficiency by using public cloud resources for temporary workloads like test environments or short-term projects. Your teams can optimize their costs by only paying for the resources they actually use, thereby optimizing the amount of money they spend on those resources.
Fast deployment of resources: The fast provisioning and de-provisioning of resources are crucial in a DevOps culture. Hybrid cloud allows for rapid deployment of environments, supporting the iterative and dynamic nature of DevOps practices.
Collaboration: Hybrid cloud solutions often include collaboration tools and platforms that promote effective communication and teamwork, aligning with the collaborative culture of DevOps.
Continuous Integration and Continuous Delivery (CI/CD): Hybrid cloud provides tools and platforms that support CI/CD pipelines and enable consistent and continuous integration, testing and transition to production (delivery/deployment).
Service Integration: Many cloud providers provide a wide range of services, including machine learning and databases, as part of their cloud offerings. Using these services will allow you to seamlessly integrate them into your applications, fostering a culture of continuous improvement and innovation.
Monitoring and feedback: Hybrid clouds provide robust monitoring, logging and analytics tools that offer valuable insights into application performance and user experiences.
Regulatory compliance: With a hybrid cloud approach, IT teams can maintain the security and compliance of sensitive data and applications by housing them in a private cloud or controlled environment. This way, organizations can meet their compliance and security standards effectively.
Conclusion
A hybrid cloud combines on-premises infrastructure with cloud computing, offering scalability, flexibility and improved security for applications and data. A hybrid cloud infrastructure should be more flexible than a traditional cloud to meet changing requirements. Therefore, businesses with hybrid cloud capabilities can adapt more quickly to business and technological changes.
Businesses can leverage several benefits by implementing DevOps practices in a hybrid cloud environment, such as improved agility, faster release cycles, enhanced scalability and better collaboration. With a hybrid cloud setup, companies can leverage the strengths of both on-premises infrastructure and cloud computing models to achieve scalability and flexibility.