devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • Platform Reality Has a Surprising Amount of Detail

    Platform Reality Has a Surprising Amount of Detail

    Developer platforms and the platform teams that create them have been the center of attention for quite some time now. There have been many (wildly different) opinions about the subject, ranging from how bad the notion of a platform team is to how essential platforms are to make an organization’s DevOps journey to cloud native successful.

    Since early this year, the CNCF TAG App Delivery team has been working on a whitepaper about platforms. It explains what platforms are, why they are needed and how platform teams might work to create them. Furthermore, there’s an abundance of end-user talks at conferences that show how platform teams and their platforms were essential in enabling developers at scale. Even on the KubeCon + CloudNativeCon keynote stage, we’ve seen success stories of platform teams like the one at Mercedes Benz.

    Developer Platforms Should Enable Developers

    There is no silver bullet to successful platform teams and platforms. However, it is important to understand that the main goal of a developer platform is to enable developers. It is essential for platform teams to focus on delivering that value, which is why we are seeing a trend to introduce actual product ownership and product management roles to platform teams.

    The CNCF whitepaper stated that the three core jobs of a platform team as follows:

    1. Research platform user requirements and plan a feature roadmap.
    2. Market, evangelize and advocate for the platform’s proposed values.
    3. Manage and develop interfaces for using and observing capabilities and services including portals, APIs, documentation and templates and CLI tools.

    I couldn’t agree more. But the reality of day-to-day business for many platform teams means they rarely get to focus on those three jobs.

    Reality Has Detail

    The situation reminded me of a blog post I read a few years ago, titled “Reality has a surprising amount of detail.” The author explained how it is easy for people to get stuck even when faced with a task that might look simple from the outside. They explain how a task as simple as building wooden stairs can, in its many subtasks, become quite complex. Furthermore, when faced with reality, even the tasks that seemed plannable get more complex and need real-life adjustments.

    Most engineers have had analog experiences in their day-to-day work. We can translate this notion into platform engineering quite well. Building a developer platform sounds straightforward at the onset. However, once we get into it, it becomes a rabbit hole of figuring out the CNCF landscape, adding company-specific glue, compliance, security, fixing bugs, getting deep into the open source projects and sometimes, faced with the reality of open source, even becoming contributors and maintainers of such projects.

    Now, I’m not saying that it’s all bad. All these things can be great and rewarding in their own way. Especially with a community like the CNCF, participating and contributing to open source can be a very rewarding experience. You’ll make lots of friends, learn DevOps and development skills and might even further your career significantly.

    While it’s a lot of work, there is a risk that every platform team basically reinvents the wheel. This is because an organization’s requirements seem too large to rely on a single product (or even a suite of products). We end up with bespoke platforms that need a lot of attention. These platforms also have platform teams that don’t have enough time to focus on the higher-level jobs that they were tasked with.

    Platform Evolution

    Over the years talking to and working with platform teams we have built a model of prototypical platform evolution and its risks at each stage.

    Usually, a platform team starts with the task of building a developer platform to enable the product teams to build great software and deliver value to the end users.

    As they build the platform from the first Kubernetes clusters to observability tooling to security, the backlog for features constantly grows. Even with a growing platform team and sub-teams for certain technology areas, there’s a risk that some of these backlog items never get attention. Especially, delayable, non-critical, daunting tasks (like upgrading to the next Kubernetes release) often fall victim to this problem.

    Next, when it comes to operations, the platform team needs to limit the possibilities and breadth of what they offer. Operations responsibility for functionality that is outside of those limits, (for example, a specialized data store) goes back to the product teams themselves, which results in developer distraction.

    Even with a wide coverage of the platform, Day 2 and operations, the next big task for the platform team is advocating for adoption. This is not just marketing for their platform, but also means the platform team has to build user-friendly interfaces and documentation. The risks and alternatives we often see here are that product teams build their own mini-platforms and systems that result in shadow IT. Another common risk is ending up with several competing platforms within big enterprises that are hard to integrate and centralize, which in turn leads to inefficiencies.

    Once we get over the adoption hurdles, it is the platform team’s job to deliver value and high-level capabilities to the product teams’ developers. However, at this stage, often the platform teams have grown and might encompass several sub-teams for technology areas. Additionally, some areas like security might already be under central IT control. The risk we see here is that the capabilities offered are reduced to technology silos. They might cover all areas like observability, connectivity, security and release engineering, but to achieve real maturity, a platform team needs to offer capabilities that span several of those areas.

    Many of the advanced capabilities that we see with more mature end users cover several or all technology areas. Think about a simple high-level capability, like progressive rollouts. It starts with release engineering capabilities to build and deploy the software. From there, we need connectivity capabilities to dynamically route traffic to the new service. Then we need observability capabilities to observe our new version and from there have automated feedback loops to roll back or go forward with the rollout to production. And if that wasn’t enough, as we’re a security-aware organization, we want the whole software supply chain that facilitates this to be secure and trusted.

    Question for the curious cloud native enthusiasts: How many tools do you need to integrate to achieve the above “simple” high-level capability in an automated way? And this is only one of many capabilities.

    PaaS-Like Platforms Without the Downsides

    So, while platforms and platform teams are definitely the way to go, we need to be aware of the amount of work and effort required.

    We need to be aware of the risks and strive towards building highly integrated platforms that deliver high-level capabilities to the product teams.

    However, cloud-native developer platforms, while having similar goals to the PaaS systems of the past, should not follow the same approach of full abstraction. If we just abstract away into bespoke systems that hide everything from the user, we go back to the world of PaaS where at some point the end-user developers hit limits and build workarounds around the platforms.

    Additionally, if we build these bespoke internal platforms, we’ll be stuck with them and need to maintain and evolve everything by ourselves.

    Or, to go back to the blog post I referenced earlier, “At every step and every level there’s an abundance of detail with material consequences.” And we have to remember what we care about. The way to avoid both of these downsides is to bet on open source and stand on the shoulder of giants.

    By building our platforms on open source and using as many of the established or soon-to–be-established standards of our community we can avoid our platforms becoming bespoke. Recent developments like Cluster API, Gateway API and sigstore are some good examples.

    Furthermore, we need to make our abstractions poke-able. While our platforms should come out of the box with user-friendly interfaces and sensible production defaults that enable new product teams to onboard quickly and get their jobs done without having to bother with deep understandings of the underlying technology, we need to keep them open to mature users that know their way around and want to tweak it to their needs. We achieve this by giving mature users the option to access said open standards and APIs safely.

    This is the strength that we gain from a community like the CNCF, so we should use it to build great platform products that stay true to the core of open source and strive to make our product teams as happy and successful as possible. And remember to be happy with ourselves as successful and appreciated platform teams.

  • DORA Report 2022: The Magnitude of Software Supply Chain Security  

    DORA Report 2022: The Magnitude of Software Supply Chain Security  

    The term ‘software supply chain security’ (SSC) can be interpreted in many ways. Following the White House executive order in May 2021 and the European Cyber Resilience Act (CRA) of 2022, both governments and corporations are taking a more active role in ensuring their software is secure. Fundamentals such as having a software bill of materials (SBOM) and being able to cross-reference it with known common vulnerabilities and exposures (CVEs) are now the standard, but supply chain security covers far more than just the software that builds your products.

    To create certainty and clarity, leading standardization bodies such as OWASP, OpenSSF and others have set up ongoing efforts to define and mitigate SSC threats. They have active working groups, SIGs and TACs all working in vendor-neutral ways to help define and solve parts of the SSC problem.

    A great example of this is the continuous work being done by Google Cloud’s DevOps Research and Assessment (DORA) initiative. More specifically, the subject was reflected in the September 2022 DORA report that focused on security in general and supply chain security in particular. Using the Supply-chain Levels for Secure Artifacts (SLSA) framework as well as the NIST’s Secure Software Development Framework (SSDF) as benchmarks for actively addressing supply chain security issues, the researchers assessed how organizations were successful in approaching and addressing threats at all levels of the organization.

    My view of the major findings of the DORA report are summarized below.

    Who Owns Software Supply Chain Security?

    The report stated that organizations should empower individual teams and developers to “own” security while administering a consistent security policy and posture across its products. While I agree with this, we need to examine both sides of the ‘ownership’ coin. By nature, this hybrid approach creates complexity—and even friction—between the security team that, by definition, is responsible for all facets of security and the developers that are being tasked to ‘fix’ security issues in addition to their daily tasks. There has to be common ground between developers, their tasks associated with security and the security team.

    Individuals in organizations that build software care about what they build; they have a stake in the overall success of the final product and this feeling of ownership strengthens that level of care. However, asking for such ownership while adding a security component to the mix without providing sufficient knowledge, tools, proofs, and “power” to make decisions could lead to  a lack of trust, confusion, execution problems and frustration. To alleviate this, the security team should provide developers the tools, education and resources they need to feel empowered to enhance security efforts at the source. Organizations that provide evidence in a scalable, digestible manner are successful in building trust and creating common ground between the developer, security and the security team.

    There’s also another dimension that could potentially create silos between developers and security teams rather than bringing the two together: There’s no one single security team or owner of SSC. SSC is tricky to define as a category; it involves shift left security, runtime security and everything in between. Some portions of it are being governed by the security office, some by AppSec, some by DevOps and some by the developer. The ownership of each isn’t well-defined and the entire effort is very difficult to consolidate as one process or methodology. In addition to upskilling the developer with tools and training, long-term education, thought leadership and access to good technology are the keys to gradually reduce these friction points..

    Security Processes Slow Down Development

    Defining the tools, methodologies and processes that should be employed in an organization to add sufficient security while not impacting development speed is not a trivial task.

    ­­The following are four principals I think correspond well with the SLSA framework and that help address the development slowdown challenge:

    Set preventive measures: Avoid using known vulnerable and malicious packages from the beginning of development. By eliminating vulnerabilities introduced in first-party code (due to bad coding practices) as well as misconfigurations, wrong encryption and secrets exposures, you are preventing known threats from entering your product.

    Examine proactively and continuously: The security state of software changes because the software itself always evolves and new vulnerabilities are continuously being introduced. It makes a software artifact that was just secured suddenly become susceptible to attack. This requires continuously rescanning and revisiting software artifacts as well as frequently recalculating any potential risk.

    Set preemptive measures: Create a plan that initiates a prompt response when a potential risk is discovered. Both development and security teams should create the plan for decision making and execution. The measures should include ways to identify vulnerable artifacts across development and production, how to understand exploitability and true impact and how to prioritize and remediate accordingly.

    Do it all in an automated manner: Create automated processes based on reliable data and scanning tools to dramatically reduce the effort needed by developers and DevOps teams.

    DORA report co-author Eric Maxwell said one of the main principles is to remain tool- and platform-agnostic while focusing on capabilities and practices. From that perspective, automation is essential and it is enabled by security-as-code. Having a common language–code–allows teams to communicate about security as well as collaborate. It drives the things we discussed earlier around democratizing the security process and helps break down silos.

    The DORA findings—alongside the topics described above—encourage community awareness and continued shared efforts to bring much-needed attention to software supply chain security. Gradually, SSC security will evolve, but as it does so, the only way to properly address it is by ensuring the developers and security teams are on the same page and both have ownership of responding to and resolving issues as they appear. For more in depth review of the DORA report I encourage you to view Leap Left for Security: The DORA Report Roundtable.

  • You’re Already a Distributed Developer

    You’re Already a Distributed Developer

    For many coders, the idea of distributed systems development might seem like a dark art. They think of high-performance computing wizards crafting mysterious spells with job schedulers to work magic with hundreds of connected nodes in massively-parallel systems. Or large, distributed content delivery networks that balance resources to deliver content across the world no matter how high the demand. The folks doing those jobs have often worked in specialist distributed computing environments for years. But the complex problems they face at a massive scale aren’t so different from those that you deal with every day. In short, you’re a distributed developer too. It’s just that no one bothered to tell you.

    Everything is Distributed

    The world is a lot more distributed today than it used to be. Back in the day, code would talk directly to one mainframe or microcomputer, generally on the same LAN. Your PDP-11 and your batch order processing software had an intimate, exclusive kind of relationship.

    Computing environments began to change with the advent of open computing in the late eighties. It ushered in the era of multiple computers working together other over a network. The concept continued to evolve beyond PCs. Now phones, servers and smart kettles all expect an easy way to talk to each other.

    Today, distributed operation is so ubiquitous that people barely see it happening. When you type anything into a browser, it fires off dozens of queries to different backend systems owned by different organizations. These systems generally run on different infrastructure.

    Even a small application that makes a database query is a distributed system. The application often talks to a system—or even multiple systems—halfway around the world.

    Many apps these days are compositions of multiple API calls to different online services. When a developer gathers weather data from a service provider and feeds it into a cloud-based online geographical information system that decides how much to irrigate crops in Iowa tomorrow, what is that if not a distributed query?

    Software is becoming more distributed over time. As software became more complex, developers have long wanted to carve it up into smaller pieces. This began in earnest with the software component movement around CORBA in the mid-to-late nineties, gravitating to XML web services in the early noughts and then to RESTful APIs. All of these involved forms of remote procedure call to increasingly granular services.

    In the last decade, this atomization of monolithic software has become the norm rather than the exception. Container-based architectures ushered in the use of microservices for better development agility and maintenance. Most developers that started their careers in the last five years or so will have been forced to think about how these services communicate from the outset. Distributed design patterns are a fact of everyday life.

    And it’s Still Difficult

    While your distributed applications might not be as complex as those facing HPC and hyperscale services, they still come with some challenges that increase as systems become more granular. What happens if a remote API takes longer than expected to deliver a result because it is overloaded, meaning that API queries to different systems arrive in a different sequence than expected? How should you react if a network or system outage or an expired access key prevents an API from responding at all?

    Problems like these arise unexpectedly, meaning that you have to get more creative with code from the outset. Good developers do more than master object-relational mapping and send queries. They program defensively to ensure that their programs retry steps when a failure somewhere else delays that job or stops it altogether.

    There are different design patterns to handle most common distributed programming scenarios today, even though you might not think of them as such. For example, developers have been building load-balancing code into their applications for decades, distributing requests between different servers to maintain performance.

    Anyone who has tried to maintain transaction integrity when writing to multiple databases will also have dabbled in two-phase commit design, which ensures that all of the databases get updated (or that none of them do).

    The more experienced you get, the more complex and esoteric these patterns become. At some point, many developers move beyond basic CRUD models in their applications to handle more complexity when reading and writing data. For example, they might read data from a website in one format but collapse multiple records into one when writing data to the website’s back-end store. That means writing code that creates different types of read and write queries. That pattern, called command and query responsibility segregation (CQRS), is also a form of distributed programming.

    Then, there’s the Saga pattern, which people use to maintain application state across long-running processes comprising multiple transactions. This pattern relies on an event bus along with messaging and queuing code to handle services with unpredictable delays in response times. It ensures that the overall process transaction finishes in an acceptable state or rolls back.

    As you venture into more sophisticated patterns that hold transactions together while maintaining performance, you’ll find your code becoming more complex and, therefore, application development becoming more difficult. Such is the life of a distributed developer, even if you didn’t realize that you were one until now.

    Gaining this knowledge and experience as a developer is important, but once learned the hard way, you don’t have to keep doing it manually. There are technology solutions that manage some of these complex distributed patterns for you. As the volume of plumbing code to support distributed design patterns increases, so does the need to offload it. In an industry increasingly built on abstraction, it’s time to turn distributed systems management into a separate service and unburden your code.

    And—if you’re looking for an example platform specifically designed to address this issue—check out Temporal, a completely open-source (MIT license) microservice orchestration platform which enables developers to build scalable applications without sacrificing productivity or reliability.

     

  • GitLab Survey Reveals DevSecOps Gains

    GitLab Survey Reveals DevSecOps Gains

    A global survey of 5,010 IT leaders, CISOs and developers published by GitLab this week found well over half (56%) of respondents are working for organizations that are using DevOps or DevSecOps practices. However, 72% of respondents said they are using a DevSecOps platform (36%) or are considering adopting one in the next year (36%).

    Overall, less than a third of survey respondents (30%) said they were “completely” responsible for application security, with 53% reporting they are part of a larger application security team. Well over a third (38%) of the 1,453 security professionals surveyed said they are part of a cross-functional team focused on security.

    License compliance checks and security capabilities for cloud-native or serverless tied for the top of the list of current priorities (19%), but shifting security left (29%) was the top focus for the coming year. Nearly three-quarters (74%) of security professionals said their organizations either shifted left or plan to in the next three years. Nearly as many (71%) said a quarter or more of all security vulnerabilities were discovered by developers.

    Top frustrations identified specifically by security professionals included testing happening too late in the development cycle (43%) and difficulty prioritizing vulnerability remediation (41%).

    Those frustrations might increase in the coming year, with 85% of security professionals reporting they either have the same or lower budget in 2023.

    On the plus side, nearly two-thirds of the 1,954 developers surveyed said they are either using artificial intelligence (AI) and machine learning (ML) algorithms in testing today or will be in the next three years. A full 62% said they used AI/ML to check code, while 53% use bots for testing and 36% use AI/ML for code review.

    Nevertheless, more than two-thirds of security professionals (67%) said they are concerned about the impact of AI/ML capabilities on their job, with 28% of them admitting they are “very” or “extremely” concerned. Of those respondents who expressed concern, 25% said they are worried about the potential for AI/ML to introduce errors that will make their job more difficult.

    David DeSanto, chief product officer at GitLab, said GitLab is also working to address those concerns by embedding a range of AI/ML capabilities into its platform to optimize DevSecOps workflows and, most recently, enhanced its GitLab Remote Development module to make it easier to centralize the maintenance and security of development environments.

    GitLab this week also announced it is partnering with Oracle to make it easier to use the GitLab continuous integration/continuous delivery (CI/CD) platform to run AI and ML workloads on the Oracle cloud platforms. AI/ML is going to play a critical role in enabling organizations to build and deploy secure code without slowing down the pace of application development, said DeSanto. In fact, many organizations will soon be consolidating toolchains to enable them to achieve that goal, he added.

    The GitLab survey found 57% of security respondents said they use six or more tools, but many of those tools are legacy software composition analysis (SCA) tools that will be replaced as more responsibility for application security shifts left toward developers, noted DeSanto. Two-thirds of survey respondents (66%) reported they want to consolidate their toolchains this year.

    Collectively, the GitLab survey makes it clear that DevOps and DevSecOps practices and platforms continue to evolve rapidly. The challenge, as always, is keeping up with the pace of that change.

  • Preparing for Machine Learning Readiness to Unlock its Full Potential

    Preparing for Machine Learning Readiness to Unlock its Full Potential

    With the rapid advancement of artificial intelligence (AI) and machine learning (ML), companies need to understand and evaluate their readiness to adopt these technologies and drive material business outcomes.

    To assess a company’s machine learning readiness, we must consider several key factors, including:

    ● Vision/strategy
    ● Data availability
    ● Expertise, infrastructure and resources
    ● MLOps and governance

    In this article, I will review these three key factors in assessing ML readiness and how they can help your organization prepare.

    1. Vision/Strategy

    A company’s vision for its AI/ML strategy is essential to consider when assessing its machine learning readiness. This includes ensuring that the expected AI transformation aligns with the company’s strategic goals and that it will ultimately be meaningful for the business. To evaluate this, organizations should consider their overall strategy for AI adoption and determine how it fits into their broader business objectives. Additionally, companies should identify specific use cases and applications of AI that support their goals and how those can be implemented to drive business outcomes. A company that has already woven an AI/ML strategy into its business planning is in a far better position to assess its ML readiness.

    2. Data Availability

    I believe data availability is another crucial factor in assessing ML readiness. Machine learning algorithms require vast amounts of data for training and validation to offer real value. It would be challenging for a company to successfully implement machine learning without access to sufficient data, or they’d risk a model that cannot be trained effectively and that would, in turn, perform poorly. Companies must consider the quality and quantity of data at their disposal, as well as any potential future data sources when assessing their readiness for this important element of machine learning. I also think organizations should consider the feasibility of collecting and integrating data from various sources, whether internal or external, to have the most impactful machine learning initiative.

    3. Expertise, Infrastructure and Resources

    It’s no secret machine learning requires a diverse set of skills, including data science, programming and statistics, and every organization must be sure they have all the necessary talent and resources when building that function. These resources include data preprocessing, model development, model validation, deployment, monitoring, maintenance and the ability to train and onboard new employees to oversee AI/ML programs. Companies should also enable collaboration across these different units and roles. If a company chooses to outsource machine learning projects to third parties, the skills needed internally may need to shift toward project management and collaboration with external teams, as well as a nuanced and sophisticated understanding of the business problem, industry and technical capabilities of the vendors, to ensure proper execution.

    A company’s systems and technology stack are key to any operation. This includes selecting the right technologies and tools to enable the end-to-end creation and use of AI/ML-powered analytical applications. Before adopting ML, companies should evaluate their current stack to determine if it’s equipped to handle AI, including the availability of hardware, software and programming languages. Organizations should also consider the costs associated with selecting and implementing the right technology stack for AI/ML, including the costs of any necessary upgrades or new investments. Decision makers should also research and evaluate different technology vendors, their products and services, and find the one that best suits their needs in regards to cost, value and alignment with long-term goals.

    4. MLOps and Governance

    To be machine learning-ready, I find it important for companies to have machine learning ops (MLOps) in place. MLOps is a set of practices that allows companies to manage the end-to-end machine learning life cycle. It covers the entire machine learning process, from data preparation to model deployment and monitoring. Implementing MLOps includes continuous integration and delivery, model versioning and management and monitoring and alerting. Additionally, MLOps practices such as automated testing, model evaluation and deployment can help organizations improve the overall efficiency and effectiveness of their machine-learning efforts over time.

    A company’s governance structure for AI/ML adoption is another factor to consider when assessing machine learning readiness. Companies should evaluate their existing governance structures and determine if they are equipped to handle the implementation of AI, including data governance, compliance and risk management. We must also consider how we will manage the deployment and scaling of AI, including the development of policies and procedures to govern the use of AI and the identification of internal stakeholders and decision-makers who will be responsible for its implementation.

    Ultimately, I believe the key to machine learning readiness is to assess current capabilities, identify areas for improvement and develop strategies to address them. By taking the time to assess their machine learning readiness, companies can ensure they are in the best position to take full advantage of the benefits that machine learning can provide.

  • Continuous Testing: The Key to Quality Assurance in the DevOps Era

    Continuous Testing: The Key to Quality Assurance in the DevOps Era

    Continuous testing has emerged as a critical aspect of quality assurance in the DevOps era. In recent years, there has been a rapid transformation in software development methodologies. With the advent of DevOps, there has been a significant shift toward automation and continuous integration and delivery (CI/CD). The traditional approaches to quality assurance are no longer adequate to meet the needs of modern software development practices. Continuous testing is the key to ensuring that the software is delivered on time, with quality and reliability.

    Definition of Continuous Testing

    Continuous testing is a process of executing automated tests continuously throughout the software development life cycle to provide rapid feedback on the quality of the code. It is an essential component of DevOps that ensures that the software is always ready for release. Continuous testing allows for the detection of defects and errors at an early stage, which reduces the risk of production issues and enhances the overall quality of the software.

    Importance of Continuous Testing in DevOps

    To understand the importance of continuous testing in DevOps, it is essential to understand the evolution of quality assurance in software development.

    Traditional Waterfall Method

    The traditional waterfall model of software development had a sequential approach to quality assurance. Testing was only performed after the development phase was completed. This approach often led to delays and increased costs due to detecting defects later.

    Agile Methodology

    The Agile methodology introduced an innovative approach to quality assurance. Testing became an integral part of the software development lifecycle, and the focus was on continuous feedback and improvement. However, even with Agile, testing was still done in batches, and there was a significant amount of manual testing involved, which could be time-consuming and prone to errors.

    DevOps Model

    The DevOps model took the concept of continuous feedback and improvement to a new level. With DevOps, testing became automated, and continuous integration and delivery became the norm. This approach significantly reduced the time to market, increased collaboration between development and operations teams, and enhanced the overall quality of the software.

    The Role of Continuous Testing in DevOps

    Definition of DevOps

    DevOps is a software development methodology that emphasizes collaboration and communication between development and operations teams to deliver software products rapidly and reliably. It involves the integration of development, testing and operations into a continuous process, with automation as a key enabler.

    Importance of Continuous Testing in DevOps

    Continuous testing is a critical component of DevOps. It enables the testing of software continuously throughout the development lifecycle, which helps to identify defects and issues early on.
    This early detection of issues is essential in reducing the risk of production issues, and it allows for timely feedback, which facilitates faster decision-making.

    Benefits of Continuous Testing in DevOps

    There are several benefits of continuous testing in DevOps, including:

    · Faster time to market: Continuous testing enables the rapid release of software, reducing the time to market significantly.
    · Improved collaboration: DevOps emphasizes collaboration between development and operations teams, and continuous testing facilitates this collaboration by providing continuous feedback on the quality of the code.
    · Enhanced quality: Continuous testing enables the early detection of defects and issues, which reduces the risk of production issues and enhances the overall quality of the software.
    · Reduced costs: Continuous testing can help to reduce costs by identifying defects and issues early on, which reduces the cost of fixing issues later in the development cycle.

    Implementing Continuous Testing in DevOps

    Choosing the Right Test Automation Tools

    The selection of the right test automation tools is critical to the success of continuous testing in DevOps. Some of the popular test automation tools for DevOps include Selenium, Appium, Cucumber and JUnit.

    It is essential to choose a tool that is easy to use, integrates well with other DevOps tools, and has strong community support.

    Building an Effective Test Automation Framework

    An effective test automation framework is essential to implementing continuous testing in DevOps. A test automation framework is a set of guidelines and standards that help in the creation and execution of automated tests.

    A good framework should be modular, flexible, and scalable. It should also be easy to maintain and extend as the project grows.

    Integrating Continuous Testing Into the DevOps Pipeline

    To implement continuous testing in DevOps, testing should be integrated into the DevOps pipeline. The DevOps pipeline is a series of steps that automate the software development process, from code commit to production release.

    Continuous testing should be integrated into every stage of the pipeline, from unit testing to acceptance testing.

    Best Practices for Continuous Testing in DevOps

    To achieve the full benefits of continuous testing in DevOps, it is important to follow best practices. Some of the best practices for continuous testing in DevOps include:
    · Test early and often: Testing should begin early in the development cycle and continue throughout the development process.
    · Automate everything: Automation is essential for continuous testing in DevOps. All tests should be automated, including unit tests, integration tests and acceptance tests.
    · Integrate testing into the DevOps pipeline: Testing should be integrated into every stage of the DevOps pipeline, from code commit to production release.
    · Use the right test data: Test data should be relevant and representative of real-world scenarios.
    · Collaborate across teams: Collaboration between development, testing and operations teams is essential for the success of continuous testing in DevOps.

    Challenges and Solutions for Continuous Testing in DevOps

    There are several challenges to implementing continuous testing in DevOps, including:
    · Test data management: It can be challenging to manage and maintain relevant test data that accurately represent real-world scenarios.
    · Test environment management: Ensuring that the test environment is consistent and reliable can be a challenge, especially in complex applications.
    · Test automation maintenance: Test automation scripts can become complex and challenging to maintain, especially as the application evolves.

    Strategies for Overcoming Continuous Testing Challenges

    There are several strategies for overcoming the challenges of continuous testing in DevOps, including:
    · Test data management: Creating synthetic test data can help ensure that the data is relevant and representative of real-world scenarios.
    · Test environment management: Using containerization technology such as Docker can help ensure a consistent and reliable test environment.
    · Test automation maintenance: Regular maintenance and refactoring of test automation scripts can help keep them up-to-date and relevant.

    Conclusion

    Continuous testing is the key to ensuring quality in the DevOps era. It enables the rapid release of software, reduces the risk of production issues and enhances the overall quality of the software. Implementing continuous testing in DevOps requires choosing the right test automation tools, building an effective test automation framework, integrating testing into the DevOps pipeline and following DevOps best practices. Although there are challenges to implementing continuous testing in DevOps, these challenges can be overcome with the right strategies and practices. By implementing continuous testing in DevOps, organizations can achieve faster time to market, improved collaboration, enhanced quality and reduced costs.

  • The Fallacy of Continuous Integration, Delivery and Testing

    The Fallacy of Continuous Integration, Delivery and Testing

    We know that continuous integration and continuous delivery (CI/CD) have become a DevOps best practice. And many have learned that by adding continuous testing (CT), they can create a virtuous loop, ensuring perpetual code quality and security. They’re not wrong.  Yes, testing continuously is good practice, but some incorrectly equate the concept of continuous with monitoring, thinking they’re able to see everything that could eventually affect a customer’s experience (CX) with their software application. That’s a fallacy. Many of the most glaring blind spots that wreak havoc on CX exist outside the environment that CI/CD/CT is able to “see.” CI/CD/CT, even when supported by application performance monitoring (APM), doesn’t and cannot monitor for and alert to blind spots across the internet stack. And there are many. Only internet performance monitoring (IPM) can take CI/CD/CT as input streams and shift to continuous resilience as a measurable outcome.

    How CI/CD/CT Typically Works

    As customer requirements evolve, software development teams write code that triggers a new software build. In the continuous model, each new build moves to a runtime environment for integration and quality assurance and, ultimately, is deployed to end users across public or hybrid clouds. This is an application-specific view of the world.

    Since quality assurance is part of the typical continuous process, monitoring is an assumed benefit or outcome. This assumption might be because many now complement their CI/CD/CT processes with APM. What is achieved with APM in this approach is testing, not monitoring. While testing provides a better picture of performance at the application level, it doesn’t provide full visibility into the blind spots that exist between the application layer and the end user.

    APM Isn’t Enough

    While APM tools focus on code—including everything that negatively affects an application such as database wait times and inefficient code—IPM focuses on the internet, which has become your network. IPM monitors what impacts your customers, workforce or application (API) experiences across the internet.

    While APM and IPM may seem similar, they are nothing alike. They may share common tools, such as using synthetic agents, but the difference is in what they monitor and how. In this age of highly distributed applications, that difference means everything.

    Where APM can “simulate” what a user may see from cloud location to cloud location, which can confirm application performance, this simulation isn’t true or comprehensive monitoring because it is blind to network-related problems with end-user experience, which are vast. Customers and employees each have a unique internet performance fingerprint and access applications from myriad devices across multiple locations and network infrastructures, not through the individual cloud locations that APM synthetics simulate.

    IPM approaches synthetic testing more holistically, replicating multiple user journeys across a much more complex Internet infrastructure to proactively flag performance issues before they impact end users. This true monitoring takes place 24/7, not as point-in-time tests.

    Forensics vs. Monitoring

    With APM and similar observability solutions that embrace real user monitoring (RUM), it’s important to focus on what monitoring really means. APM passively collects data that is a proxy for user experience, enabling postmortem assessment of causes and possible fixes, but after the fact. Observability looks at logs, metrics and traces and provides an assessment of whether users are having a good experience or why they did not. Neither solution by itself will prevent poor experience, and many fixes won’t take place until a postmortem analysis feeds the next iteration. The poor experience has already done its damage, and the costs are already sunk.

    With thousands of potential blind spots across the globe that can slow or disrupt applications, from ISPs and wireless carriers to CDNs, it’s not hard to grasp the increasing and differentiated importance of IPM. Where the combination of CI/CD/CT with APM can proactively prevent deployment of poor code that would affect user experience, only IPM is continuously replicating – even more valuable than monitoring–millions of experiences across a dynamic multi-node network where the unexpected is now expected.

    Internet Resilience Vs. Retrospection

    Businesses today understand the importance of resilience: It’s a board-level concept. Any business that has lost market share and market cap during sustained downtime or poor user experience understands what’s at stake: Millions if not billions of dollars. IPM is squarely focused on “internet resilience,” and its pillars are availability, reachability, performance and reliability. For businesses reliant on the cloud and employers operating with highly distributed workforces, retrospection alone is a risky proposition. Now that the internet is the new corporate network, internet resilience is a new organizational mantra.

    The Takeaway

    Whether your organization embraces CI/CD/CT already or is rethinking its approach to DevOps, this article should give you pause. Your job–perhaps as part of a larger team–is to catch performance issues and potential disruptions with your application before client impact is realized. Without IPM, only part of that job is being done.

    Why is this so important? Even conservative estimates put the cost of an outage at $6,700 per minute (Gartner) and the “cost of slow,” where something causes an application to perform poorly and it goes unnoticed, is likely significantly higher because it happens all the time. The fallacy isn’t that there’s value in CI/CD/CT–there is. The fallacy is that you’re in control of the entire CX of your internet-deployed products without monitoring the internet stack and ensuring internet resilience. Only by adding IPM can you have this assurance.

  • The Rise of NetDevOps and CI/CD Pipeline Solutions

    The Rise of NetDevOps and CI/CD Pipeline Solutions

    Over time, as organizations, networks and technologies have changed, the way enterprises think about managing networks has shifted as well. Enterprises are continuing to evolve their networking strategies, specifically with regard to how they expand their efforts to adopt infrastructure-as-code (IaC) practices. Looking ahead, the industry will continue to see the proliferation of solutions that help teams adopt a modern network operating model that delivers an agile, data-driven change management process. In most organizations, the software development team members and the networking team members have operated independently, with separate tools and processes. Developers would leverage pipelines to test and deploy code and then open a ticket for network changes such as VPNs or firewall changes. The network team would implement the changes, and end users could then use the application. The handoff between dev and network could be immediate or, more likely, it could take a few days–delaying the ability of the end users to access the new functionality. NetDevOps is focused on addressing this situation, which has become one of the market’s more critical infrastructure questions: How do you build an organization that can function smoothly across different skill sets, different technologies and different processes? Those same enterprises will be looking to leverage continuous integration/continuous delivery (CI/CD) practices from their software development best practices to help them overcome some of the challenges involved with implementing automation and orchestration into their networks.

    CI/CD Accelerates the Automation Process

    Let’s start with CI/CD. Adoption of network automation and orchestration will accelerate as network operators change the way they think about network updates and configuration management. According to Cisco’s recent white paper on how “Automation is driving network engineer skills transformation,” the reasons we’re seeing an increase in automation adoption is due to organizations relying on continuous connectivity, a greater need for the network to become more dynamic and the increased rate of changes both in the service supply side (applications) as well as in the service demand side (application consumers).

    However, some enterprises trying to build their own automation infrastructure struggle with resourcing, skill sets and maintenance effort struggles. To solve this, many of them are looking to incorporate the traditional DevOps practice of building and updating code with traditional continuous integration/continuous delivery (CI/CD) pipelines into their network automation and orchestration initiatives.

    Establishing a CI/CD pipeline is a workflow best practice for DevOps teams to follow. Today, organizations rely on CI/CD pipelines to help minimize bottlenecks in the software development and delivery process, while continuous integration optimizes building, integrating and testing code within a development environment.

    By integrating both CI/CD with network automation, teams will gain greater velocity and improve delivery. And by integrating network automation with an application CI/CD pipeline, they can reduce the time to perform network changes and enable secure access between the application and an in-house sales database.

    Why NetDevOps?

    NetDevOps brings the culture, technical methods, strategies and best practices of DevOps to networking.

    When DevOps principles are applied to network operations, organizations minimize operator interactions and optimize programmable network processes to configure, deploy and manage the environment, increasing efficiency and ensuring network availability, quality and reliability.

    NetDevOps takes those principles and applies them in a networking environment, offering tools to empower teams to collaborate and build an organization that can function across different skill sets and different processes — enabling everyone to participate in network orchestration.

    And it’s a movement that’s taking hold. According to Cisco’s Hybrid Cloud Trends report, 55% of those surveyed have created a cross-functional team with technical and business representation and 50% have a centralized CloudOps and NetOps function to help ensure their organization’s hybrid cloud strategy meets business objectives.

    Network Management Tools Will Continue to Evolve

    In today’s networking environment, where the expectation is to have an “always on” approach, teams are expected to build and release new applications at a faster pace than ever before. And every team has its own processes and methods for doing that which can create delays in the time it takes to execute the application. Incorporating network automation into the pipeline can empower teams to make application changes and deploy them to the network in seconds instead of the multiple days/weeks required for manual network changes. By integrating CI/CD pipelines and NetDevOps principles with network automation and orchestration, organizations can reduce the network change bottleneck even more.

    As we face the new year, we expect the way in which teams consume network automation and orchestration services as part of their software pipelines to become more sophisticated, with NetDevOps and CI/CD practices becoming a standard. And as more services become available, the adoption of IAC becomes easier. By abstracting and tweaking to abstract the traditional CI/CD pipeline, organizations of all sizes can accelerate the use of network automation and orchestration.

  • AI Chatbots: Game Changers for DevOps Transformations

    AI Chatbots: Game Changers for DevOps Transformations

    In today’s fast-paced digital world, organizations are constantly looking for ways to accelerate their software development processes and ensure the seamless integration of their development and operations teams. DevOps has emerged as a popular methodology to achieve this goal, but even with its adoption, many companies still face challenges in their transformation journey. AI chatbots can revolutionize DevOps transformations, making them more efficient and effective than ever before.

    How AI ChatBots Improve DevOps Transformations

    Enhance collaboration by automating routine tasks and facilitating communication between teams: AI chatbots can handle repetitive tasks, freeing up team members to focus on more critical issues while also providing a centralized communication hub for both development and operations teams.
    Expedite incident management by providing real-time alerts and smart recommendations: AI chatbots can analyze logs and system metrics to quickly identify issues and suggest the most appropriate resolution steps, reducing downtime and improving overall system performance.
    Streamline deployment and monitoring processes using AI-driven automation: AI chatbots can intelligently manage deployment pipelines, ensuring that the right code gets deployed to the right environments while also monitoring system health and performance in real-time.
    Offer real-time analytics and insights into the development pipeline for informed decision-making: AI chatbots can gather and analyze data from various sources, providing valuable insights that help teams make better decisions related to code quality, release schedules and resource allocation.
    Identify skill gaps and recommend targeted training resources for continuous improvement: AI chatbots can analyze team performance and individual skill sets, helping organizations identify areas for improvement and providing personalized learning resources to close those gaps.

    Roadmap to Implement AI Chatbots in DevOps Transformations

    Evaluate your organization’s current DevOps maturity and identify areas for improvement: By assessing your existing processes and infrastructure, you can better understand the gaps where AI chatbots can make the most significant impact.
    Select an AI chatbot platform that aligns with your organization’s technology stack and goals: Choose a platform that complements your existing tools and supports your specific DevOps objectives to ensure seamless integration and adoption.
    Define and prioritize use cases for AI chatbot implementation in your DevOps processes: Identify the key areas where AI chatbots can provide the most value, such as incident management or deployment automation and prioritize their implementation accordingly.
    Develop a phased implementation plan that includes integration, testing and deployment of AI chatbots: Create a step-by-step plan to integrate AI chatbots into your existing DevOps processes, ensuring proper testing and validation at each stage to minimize potential disruptions.
    Train your teams on how to leverage AI chatbots effectively and continually refine their usage based on feedback and performance metrics: Provide education and training resources to help your team members understand how to use AI chatbots to their full potential and foster a culture of continuous improvement.

    Benefits of AI Chatbots in DevOps Transformations

    AI chatbots offer several benefits that can significantly improve DevOps transformations, including:

    Increased efficiency by automating manual tasks and streamlining processes: This leads to faster development cycles, quicker releases and a more agile response to changing business requirements.
    Enhanced collaboration and communication among development and operations teams: AI chatbots can break down silos between teams, fostering a more collaborative and unified approach to software development and delivery.
    Faster incident resolution and reduced downtime: With AI chatbots’ help, issues can be identified and resolved more quickly, minimizing the impact on end users and maintaining a high level of service quality.
    Real-time insights into the development pipeline for more informed decision-making: AI chatbots enable teams to make data-driven decisions by providing real-time analytics and insights into various aspects of the development process, such as code quality, resource allocation and release timelines.
    Continuous learning and improvement through AI-driven recommendations: By identifying skill gaps and suggesting targeted training resources, AI chatbots can help teams continuously improve their knowledge and expertise, ultimately leading to better performance and more successful DevOps transformations.

    Summary

    In conclusion, AI chatbots are poised to revolutionize DevOps transformations by addressing common challenges, enhancing efficiency and fostering a culture of continuous improvement. By automating routine tasks, facilitating communication and providing valuable insights into the development pipeline, they can significantly improve the effectiveness of DevOps processes. As organizations continue to adopt and refine their DevOps strategies, incorporating AI chatbots into their workflows will become increasingly essential to remain competitive in the ever-evolving digital landscape.

  • Report: Most IT Teams Can’t Fix Open Source Software Security

    Report: Most IT Teams Can’t Fix Open Source Software Security

    Lineaje, a provider of a platform for securing software supply chains, today published an analysis of 41,989 open source components embedded in the top 44 popular projects managed by the Apache Software Foundation (ASF). That analysis found more than a quarter (26%) of vulnerabilities are not patchable by the application development team that deployed them.

    In addition, the report found that 64% of the vulnerabilities analyzed have no patch as of yet. Overall, 68% of the vulnerabilities analyzed are due to dependencies created when an open source software project included a component or package developed by another open source project maintainer.

    Lineaje CEO Javed Hasan said that means that 90% of open source dependencies are transitive, in the sense that they are created when maintainers of a project included a vulnerable open source component created by another entity. Only 10% of the vulnerabilities discovered by Lineaje are the result of a dependency that an application development team could actually address on their own, he added.

    Overall, the report found a full 82% of components that are relied on to build open source software are inherently risky due to vulnerabilities, security issues, code quality or maintainability concerns. In addition, 5% of components failed a basic integrity check, with 3% having no known origin.

    Open source software is not created equal no matter what consortium is overseeing its development, noted Hasan. Unfortunately, there is no universally accepted method for validating the integrity of open source software, so application development teams should proceed with caution whenever employing open source software, he said.

    The challenge, of course, is that developers have been reusing open source software within applications for years now. It’s only in the recent wake of breaches involving open source software that organizations have been taking a harder look at how applications are developed.

    The Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation, is now at the forefront of an effort to better secure open source software by focusing on 10 streams of investment that, in total, will require more than $150 million in funding to drive greater adoption of DevSecOps best practices among maintainers of open source software projects. The issue is that many of those projects are maintained by a small number of programmers that voluntarily contribute their time and effort to build components that others are free to use. Like any other developer, the amount of security expertise those individuals have is limited. The onus for making sure open source software is secure when deployed in a production environment belongs to the organization that deploys it.

    Hopefully, following a recent executive order issued by the Biden administration, the level of open source security will steadily improve in the months ahead—assuming, of course, the organizations that rely heavily on open source software make more substantive contributions to securing it. In the meantime, organizations can count on the fact that cybercriminals have taken note of the vulnerabilities in applications that can be traced back to flaws in a wide range of open source software.