devopsy.kinsta.cloud

Category: Doin’ DevOps

  • MediaOps, Boca Code Award ‘Engineer the Change’ Scholarship

    MediaOps, Boca Code Award ‘Engineer the Change’ Scholarship

    I am really proud to announce that, in partnership with Boca Code, MediaOps has awarded a scholarship for the Fall 2021 term to the Software Engineering Career Course, a 10-week rigorous and fast-paced coding boot camp from Boca Code. Alina Maliuk, a first-generation immigrant from Ukraine, is the recipient.

    The Engineer the Change scholarship was initiated by MediaOps to help foster diversity in the local tech industry and empower individuals from underserved communities to develop the skills needed to succeed in IT. Boca Code offers comprehensive training in software development to enable course attendees to develop these skills. 

    Need for Diversity in STEM fields

    Research shows a staggering lack of diversity in STEM fields, most noticeably within computer and engineering positions. The tech industry is predominantly white and male, which appalls me (yes, even though I am a white male), and I wanted to do something to encourage more diversity across our industry. Gender and race inequity are glaring workforce disparities that require both recognition and responsibility from those at the highest levels in the tech world to create a more diversified and equitable workforce.

    Two Full Scholarships Awarded Annually

    MediaOps sponsors two $10,000 scholarships each year to enable students to enroll in Boca Code’s flagship Software Engineering Career Course. The full-time, 10-week intensive course combines theory and hands-on, project-based learning to prepare students for a career in software engineering. As part of their coursework, students take on development projects for local businesses, providing the students immediate, hands-on experience with real projects.

    The Engineer the Change scholarships combine two goals that are core to the MediaOps team and me. The first is doing our part to make South Florida a world-class hub for the tech industry. The second is providing opportunities for individuals from disadvantaged communities to acquire the skills they need for a successful career in tech. My hope is that we also serve as an inspiration to other local companies to fund similar programs in support of these worthy goals.

    About Our Winner, Alina Maliuk

    Alina Maliuk scholarship
    Alina Maliuk, Engineer the Change scholarship winner

    Alina hails from Ukraine, where she earned a bachelor’s degree in management. After college, she married and had her first son. During her maternity leave in 2012, her family emigrated to New York. Being first-generation immigrants, they had to start their life here in the United States from scratch. For the first few years, Alina and her husband worked in entry-level jobs. In 2018, now a mother of two, Alina took several online classes in web development and rediscovered her love of learning—particularly software development. Now a resident of Boca Raton, Alina found Boca Code and attended a workshop in web development.

    Alina was energized by that workshop, which furthered the continuing education she had already begun. Through that initial course, the Boca Code team inspired her to go even further. She is ready to move on to her next career, which she wants to be in software engineering. Thanks to the Engineer the Change scholarship, she will now have an opportunity to accelerate her tech career. At the same time, she is demonstrating to other female immigrants in Florida that if you can dream it and work hard to achieve it, you can make it happen.

    Alan Shimel

    CEO, MediaOps

    Additional Resources

  • A New DevOps Test Environment Platform is Needed

    A New DevOps Test Environment Platform is Needed

    Test environment management is bottlenecking DevOps transformations and is driving the need for a new test environment management platform solution. On-demand test environments in which developers and testers do not have to wait for a shared environment are sorely needed.

    The evidence is clear. In my book Engineering DevOps I indicated that finding and setting up test environments is a primary source of bottlenecks in DevOps value streams.

    In my February 2021 webinar with Plutora, I presented Best Practices for Developing Test Management Environments. I indicated that orchestrating and managing test environments is often a primary source of delays and waste within value streams.

    The State of Testing Report 2020 by PractiTest indicates that DevOps practices increased 57%; from 26% in 2017 to 41% in 2020. The report revealed that 46% of testing has shifted to using CI/CD processes, 55% are actively involved in shifting testing left and 36% are testing in production. This is clear evidence that DevOps testing is on the rise.

    Wolfgang Platz’s December 2019 article Why Software Testing Remains a Bottleneck indicates that testing is responsible for more delays than any other part of the development process – accounting for 52% of the bottlenecks in the development process. And  53% of respondents to the World Quality Report 2020-21 indicated that test environments are not available when needed, 67% lack the capability to book and manage test environments and 71% lack visibility of available test environments.

    An article by Velocity echoes the problems with pre-production environments. “With the rise of microservices and cloud-native applications, environments are becoming more complex. Often you need to wait or compete to get access to develop on a specific microservice, or can spend time waiting to deploy to a shared staging environment only to realize something already changed, and you’re back to square one.”

    The Continuous Testing Report – 2020 by Capgemini defines test environment management (TEM) as managing infrastructures, test data and applications to support testing on demand. These environments are one of the greatest impediments to continuous testing. In this report, 36% of respondents said they are spending more than half their time building and managing test environments; 44% of respondents indicate they spend weeks of their time searching, managing and generating test data.

    Test environments should be regarded as functions to be spun up or destroyed at will, with virtualization, infrastructure-as-code (IaC)  practices, service virtualization, containerization and improved test data management.

    Given all the above evidence, it is clear that a new approach to DevOps test environment management is needed.

    The following are examples of benefits that DevOps test environment management platforms need to offer:

    • Compatible with current tools:
      • Automatically map and configure application infrastructure to develop and test multiple services in an environment that runs just like it would in production. This needs to be fully agnostic to cloud and tech stacks, such as Kubernetes and other environments.
      • The platform needs to work on applications in parallel without disrupting others, with full access to data and third-party dependencies.
    • Automatic configuration and maintenance:
      • The platform needs to automatically configure and update any changes within the production infrastructure’s mapping in real-time so ephemeral environments will always be synced with what is in production. Instead of having to spend time making sure any changes are configured, the platform will ensure the right environment that replicates production, without having to wait.
    • Data and third-party dependencies:
      • The platform needs to load the relevant data and third-party resources that are vital parts of the environment to test, work and debug against, so test environments will, as closely as possible, mimic working in production.
    • Team collaboration:
      • The platform needs the capability to share ephemeral test environments with other stakeholders so they get visibility and can collaborate for faster feedback cycles. This is also important for collaborating for debugging.

    What This Means

    DevOps testing is on the rise, and test environments are a major bottleneck for DevOps testing. A new DevOps test environment management platform is needed to reduce the bottlenecks. On-demand test environments in which developers and testers do not have to wait for a shared test environment are needed. Such a solution would improve agility and reduce wasteful orchestration bottlenecks in test, improve quality and risk management through comprehensive and automated testing and enable continuous improvement by providing more intelligent insights and feedback.

  • Engineering Applications for DevOps (Part 6)

    Engineering Applications for DevOps (Part 6)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series include:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers five levels of application maturity. You can read part one here, part two here, part three here, part four here and part five here.

    Five Levels of Application Maturity

    Why is it that younger applications are more mature than older applications? DevOps works best with incremental changes rather than large complex changes that are time-consuming to change, build, test and deploy. DevOps also works best in ephemeral cloud computing infrastructures that can be scaled up and down quickly depending on short-term workload demands of supporting build, test and deployment requirements.

    While existing applications will eventually be modernized to take advantage of the cloud, greenfield applications should be designed to exploit recent advancements in cloud-based platforms and application delivery.

    The following paragraphs describe the characteristics of five levels of application maturity.

    Spaghetti applications are older, monolithic legacy application designs that tend to be large and complex, like a big messy bowl of spaghetti. This complexity occurs partly because these older apps accumulate a massive amount of code and patches over time and partly because the architectural approaches at the time they were conceived did not have modular, cloud-oriented requirements in mind. Spaghetti applications are the least capable of taking advantage of cloud infrastructures.

    Modular applications emphasize architecture designs that separate application components into functionally independent, interchangeable modules, such that each contains everything necessary to execute only one aspect of the desired functionality.

    A module interface expresses the elements that are provided and required by the module. The elements defined in the interface are detectable by other modules. Modules often use a design style known as service-oriented architecture (SOA), where services are self-contained black boxes that provide capabilities to the other application modules, as a service, through a network protocol. SOA may be confused with microservices architecture.

    There are key differences between SOA and microservices architecture that are important to DevOps. Microservices are preferred over SOA architecture because microservices are more fine-grained and restricted in what they can do. SOA uses an enterprise service bus for communication, whereas microservices use more lightweight stateless protocols and APIs managed as a product.

    Changes to SOA modules usually require a new application release, while a change to a microservice usually only requires the microservice to be replaced. Migration of modular applications to the cloud may require rework to take advantage of cloud features such as autoscaling.

    Cloud-ready applications that use the twelve-factor application modular design principles are more advanced. These modular designs emphasize declarative formats for automation, clean contracts with the underlying operating system and portability between execution environments. Deployment on modern cloud platforms obviates the need for servers and systems administration, minimizing divergence between development and production and enabling continuous deployment for maximum agility.

    These characteristics and capabilities enable the application to scale up without significant changes to tooling, architecture or development practices. Cloud-ready applications can be easily mapped to virtual machines (VMs) available through cloud infrastructure services (IaaS) for VMs based on VMware vSphere or Microsoft Hyper-V.

    Cloud-ready apps do not require rearchitecting or refactoring. They offer least-effort migration to the cloud with a lower availability rate. However, workloads such as legacy databases, third-party applications and homegrown business applications that are typically not designed for clustering or high availability suffer from a single point of failure.

    Cloud-ready apps are expensive to maintain. This is primarily due to the way they consume computing, storage and network resources, even when idle. Each component of an app runs in a dedicated VM with no scope for optimization. Upgrading or patching results in downtime and makes maintenance complex and expensive. Ultimately, cloud-ready apps derive less value from the cloud than cloud-optimized or cloud-native applications.

    Cloud-optimized applications take advantage of managed services offered by cloud providers. For example, instead of running a database in a VM, they consume database-as-a-service (DBaaS). Cloud-optimized apps typically target the platform-as-a-service (PaaS) delivery model of the cloud. They are portable and can be deployed in IaaS or PaaS. This reduces the deployment footprint, offering flexibility to IT teams. These apps are elastic, with the ability to easily scale in and scale out which results in optimal utilization of resources. Upgrading, patching and maintaining cloud-optimized apps is more manageable when compared to cloud-ready apps.

    The downside of cloud-optimized apps is the learning curve involved in migrating to the cloud. Developers, operators and system administrators will have to learn new ways of deploying and managing workloads. Line-of-business applications with access to source code are great candidates for being cloud-optimized apps. They can be redesigned and rearchitected to take advantage of the cloud. Cloud-optimized apps can integrate and interoperate with cloud-ready applications.

    Cloud-native applications are the most recent iteration of applications that are born in the cloud. They take full advantage of the cloud by exploiting capabilities such as elasticity, event-driven resource optimization and faster release cycles. Apps that are in the early stages of design and architecture are perfect candidates for cloud-native.

    Cloud-native applications are designed as federated microservices, packaged and deployed as containers and managed through modern DevOps processes. When your organization needs to scale to thousands of services in the cloud, microservices application architectures offer major benefits over monolithic modular architectures, including the following:

    • Separate microservices are better suited to different technology stacks.
    • Federated microservices enable independent deployment of different parts of the system.
    • Isolated deployments add to the resilience and fault-tolerance of the system.
    • Scaling characteristics may be different for each microservice.
    • APIs are used as connectors.
    • Stateless protocols help assure seamless recovery.
    • Event-driven programming reduces overhead and process latency.
    • Antifragile practices inoculate against unanticipated failure modes.

    The target deployment environment for cloud-native applications includes containers-as-a-service (CaaS) and functions-as-a-service (FaaS). This breed of apps delivers ultimate scalability and availability. Combined with continuous integration and continuous deployment, software is delivered rapidly, which accelerates time-to-market. It brings closed-loop feedback among developers, testers, operators and end users.

    Cloud-native apps demand new skills of containerization, microservices and build and release management. However, the ROI is higher when compared to other app models. Greenfield applications that are still in the design phase are ideal for adopting the cloud-native pattern.

    Diverse Applications for Cloud Migrations

    Large enterprises moving to the cloud will deal with a diverse set of applications. Through hybrid connectivity and integration, spaghetti, modular, cloud-ready, cloud-optimized and cloud-native applications will co-exist and interoperate with each other. If you are after the benefits of modularity, make sure you do not trick yourself into a microservices-only mindset.

    Explore the in-process modularity features or frameworks of your favorite technology stack. You will get support to enforce modular design instead of having to just rely on conventions to avoid spaghetti code. Then, make a deliberate choice whether you want to incur the complexity penalty of microservices. Sometimes you have no choice, but often, you can find a better way forward.

    What This Means

    This blog is one of a series that explained how to engineer applications to be most suitable for DevOps. Topics covered in this blog series include:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers five levels of application maturity. You can read part one here, part two here, part three here, part four here and part five here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 5)

    Engineering Applications for DevOps (Part 5)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:

    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers DevOps applied to manufactured (embedded) systems software. You can read part one here, part two here, part three here and part four here.

    DevOps Applied to Manufactured (Embedded) Systems Software

    In the late 1970s (yes, forty years ago!) as a young engineer, I was given an assignment to figure out how to accelerate the software development and delivery cycles for an innovative packet switch called SL-10, (predecessor to the more recent model called Data Packet Node [DPN]) manufactured by Northern Telecom. Each SL-10 “node” consisted of millions of lines of original embedded software source code and custom hardware configured with many interconnected processors and protocols interfacing with other nodes, host devices, and terminal devices.

    Given the complexity of the system and lack of automated build, test and deployment tools, it was taking more than twelve months to generate each release of the system, and there typically were lots of bugs even after the release. Without knowing much about DevOps (the word “DevOps” was not invented for another twenty-nine years), we proceeded to implement a continuous delivery system that included automated build, test and delivery tools and workflows.

    The result was remarkable. Lead times for releases to manufacturing were reduced more than tenfold, from twelve months to one month, and quality, measured as defects found by customers per release, also reduced tenfold. This dramatic shift in performance and capabilities made it possible for the engineering and manufacturing teams to accelerate the rate of innovation and trample the competition. The SL-10 product, and its rebranded DPN version, quickly dominated the connection-oriented packet switch market, and it became the pride of the Nortel corporation and the Canadian nation. Even cooler, I got a nifty president’s pin and promotion which paved the way for my career that led to writing a book and this series of blogs! If this real-life example is not sufficient to demonstrate that DevOps can be applied to manufactured embedded software products, then I don’t know what is.

    There have been many other examples published that show how DevOps practices applied to embedded manufactured systems have reaped similar benefits. A notable example was published in 2013 by Gary Gruver in A Practical Approach to Large-Scale Agile Development: How HP Transformed LaserJet FutureSmart Firmware.

    That is not to say that DevOps applied to manufactured software embedded systems is the same as applying DevOps to enterprise applications or COTS systems. A paper published on the CloudBees blog, Five Aspects that Make Continuous Delivery for Embedded Different, explained some of the key DevOps practices used with embedded systems.

    DevOps Benefits for Embedded Systems

    Benefits of DevOps for embedded systems are summarized as follows:

    • Retrofitting the huge code base typically found in legacy embedded software systems into a DevOps model is not easy. It is likely to be expensive, but it is almost guaranteed to be a worthwhile effort, especially if you have a longer-term vision and intend for your product to stay competitive in the future.
    • Developers of embedded software need access to massive amounts of computer resources for testing their real-time embedded code on powerful hardware simulators.
    • Developers and testers need access to physical product hardware to run their binaries for system tests that cannot be run on simulators alone. Lab-as-a-service (LaaS) systems that orchestrate testing topologies of real devices can improve sharing of critical resources. LaaS systems provide a test framework that can be integrated through APIs into a continuous delivery pipeline.
    • Long build and test processes may need to be reengineered to reduce serialized lead times into quicker build times made possible by scaling build and test resources.
    • Long compliance and conformance test processes for embedded systems need to be accelerated with automation and resource scaling.
    • Focus on continuous delivery rather than continuous deployments in which shippable product versions are frequently delivered to manufacturing for verification, but the frequency of deployments to end customers is determined by other business factors.

    What This Means

    This blog is one of a series explaining how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers DevOps applied to manufactured (embedded) systems software. You can read part one here, part two here, part three here and part four here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 4)

    Engineering Applications for DevOps (Part 4)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:
    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers DevOps applied to commercial off-the-shelf (COTS) systems. You can read part one here, part two here and part three here.

    DevOps Applied to COTS Systems

    I often hear that DevOps does not apply to systems that use commercial off-the-shelf (COTS) software. The idea that DevOps cannot be used with COTS systems is, frankly, a cop-out. Just because someone else designed and developed an application’s source code is not a sufficient reason to avoid engineering a DevOps solution.

    Consider the value stream chain of activities from the source vendor through integration and deployment in your environment. There are limitations and constraints to the level of automation allowed, the return on investment in automation and influence over the development life cycle that DevOps practices can introduce when dealing with COTS applications.

    COTS applications will typically have predefined and supported interfaces for managing configuration or accessing data. These endpoints may not be accessible or compatible with your existing toolchain or automated change management practices.

    Types of COTS Applications

    COTS applications typically come in one of three flavors; namely, closed, open and platform, and each should be evaluated differently.

    Closed COTS applications allow little to no customization to their functionality and/or interfaces. They typically have predefined management consoles and a published list of commands or APIs that will allow external applications to interact with the COTS applications. A good example of a closed COTS application is Microsoft Exchange.

    Key considerations for DevOps with closed COTS applications are as follows:

    • Can configurations be scripted and integrated with existing automation and orchestration toolchains?
    • Can configuration scripts be managed and maintained in external version control systems?
    • How are updates, patches and service packs applied and managed? Can these practices be automated?
    • What developer tools (APIs, software development kits [SDKs], etc.) are available to extend and/or interact with the application and/or data?
    • Are these tools compatible with existing tools and expertise in the enterprise?
    • Can these developer tools be integrated within the existing toolchain?
    • How frequently are changes made to the base product?

    If these applications can be managed by external systems and code developed to automate deployment and configuration and/or integration can be managed through a version-controlled pipeline, closed COTS applications make good candidates for DevOps and continuous delivery (CD). By applying CD principles, COTS binaries are versioned and stored in an artifact repository; installation and configuration scripts are managed through version control and IaaS workflows and applications and integration endpoints are auto-tested through deployment pipelines.

    This is only made possible when consumers of the COTS application collaborate with operators of the COTS platform to ensure that the solution is aligned with enterprise goals, objectives and standards related to portfolio management.

    Open COTS applications allow significant modifications to functionality, data and/or interfaces. These platforms typically have rich SDKs, APIs and embedded developer utilities that enable users and developers to modify all layers of the applications; namely presentation, business logic and data. Open COTS applications typically have a large, complex footprint consisting of core services, data and/or GUI customizations, embedded applications, etc. Classic examples of open COTS solutions include ERP/ CRM platforms (like SAP or Oracle) or portal platforms (like SharePoint, ServiceNow or WebSphere).

    Key questions to ask when considering whether DevOps can be applied to an open COTS application:
    • How are customizations (created through internal design editors, logic builders or data schema extensions) version controlled? Are these configurations stored in a database or code?
    • How are customizations created through internal tools and packaged, installed and configured in higher-level environments like staging or production?
    • What does the development life cycle for customization created with internal tools look like? What quality gates are required? How about architectural standards?
    • How are custom, embedded applications/applets packaged, installed and configured?
    • Can these custom applications/applets be run and tested separately from the underlying COTS application?
    • What does the development life cycle for customizations created with external tools look like? What quality gates are required? How about architectural standards?
    • Does the COTS application provide testing frameworks, mocks and/or stubs for testing external dependencies?
    • What automated test suites are supported by the platform (unit, functional, regression, capacity tests, etc.)?
    • What is the upgrade path for customizations (historically)? As you can imagine, open COTS applications add layers of complexity.

    When considering DevOps and CD with open COTS solutions, it is recommended that you build multiple pipelines to support each layer of the application in the dev or continuous integration (CI) stage of the value stream. Subsequent stages, like staging and production, will use a converged pipeline built from known good artifacts of the dev and CI stages.

    Before implementing a solution, it is important to study the development life cycle and practices of the target COTS application before building your toolchain. Understanding how the end-state platform is created and how change is introduced will strongly influence your toolchain and pipeline design.

    Platform COTS Applications provide a set of services and tools, as well as proprietary runtimes, that enable users to build and run custom applications on the platform. These are the most difficult type of applications to integrate into existing DevOps practices and CD toolchains because common practices like version control, testing and build are done through the platform rather than using external tools.

    Platform COTS are packaged applications that enable customers to build applications on top of the base platform. As a blend of closed and open systems, all the previous considerations are valid. In most cases, the base platform acts as a closed system, whereas the custom-developed applications on top act like open systems.

    For this type of application, it is critical to understand what hooks and services are available to support DevOps practices and CD. Many of these platforms are not going to fit within your existing toolchain and may not be able to employ the workflows, dashboards and reporting you have grown accustomed to from your current toolchain.

    What This Means

    This blog one of a series that explains how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covered DevOps applied to COTS systems. You can read part one here, part two here and part three here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 3)

    Engineering Applications for DevOps (Part 3)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • DevOps applied to software services.
    • Five levels of application maturity.

    This blog covers the topic of DevOps applied to enterprise apps and software services; part three in the series. You can read part one here and part two here.

    DevOps Applied to Enterprise Apps

    An enterprise application assists an organization in solving business problems. Enterprise applications are typically designed to interface or integrate with other enterprise applications used within the organization and to be deployed across a variety of networks (internet, intranet and corporate networks) while meeting strict requirements for security and administration management.

    Proprietary enterprise applications are usually designed and deployed in-house by specialized IT development teams within the organization. However, an enterprise may outsource some or all the development of the application and bring it back in-house for deployment.

    The size and complexity of large enterprise applications, especially older monolithic-architected applications that are not designed in modules, are a challenge when it comes to implementing DevOps. Given the large scale and distributed nature of the people, process and technology components of enterprise applications, it is recommended that enterprises engage very experienced DevOps consultants to determine strategic requirements, roadmaps and plans for DevOps that fully consider all three dimensions (people, processes and technologies) and nine pillars of DevOps before embarking on a major DevOps transformation journey.

    Despite the complexity, with the right strategy and plan, DevOps can deliver enormous ROI for enterprise applications. Amazon and Adobe are just two of many examples of large corporations that have benefited enormously from applying DevOps across the enterprise.

    DevOps Applied to Software Services

    I often hear people ask, “Does DevOps apply to software services as well as it does products?” These are usually people who have only worked in product-oriented organizations, such as companies that manufacture things that have embedded software.

    Ironically, I often hear people ask the opposite question, “Does DevOps apply to products with embedded software as well as it does to services?” These are usually people who have only worked organizations in which software is consumed as a service over an intranet or internet for a price or for free. Examples of these organizations are IT departments (even the IT departments of manufacturers!), government agencies and enterprises such as banks, insurance companies, hospitals and financial trading companies.

    You already know the answer to these two questions if you read the earlier parts of this blog series. The answer to both is clearly ‘Yes!’ Cloud services are doubly interesting to DevOps because not only are cloud services used by DevOps as a preferred infrastructure, as described in the next chapter, but cloud services also use DevOps in the process of creating and deploying software for the implementation of cloud components.

    While we are on the topic of DevOps and services, I want to help clarify some confusion regarding whether “newer” DevOps renders “older” IT service management (ITSM) obsolete or less important. This confusion exists because ITSM has often been regarded as a gatekeeper run by Ops to prevent Dev from getting out of control. While some organizations may have, in fact, implemented ITSM that way, this is not the purpose of ITSM.

    “DevOps tends to focus on the software delivery life cycle and on building and delivering software,” said Jayne Groll, president of the ITSM Academy. For its part, ITSM tends to have a broader focus; it is really looking at all aspects of customer engagement, from fulfilling standard services to ongoing support and operations activities. So, ITSM is “almost like the left and right ends of DevOps.”

    Using ITSM With DevOps

    Here are some recommended engineering practices for using ITSM with DevOps:

    • Training to help DevOps professionals gain an understanding of ITSM, and vice versa, will help both teams find points of integration—not just in technology, but in process, as well.
    • Find ways to automate service change approvals implementation of known, good changes to allow for agility, iteration and innovation. At the same time, hold back high-risk, high-impact changes that might need more than just an individual contributor to sign off on before they make it in front of a customer.
    • Self-service requests, automation flows and IT orchestration are now the front door to IT service management actions. These can and should be happening in the background, automagically created, maintained and measured by our DevOps tools.
    • Embrace Agile service management to take Agile values and apply Scrum-like development practices to process design and improvement. Aim to make more frequent incremental changes rather than design and implement a total end-to-end process in the traditional way.

    What This Means

    This blog is part three of a series to explain how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covered how DevOps can be applied to enterprise apps and software services. You can read part one here and part two here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 2)

    Engineering Applications for DevOps (Part 2)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:
    • Factors to consider when deciding whether an application is a good candidate for DevOps.
    • Best practices for engineering designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog is part two, and covers best practices for engineering designs for DevOps. You can read part one here.

    Poor design practices severely limit the potential for improvements provided by DevOps. As stated in my blog post Design for DevOps—Recommended Engineering Practices, “Without good design practices, a DevOps implementation has no hope of delivering on its promise of accelerating innovation with high quality and scale.”

    Following DevOps-recommended engineering practices for application design is important for legacy, greenfield or brownfield products, platform projects, feature enhancements or repairs. It is important for enterprise three-tier applications or multilayer products. And it is true whether the developers are using Agile, waterfall or ITIL development processes. In every application design case, designers face tough challenges and are expected to balance conflicting goals.

    Designers are expected to rapidly drive down the work backlog yet produce quality products that avoid costly rejections and rollbacks. In addition, there is pressure to increase the percentage of effort spent on creative content rather than corrective content and do so with limited time and resources. Balancing these conflicting goals is a tall order. If not supported by recommended engineering practices, the overall design experience is a cauldron of stress and a fast route to burnout.

    Recommended Engineering Practices

    The following recommended engineering practices improve the design experience and the products of that design in a way that is streamlined for the DevOps pipeline:

    • Designers must thoroughly understand customer use cases.
    • Usability, reliability, scaling, availability, testability and supportability are more important than individual features! Quality over quantity is recommended. Designs that anticipate actual customer use are the most successful.
    • The culture needs to support designers.
    • Leaders must support the designers with motivation, mentoring and training.
    • No designer can be expected to know everything. It is OK for designers to make some mistakes if lessons are learned and improvement quickly follows. Continuous monitoring and quick remediation are examples of good DevOps practices that help minimize the impact of any mistakes.

    Design Coding Practices

    Design coding practices are critical in the following ways:

    • Products are architected to support modular independent packaging, testing and releases. In other words, the product itself is partitioned into modules with minimal dependencies between modules. In this way, the modules can be built, tested and released without requiring the entire product to be built, tested and released all at once.
    • Where possible, applications are architected as modular, immutable microservices ready for deployment in cloud infrastructure, in accordance with the tenets of twelve-factor immutable non-monolithic apps, rather than monolithic mutable architectures.
    • Software code changes are pre-checked using peer code reviews prior to commit to the integration/trunk branch.
    • Software changes are integrated in a private environment together with the most recent integration branch version and tested using functional testing prior to committing the software changes to the integration/trunk branch.
    • Developers commit their code changes regularly—at least once per day.

    Good Design Practices Support QA and Ops

    Good DevOps design practices support QA in the following ways:

    • Design practices consider and understand the QA process.
    • Software code changes are pre-checked with unit tests prior to commit to the integration/trunk branch.
    • Software source code changes are pre-checked with static analysis tools prior to commit to the integration branch. Static analysis tools are used to ensure the modified source code does not introduce critical software faults such as memory leaks, uninitialized variables, and array-boundary problems.
    • Software code changes are pre-checked with dynamic analysis and regression tests prior to commit to the integration/trunk branch to ensure software performance has not degraded.

    Good DevOps design practices support Ops in the following ways:

    • Design practices consider and understand delivery and deployment pipeline processes.
    • Software features are tagged with software switches (i.e., feature tags or toggles) during check-in to enable selective feature-level testing, promotion and reverts.
    • Automated test cases are checked in to the integration branch when code changes are checked in. Evidence that the tests passed are included with the check-in.
    • Tests are conducted in a pre-fight test environment that is a close facsimile of the production environment.

    Supporting Tools

    The following supporting tools are needed to realize design for DevOps best practices:

    • Elastic infrastructure that can be easily orchestrated, created and released, as needed, to support designers’ tasks on-demand with minimal delay.
    • Design, code management, monitoring and test tools are readily available and scalable with minimal delay.
    • Monitoring tools that track application process performance and report the results to designers in easily consumable formats without delay.

    Applications for Which DevOps Does Not Apply

    “If you build it, they will come,” made for a great movie idea in Field of Dreams, but the reality is that many DevOps journeys led to nowhere despite the excellence of the people, processes and technology involved. As indicated above, applications that will not benefit from faster lead times, incur insufficient costs to justify investment in DevOps or that are too rigid to accept change are not good candidates for DevOps.

    What This Means

    This blog is part two of a series explaining how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps, and covers topics including:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covered practices used to engineer designs for DevOps. You can read part one here.

    For more information on how to engineer applications for DevOps, refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 1)

    Engineering Applications for DevOps (Part 1)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:
    • Factors to consider when deciding whether an application is a good candidate for DevOps.
    • Best practices for engineering designs for DevOps
    • DevOps applied to enterprise apps and software services
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity

    This blog covers factors to consider when deciding whether an application is a good candidate for DevOps.

    Factors to Consider for Engineering Applications

    The extreme worldwide hype around DevOps products and services seems to suggest that DevOps is good for everybody and applies equally well to every software-based product or service in every organization. Here is my experience with that assumption: It is not true!

    On the other hand, I have heard people say that DevOps does not apply to applications for which it absolutely does! Before you go a step further in your DevOps journey—or decide that your application does not qualify for DevOps—you should consider carefully how well your application is engineered to become a model for DevOps implementations.

    Best Practices for DevOps Determination

    The following are important factors you can use to decide whether an application is a good candidate for DevOps:
    • The application will benefit from faster lead times.
    • Leaders with responsibility for this application are open to collaboration and will be sponsors of change.
    • Team players that are associated with the application team (product owners, Dev, QA, Ops, infrastructure, security, PM) are open to collaboration and change.
    • The application is currently using or planning to use service-oriented, modular architectures or microservices.
    • There are at least ten people associated with the application including product owners, Dev, QA, Ops, infrastructure, security, PM. Smaller projects with smaller teams may not show sufficient impact or justify substantive DevOps investment.
    • The application is expected to undergo changes for more than a year. Less time would not likely yield return on the investment required for DevOps.
    • The application represents a good level of business impact and visibility but does not involve an extreme amount of risk for the business.
    • The application experiences frequent demands for changes from the business.
    • Not all tools used with the application need to be replaced to implement a DevOps toolchain. The costs of changing out all tools can negatively affect the ROI and business case for DevOps transformations.
    • Efforts to build, test or deploy releases of the application are substantial and could be reduced significantly by automation.

    It is not necessary for an application to perfectly fit all the factors, but the more closely it fits the more likely your application will benefit from using DevOps and will, therefore, be worth the cost to implement a well-engineered DevOps practice.

    DevOps can be used with all types of software applications and services. However, not all applications and services should be engineered to use DevOps. The factors that determine whether to engineer DevOps for an application have more to do with business factors than technology differences between different types of applications. Applications must be engineered in specific ways for DevOps to operate effectively.

    What This Means

    This blog is part one of a series explaining how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors to consider when deciding whether an application is a good candidate for DevOps.
    • Best practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity

    This blog covers the first two topics: factors to decide whether an application is a good candidate for DevOps and best practices to use when engineering designs for DevOps.

    For more information on engineering applications for DevOps, refer to my book Engineering DevOps.

  • A New DevOps Database Platform is Needed

    A New DevOps Database Platform is Needed

    The diverse number of database platforms, deployment environments and data variants needed to support the plethora of applications and services required for rapid DevOps continuous delivery lead times, coupled with the challenges of integrating database changes for application pipelines, is a growing challenge for many organizations as they strive to accelerate their DevOps transformations.

    When you stand back and look at the bigger picture, an enterprise with many databases and various data requirements for DevOps continuous delivery presents a problem. At best, it makes things chaotic; at worst, it is a gross impediment to overall organization agility—which undermines the very purpose that DevOps is meant to serve.

    In my book, Engineering DevOps, I explained that databases offer several challenges for CI/CD pipelines.

    In the simplest case, when a database is used solely by one application or microservice, the database pipeline can be integrated with the application pipeline to ensure its changes use the same version management system and change controls to ensure the database changes are kept in sync with the application. However, many database platforms are not as responsive to changes as the applications themselves and become a bottleneck for the pipeline.

    If a database is used by several applications, each with its own DevOps pipeline, then changes to the database must be coordinated with all applications that depend on it. Maintaining a separate database pipeline linked to the application pipelines through CI/CD toolchains, as a strategy, is overly complicated and can often introduce additional bottlenecks and synchronization issues.

    Another challenge with database pipelines is the management of data in the database during the CI and CD stages. Data in production needs to be persistent and is ever-changing, even while the underlying database code is changing. During the CI process, sample data from production needs to be used to ensure testing is realistic, and new test data must be introduced to verify new database capabilities are working.

    During the CD stages, rollbacks to prior versions of database code must avoid rolling back to versions that are no longer compatible with production data. This is a key reason why database changes must follow a disciplined approach that fully tests database changes with applications prior to full deployment. Rolling deployments, in which applications and database changes are deployed gradually in separate clusters, can minimize the blast radius in case of problems. The required coordination of separate systems adds complexity and bottlenecks.

    So, why not simply integrate database changes as artifacts within the same DevOps pipeline as the application?

    Gilad David Maayan’s September 2020 article DevOps Database Strategies and Challenges: Automation, Scalability, and More explained some of the specific challenges that arise with integration of traditional databases with DevOps.

    Database tools were not designed for integration with DevOps tools. These tools tend to be database- and environment-specific, which can make it difficult to incorporate into pipelines designed for flexibility.

    Database performance and ease of management often do not perform well in DevOps environments where multiple applications need to use the database in a short time, such as in the accelerated lead times for DevOps continuous delivery. This is especially true with larger-scale databases. If an application change must sync with a database schema, changing the pipeline can become complicated especially if there is a need to sync across multiple application pipelines to meet continuous delivery goals.

    While DevOps processes are proven for continuous delivery of applications, incorporating database changes into the DevOps pipeline is hindered by large data sets, limited database automation and security concerns. As a result, database development and testing continue to rely on outdated processes, with fixed and often shared instances refreshed from backups. A new DevOps Database platform is needed to provide highly dynamic and efficient cloud-native database solutions for modern enterprise digital transformations, and that platform must include DataOps use cases for DevOps, DevSecOps, continuous testing and governance. This new kind of data repository must be designed to integrate seamlessly with DevOps CI/CD pipelines through flexible APIs and cloud-native architectures.

    Given the points above, traditional database solutions and parallel data pipeline strategies are not sufficient to meet the challenges of enterprise DevOps transformations. What is needed is a new kind of DevOps database platform that allows the database, and database changes, to be as agile as any other artifact in the DevOps pipeline.

    Paul Stanton, VP of product management at Windocks, in his June 2021 YouTube video Windock DevOps Data Repo, said, “Imagine terabyte class database environments managed as DevOps artifacts with entire versioned databases available in seconds.” That’s what we need.

    The following are examples of the capabilities required for such a new DevOps database platform.

    • Automated database provisioning
    • Distributed repositories support
    • Simple declarative builds
    • Dynamically scalable
    • Virtual hard drive images for a wide variety of DBs – Linux, SQL, Oracle and others
    • Distributed versioned database environments working on standard file systems and storage systems delivery writeable data at the speed of DevOps – in other words, in seconds.
    • Deploys equally to all operating systems and cloud environments, Docker containers and Kubernetes.
    • Many clones per virtual machine (VM)
    • Security features including access management, data masking and encryption
    • Compatible with and easily integrates with CI/CD platforms
    • Compatible with multi-cloud (AWS, Azure, GCS)
    • Built-in data security and governance into DevOps data.

    I will explain these more completely in a subsequent article.

    What This Means

    Digital transformations are imperative for modern enterprises and organizations if they are to remain competitive. However, the transformations of many enterprises and organizations are being seriously impeded by having too many database platforms and tools that do not integrate well and that bottleneck continuous delivery goals. A new kind of DevOps database platform, in which the database is treated as any other artifact, is needed to resolve technical problems with database integration to DevOps application pipelines. A new DevOps database platform solution is needed to accelerate time-to-value and realize efficient, safe operation for digitally transforming enterprises and organizations.

  • Continuous Testing Practices Gap Assessment

    Continuous Testing Practices Gap Assessment

    Continuous testing is an essential part of DevOps. To get the maximum value from continuous testing, it is important to use recommended best practices, inclusive of people, processes and technologies. A gap assessment is a great way to efficiently evaluate an organization’s practices for DevOps and determine a strategy for improvement. Gap analysis provides valuable input for formulating a strategy and a roadmap to improve continuous testing.

    My SlideShare, Gap Survey, Assessment and Analysis For Continuous Testing, explains how my continuous testing gap assessment tool greatly speeds up the gap survey, gap assessment and gap analysis process for continuous testing.

    It is important to clarify a few definitions for this blog because I have found that different people and organizations use these terms differently. In this blog, and in my consulting work, I use the following definitions.

    • Gap survey is a discovery tool used to collect information about the current state of practices for a topic such as continuous testing.
    • Gap assessment is a process to determine differences (gaps) between current practices and recommended practices for a topic. Gap assessment is also used to refer to the entire collection of activities in this list.
    • Gap assessment workshop is a meeting in which the gap assessment is validated prior to gap analysis.
    • Gap analysis is the process of determining priorities for reducing gaps found during a gap assessment.
    • Gap assessment tool is software that helps perform gap surveys, gap assessments and gap analysis.

    I define and use a seven-step process for gap assessments.

    The steps are as follows:

    1. Pick a topic to analyze. The same processes and tools can be applied equally well to almost any topic, such as continuous testing.
    2. Determine recommended practices for the topic. Something that is worth doing a gap assessment for, such as continuous testing, will normally be fairly complex with tens if not hundreds of practices. Categorize the practices into groups (I call the groups “pillars”) to make the list of practices more understandable and meaningful. To do this requires some research and discovery work. For example, I have a database of practices and pillars for DevOps, continuous testing, DevSecOps and SRE as well as some other topics.
    3. Enter the practices from step two into a gap assessment tool. The tool can be a simple survey tool you create, a commercial survey tool or something custom-made.
    4. Determine who needs to be included in the gap survey.
    5. Collect data using the gap survey.
    6. Perform a gap assessment.
    7. Perform a gap analysis.

    For the remainder of this blog, I will explain in detail the seven steps for continuous testing.

    Step One: Pick a Topic to Analyze

    In this blog, the topic is continuous testing.

    Step Two: Determine Recommended Practices for Continuous Testing

    For an example continuous testing assessment, I used the 15 practices categories described in this three-part series:
    Continuous Testing Practices Part 1
    Continuous Testing Practices Part 2
    Continuous Testing Practices Part 3

    The 15 practice category pillars are:

    • Continuous testing tenets
    • Leadership
    • Culture
    • Test strategy and plans
    • Test management
    • Test automation
    • Test tools
    • Test infrastructure management
    • Test scripts
    • Test results
    • Security tests
    • Pre-flight testing
    • Integration testing
    • Pre-production testing
    • Test in production

    Step Three: Enter the Practices from Step Two Into a Gap Assessment Tool

    A gap assessment tool, pre-loaded with sample continuous testing practices, can be found in a file called “Continuous Test Practices Assessment Tool v2. It can be downloaded for free from one of the resource pages found on my website Engineering DevOps.

    You can edit the practices categories and add/delete practices from each category if you prefer to make changes.

    Step Four: Determine Who to Include in the Gap Survey

    For the gap analysis to be comprehensive, people in roles that are affected by continuous testing practices need to be surveyed, or at least represented, to ensure their perspectives are included. The following are example roles that typically are included.

    • Business leaders – because they influence culture and budgets for continuous testing.
    • Developers – because they need to design in accordance with continuous testing practices.
    • Project owners – because they influence product work priorities.
    • QA testers – because they need to test in accordance with continuous testing practices.
    • Ops – because continuous testing practices affect operations.
    • Security – because continuous testing affects security practices.

    The survey can be conducted for an individual application, a group of applications, or all the applications in the enterprise. However, it is important that the gap assessment and gap analysis be performed on the organizational segment that is being targeted for improvement.

    Step Five: Collect Data Using the Gap Survey

    A gap survey should allow each surveyed person to enter an importance level score, a practice level score and comments for each practice. All this information is essential for the gap assessment and gap analysis. In the gap survey included within my gap assessment tool, survey respondents are asked to score the importance of each practice as one of 0=Not relevant, 1=not Important, 2=Nice to have, 3=Important, 4=Very important or 5=Critical. Practice level choices are 0=Not sure, 1=Rarely, if ever, 2= Sometimes, 3=Most of the time, 4=Always or 5=We are really good at this. Comments that are relevant to qualify or explain the scores entered for each practice should also be entered, especially if there is any doubt or ambiguity.

    Step Six: Perform a Gap Assessment

    The gap assessment process requires all the practices scores collected from the surveys to be collected and assembled to calculate an aggregate set of scores. Gap scores are calculated using a formula that weights each practice level score with the corresponding importance level score. A visual representation helps to identify practices areas and individual practices that have the highest, most important gaps.

    No matter how professionally written the practices and score definitions are, it is not unusual for some people to misunderstand and to enter scores that they would not have otherwise. For this reason, it is important to ensure that the data collected is validated before conducting the gap analysis. The preferred approach is to conduct a gap assessment workshop with key representatives from each role that participated in the survey. During the workshop, the scores for each practice that have a high deviation between survey responses are discussed and, if necessary, adjusted.

    Step Seven: Perform a Gap Analysis

    The gap analysis process involves extracting the high gap practices and tagging and ranking each of them against solution categories determined by a consultant or topic expert. The results of the gap analysis indicate where solution strategies and implementation roadmaps need to be focused to reduce the most important gaps.

    What This Means

    Those who are serious about DevOps will do well to learn from others and be serious about using recommended continuous testing practices. This blog explained a gap assessment approach, which uses my 15 categories of continuous testing practices. My free Continuous Testing Gap Assessment Tool facilitates performing a gap analysis for continuous testing and outlines priorities to improve DevOps practices for an organization. For those that want to dig deeper on this topic, additional definitions and contextual information related to the way I do gap assessments are clarified in my book Engineering DevOps.