devopsy.kinsta.cloud

Tag: application engineering

  • From Frontend to Full Stack: How I Leveraged Modern Dev Tools to Make the Leap 

    From Frontend to Full Stack: How I Leveraged Modern Dev Tools to Make the Leap 

    A few years ago, I had an idea about how to make publishing recipes on the internet more accessible and profitable. As a front-end engineer-turned-product manager, I could write code as long as it involved JavaScript. But I needed someone who could handle the whole stack, especially the back end—you can’t build a platform with frontend tools only. So I searched for a technical co-founder, poked around online forums and talked to friends. Eventually, I found a candidate (let’s call him Tom) who seemed like a good fit. He’d built a recipe app in the past and was comfortable with both the front and the back end. It seemed great. We struck a deal and dove in.

    Tom started building the API and database while I made prototypes with a no-code builder called Bubble. I loved using Bubble. Without writing any code, I could build things fast and see what I was creating. Eventually, though, there were too many limitations with the no-code approach, including the lack of a Git repository for version control and history. So we decided to switch to full code with React Native mobile apps. 

    A few years prior, I’d switched career paths and became a product manager, but it had been a while since I had dug deep into writing code. And although I had a background in React, I felt insecure about building the frontend of a whole web platform on my own. But alas, you do all sorts of scary and undesirable things when you’re a startup founder!

    After jumping into it, though, I quickly remembered that I really enjoyed coding. I picked up React Native pretty quickly and started building screens. Tom took care of the data while I took care of the UI. All was good.

    Well, all was good until it wasn’t. We were a few weeks away from launch day and were finishing up. We even had a few early-access bloggers lined up to launch with us. And then, surprise! Tom decided to take a job as the CTO of another startup. His parting words were, “You’re technical enough; seems like you can handle it.” 

    I was floored. Aside from the front end, I had no idea how anything worked. We had cloud functions running on Pulumi hosted in Google Cloud, and I had no idea how they worked. I didn’t know how to train the machine learning AI models we used for ingredient recognition. The most I knew about SQL was that you used it to get stuff from a database. And to top it all off, our recipe scraping API was written in Python, a language I had no experience with, running a version that did not work on my M1 Mac. What in the heck was I supposed to do?

    Well, I guess I did what any engineer would do. I figured it out step by step. When the Postgres database ran out of rows on the free Heroku plan, I figured out how to migrate it. When the recipe scraper API needed updating, I learned how to code in Python and deployed it with Docker. It was an incredibly stressful time. I was up against a looming launch deadline with a codebase teetering on collapse at any moment—mostly because I didn’t know how to hold it up.

    The silver lining was the GraphQL API platform, Hasura. It was by far the easiest part of the stack for me to grasp. Understanding how our data was laid out was easy enough—I could simply explore it like a spreadsheet. I wasn’t sure how to build GraphQL queries, but the tutorials on the Hasura website helped me learn the basics and the query builder in the console did the rest. It’s no-code, but better—a no-code console makes it super easy to add new tables and modify permissions, and a Git repository keeps everything in check and easy to reference. 

    Once I understood how Hasura and the rest of the stack worked, I was able to finish the screens that I needed for launch.

    Today, the company has grown significantly. We have a new co-founder, but she’s nontechnical. After figuring out how to run the entire stack, I decided I was technical enough to be the primary engineer. I’ve learned how to use all sorts of Hasura functionality that makes our lives so much easier—listening for events, firing off actions in our API to run business logic, and much more. Through all this, I now consider myself a full-stack engineer. 

     

  • Engineering Applications for DevOps (Part 6)

    Engineering Applications for DevOps (Part 6)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series include:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers five levels of application maturity. You can read part one here, part two here, part three here, part four here and part five here.

    Five Levels of Application Maturity

    Why is it that younger applications are more mature than older applications? DevOps works best with incremental changes rather than large complex changes that are time-consuming to change, build, test and deploy. DevOps also works best in ephemeral cloud computing infrastructures that can be scaled up and down quickly depending on short-term workload demands of supporting build, test and deployment requirements.

    While existing applications will eventually be modernized to take advantage of the cloud, greenfield applications should be designed to exploit recent advancements in cloud-based platforms and application delivery.

    The following paragraphs describe the characteristics of five levels of application maturity.

    Spaghetti applications are older, monolithic legacy application designs that tend to be large and complex, like a big messy bowl of spaghetti. This complexity occurs partly because these older apps accumulate a massive amount of code and patches over time and partly because the architectural approaches at the time they were conceived did not have modular, cloud-oriented requirements in mind. Spaghetti applications are the least capable of taking advantage of cloud infrastructures.

    Modular applications emphasize architecture designs that separate application components into functionally independent, interchangeable modules, such that each contains everything necessary to execute only one aspect of the desired functionality.

    A module interface expresses the elements that are provided and required by the module. The elements defined in the interface are detectable by other modules. Modules often use a design style known as service-oriented architecture (SOA), where services are self-contained black boxes that provide capabilities to the other application modules, as a service, through a network protocol. SOA may be confused with microservices architecture.

    There are key differences between SOA and microservices architecture that are important to DevOps. Microservices are preferred over SOA architecture because microservices are more fine-grained and restricted in what they can do. SOA uses an enterprise service bus for communication, whereas microservices use more lightweight stateless protocols and APIs managed as a product.

    Changes to SOA modules usually require a new application release, while a change to a microservice usually only requires the microservice to be replaced. Migration of modular applications to the cloud may require rework to take advantage of cloud features such as autoscaling.

    Cloud-ready applications that use the twelve-factor application modular design principles are more advanced. These modular designs emphasize declarative formats for automation, clean contracts with the underlying operating system and portability between execution environments. Deployment on modern cloud platforms obviates the need for servers and systems administration, minimizing divergence between development and production and enabling continuous deployment for maximum agility.

    These characteristics and capabilities enable the application to scale up without significant changes to tooling, architecture or development practices. Cloud-ready applications can be easily mapped to virtual machines (VMs) available through cloud infrastructure services (IaaS) for VMs based on VMware vSphere or Microsoft Hyper-V.

    Cloud-ready apps do not require rearchitecting or refactoring. They offer least-effort migration to the cloud with a lower availability rate. However, workloads such as legacy databases, third-party applications and homegrown business applications that are typically not designed for clustering or high availability suffer from a single point of failure.

    Cloud-ready apps are expensive to maintain. This is primarily due to the way they consume computing, storage and network resources, even when idle. Each component of an app runs in a dedicated VM with no scope for optimization. Upgrading or patching results in downtime and makes maintenance complex and expensive. Ultimately, cloud-ready apps derive less value from the cloud than cloud-optimized or cloud-native applications.

    Cloud-optimized applications take advantage of managed services offered by cloud providers. For example, instead of running a database in a VM, they consume database-as-a-service (DBaaS). Cloud-optimized apps typically target the platform-as-a-service (PaaS) delivery model of the cloud. They are portable and can be deployed in IaaS or PaaS. This reduces the deployment footprint, offering flexibility to IT teams. These apps are elastic, with the ability to easily scale in and scale out which results in optimal utilization of resources. Upgrading, patching and maintaining cloud-optimized apps is more manageable when compared to cloud-ready apps.

    The downside of cloud-optimized apps is the learning curve involved in migrating to the cloud. Developers, operators and system administrators will have to learn new ways of deploying and managing workloads. Line-of-business applications with access to source code are great candidates for being cloud-optimized apps. They can be redesigned and rearchitected to take advantage of the cloud. Cloud-optimized apps can integrate and interoperate with cloud-ready applications.

    Cloud-native applications are the most recent iteration of applications that are born in the cloud. They take full advantage of the cloud by exploiting capabilities such as elasticity, event-driven resource optimization and faster release cycles. Apps that are in the early stages of design and architecture are perfect candidates for cloud-native.

    Cloud-native applications are designed as federated microservices, packaged and deployed as containers and managed through modern DevOps processes. When your organization needs to scale to thousands of services in the cloud, microservices application architectures offer major benefits over monolithic modular architectures, including the following:

    • Separate microservices are better suited to different technology stacks.
    • Federated microservices enable independent deployment of different parts of the system.
    • Isolated deployments add to the resilience and fault-tolerance of the system.
    • Scaling characteristics may be different for each microservice.
    • APIs are used as connectors.
    • Stateless protocols help assure seamless recovery.
    • Event-driven programming reduces overhead and process latency.
    • Antifragile practices inoculate against unanticipated failure modes.

    The target deployment environment for cloud-native applications includes containers-as-a-service (CaaS) and functions-as-a-service (FaaS). This breed of apps delivers ultimate scalability and availability. Combined with continuous integration and continuous deployment, software is delivered rapidly, which accelerates time-to-market. It brings closed-loop feedback among developers, testers, operators and end users.

    Cloud-native apps demand new skills of containerization, microservices and build and release management. However, the ROI is higher when compared to other app models. Greenfield applications that are still in the design phase are ideal for adopting the cloud-native pattern.

    Diverse Applications for Cloud Migrations

    Large enterprises moving to the cloud will deal with a diverse set of applications. Through hybrid connectivity and integration, spaghetti, modular, cloud-ready, cloud-optimized and cloud-native applications will co-exist and interoperate with each other. If you are after the benefits of modularity, make sure you do not trick yourself into a microservices-only mindset.

    Explore the in-process modularity features or frameworks of your favorite technology stack. You will get support to enforce modular design instead of having to just rely on conventions to avoid spaghetti code. Then, make a deliberate choice whether you want to incur the complexity penalty of microservices. Sometimes you have no choice, but often, you can find a better way forward.

    What This Means

    This blog is one of a series that explained how to engineer applications to be most suitable for DevOps. Topics covered in this blog series include:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers five levels of application maturity. You can read part one here, part two here, part three here, part four here and part five here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 5)

    Engineering Applications for DevOps (Part 5)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:

    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers DevOps applied to manufactured (embedded) systems software. You can read part one here, part two here, part three here and part four here.

    DevOps Applied to Manufactured (Embedded) Systems Software

    In the late 1970s (yes, forty years ago!) as a young engineer, I was given an assignment to figure out how to accelerate the software development and delivery cycles for an innovative packet switch called SL-10, (predecessor to the more recent model called Data Packet Node [DPN]) manufactured by Northern Telecom. Each SL-10 “node” consisted of millions of lines of original embedded software source code and custom hardware configured with many interconnected processors and protocols interfacing with other nodes, host devices, and terminal devices.

    Given the complexity of the system and lack of automated build, test and deployment tools, it was taking more than twelve months to generate each release of the system, and there typically were lots of bugs even after the release. Without knowing much about DevOps (the word “DevOps” was not invented for another twenty-nine years), we proceeded to implement a continuous delivery system that included automated build, test and delivery tools and workflows.

    The result was remarkable. Lead times for releases to manufacturing were reduced more than tenfold, from twelve months to one month, and quality, measured as defects found by customers per release, also reduced tenfold. This dramatic shift in performance and capabilities made it possible for the engineering and manufacturing teams to accelerate the rate of innovation and trample the competition. The SL-10 product, and its rebranded DPN version, quickly dominated the connection-oriented packet switch market, and it became the pride of the Nortel corporation and the Canadian nation. Even cooler, I got a nifty president’s pin and promotion which paved the way for my career that led to writing a book and this series of blogs! If this real-life example is not sufficient to demonstrate that DevOps can be applied to manufactured embedded software products, then I don’t know what is.

    There have been many other examples published that show how DevOps practices applied to embedded manufactured systems have reaped similar benefits. A notable example was published in 2013 by Gary Gruver in A Practical Approach to Large-Scale Agile Development: How HP Transformed LaserJet FutureSmart Firmware.

    That is not to say that DevOps applied to manufactured software embedded systems is the same as applying DevOps to enterprise applications or COTS systems. A paper published on the CloudBees blog, Five Aspects that Make Continuous Delivery for Embedded Different, explained some of the key DevOps practices used with embedded systems.

    DevOps Benefits for Embedded Systems

    Benefits of DevOps for embedded systems are summarized as follows:

    • Retrofitting the huge code base typically found in legacy embedded software systems into a DevOps model is not easy. It is likely to be expensive, but it is almost guaranteed to be a worthwhile effort, especially if you have a longer-term vision and intend for your product to stay competitive in the future.
    • Developers of embedded software need access to massive amounts of computer resources for testing their real-time embedded code on powerful hardware simulators.
    • Developers and testers need access to physical product hardware to run their binaries for system tests that cannot be run on simulators alone. Lab-as-a-service (LaaS) systems that orchestrate testing topologies of real devices can improve sharing of critical resources. LaaS systems provide a test framework that can be integrated through APIs into a continuous delivery pipeline.
    • Long build and test processes may need to be reengineered to reduce serialized lead times into quicker build times made possible by scaling build and test resources.
    • Long compliance and conformance test processes for embedded systems need to be accelerated with automation and resource scaling.
    • Focus on continuous delivery rather than continuous deployments in which shippable product versions are frequently delivered to manufacturing for verification, but the frequency of deployments to end customers is determined by other business factors.

    What This Means

    This blog is one of a series explaining how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers DevOps applied to manufactured (embedded) systems software. You can read part one here, part two here, part three here and part four here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 4)

    Engineering Applications for DevOps (Part 4)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:
    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covers DevOps applied to commercial off-the-shelf (COTS) systems. You can read part one here, part two here and part three here.

    DevOps Applied to COTS Systems

    I often hear that DevOps does not apply to systems that use commercial off-the-shelf (COTS) software. The idea that DevOps cannot be used with COTS systems is, frankly, a cop-out. Just because someone else designed and developed an application’s source code is not a sufficient reason to avoid engineering a DevOps solution.

    Consider the value stream chain of activities from the source vendor through integration and deployment in your environment. There are limitations and constraints to the level of automation allowed, the return on investment in automation and influence over the development life cycle that DevOps practices can introduce when dealing with COTS applications.

    COTS applications will typically have predefined and supported interfaces for managing configuration or accessing data. These endpoints may not be accessible or compatible with your existing toolchain or automated change management practices.

    Types of COTS Applications

    COTS applications typically come in one of three flavors; namely, closed, open and platform, and each should be evaluated differently.

    Closed COTS applications allow little to no customization to their functionality and/or interfaces. They typically have predefined management consoles and a published list of commands or APIs that will allow external applications to interact with the COTS applications. A good example of a closed COTS application is Microsoft Exchange.

    Key considerations for DevOps with closed COTS applications are as follows:

    • Can configurations be scripted and integrated with existing automation and orchestration toolchains?
    • Can configuration scripts be managed and maintained in external version control systems?
    • How are updates, patches and service packs applied and managed? Can these practices be automated?
    • What developer tools (APIs, software development kits [SDKs], etc.) are available to extend and/or interact with the application and/or data?
    • Are these tools compatible with existing tools and expertise in the enterprise?
    • Can these developer tools be integrated within the existing toolchain?
    • How frequently are changes made to the base product?

    If these applications can be managed by external systems and code developed to automate deployment and configuration and/or integration can be managed through a version-controlled pipeline, closed COTS applications make good candidates for DevOps and continuous delivery (CD). By applying CD principles, COTS binaries are versioned and stored in an artifact repository; installation and configuration scripts are managed through version control and IaaS workflows and applications and integration endpoints are auto-tested through deployment pipelines.

    This is only made possible when consumers of the COTS application collaborate with operators of the COTS platform to ensure that the solution is aligned with enterprise goals, objectives and standards related to portfolio management.

    Open COTS applications allow significant modifications to functionality, data and/or interfaces. These platforms typically have rich SDKs, APIs and embedded developer utilities that enable users and developers to modify all layers of the applications; namely presentation, business logic and data. Open COTS applications typically have a large, complex footprint consisting of core services, data and/or GUI customizations, embedded applications, etc. Classic examples of open COTS solutions include ERP/ CRM platforms (like SAP or Oracle) or portal platforms (like SharePoint, ServiceNow or WebSphere).

    Key questions to ask when considering whether DevOps can be applied to an open COTS application:
    • How are customizations (created through internal design editors, logic builders or data schema extensions) version controlled? Are these configurations stored in a database or code?
    • How are customizations created through internal tools and packaged, installed and configured in higher-level environments like staging or production?
    • What does the development life cycle for customization created with internal tools look like? What quality gates are required? How about architectural standards?
    • How are custom, embedded applications/applets packaged, installed and configured?
    • Can these custom applications/applets be run and tested separately from the underlying COTS application?
    • What does the development life cycle for customizations created with external tools look like? What quality gates are required? How about architectural standards?
    • Does the COTS application provide testing frameworks, mocks and/or stubs for testing external dependencies?
    • What automated test suites are supported by the platform (unit, functional, regression, capacity tests, etc.)?
    • What is the upgrade path for customizations (historically)? As you can imagine, open COTS applications add layers of complexity.

    When considering DevOps and CD with open COTS solutions, it is recommended that you build multiple pipelines to support each layer of the application in the dev or continuous integration (CI) stage of the value stream. Subsequent stages, like staging and production, will use a converged pipeline built from known good artifacts of the dev and CI stages.

    Before implementing a solution, it is important to study the development life cycle and practices of the target COTS application before building your toolchain. Understanding how the end-state platform is created and how change is introduced will strongly influence your toolchain and pipeline design.

    Platform COTS Applications provide a set of services and tools, as well as proprietary runtimes, that enable users to build and run custom applications on the platform. These are the most difficult type of applications to integrate into existing DevOps practices and CD toolchains because common practices like version control, testing and build are done through the platform rather than using external tools.

    Platform COTS are packaged applications that enable customers to build applications on top of the base platform. As a blend of closed and open systems, all the previous considerations are valid. In most cases, the base platform acts as a closed system, whereas the custom-developed applications on top act like open systems.

    For this type of application, it is critical to understand what hooks and services are available to support DevOps practices and CD. Many of these platforms are not going to fit within your existing toolchain and may not be able to employ the workflows, dashboards and reporting you have grown accustomed to from your current toolchain.

    What This Means

    This blog one of a series that explains how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covered DevOps applied to COTS systems. You can read part one here, part two here and part three here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 3)

    Engineering Applications for DevOps (Part 3)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • DevOps applied to software services.
    • Five levels of application maturity.

    This blog covers the topic of DevOps applied to enterprise apps and software services; part three in the series. You can read part one here and part two here.

    DevOps Applied to Enterprise Apps

    An enterprise application assists an organization in solving business problems. Enterprise applications are typically designed to interface or integrate with other enterprise applications used within the organization and to be deployed across a variety of networks (internet, intranet and corporate networks) while meeting strict requirements for security and administration management.

    Proprietary enterprise applications are usually designed and deployed in-house by specialized IT development teams within the organization. However, an enterprise may outsource some or all the development of the application and bring it back in-house for deployment.

    The size and complexity of large enterprise applications, especially older monolithic-architected applications that are not designed in modules, are a challenge when it comes to implementing DevOps. Given the large scale and distributed nature of the people, process and technology components of enterprise applications, it is recommended that enterprises engage very experienced DevOps consultants to determine strategic requirements, roadmaps and plans for DevOps that fully consider all three dimensions (people, processes and technologies) and nine pillars of DevOps before embarking on a major DevOps transformation journey.

    Despite the complexity, with the right strategy and plan, DevOps can deliver enormous ROI for enterprise applications. Amazon and Adobe are just two of many examples of large corporations that have benefited enormously from applying DevOps across the enterprise.

    DevOps Applied to Software Services

    I often hear people ask, “Does DevOps apply to software services as well as it does products?” These are usually people who have only worked in product-oriented organizations, such as companies that manufacture things that have embedded software.

    Ironically, I often hear people ask the opposite question, “Does DevOps apply to products with embedded software as well as it does to services?” These are usually people who have only worked organizations in which software is consumed as a service over an intranet or internet for a price or for free. Examples of these organizations are IT departments (even the IT departments of manufacturers!), government agencies and enterprises such as banks, insurance companies, hospitals and financial trading companies.

    You already know the answer to these two questions if you read the earlier parts of this blog series. The answer to both is clearly ‘Yes!’ Cloud services are doubly interesting to DevOps because not only are cloud services used by DevOps as a preferred infrastructure, as described in the next chapter, but cloud services also use DevOps in the process of creating and deploying software for the implementation of cloud components.

    While we are on the topic of DevOps and services, I want to help clarify some confusion regarding whether “newer” DevOps renders “older” IT service management (ITSM) obsolete or less important. This confusion exists because ITSM has often been regarded as a gatekeeper run by Ops to prevent Dev from getting out of control. While some organizations may have, in fact, implemented ITSM that way, this is not the purpose of ITSM.

    “DevOps tends to focus on the software delivery life cycle and on building and delivering software,” said Jayne Groll, president of the ITSM Academy. For its part, ITSM tends to have a broader focus; it is really looking at all aspects of customer engagement, from fulfilling standard services to ongoing support and operations activities. So, ITSM is “almost like the left and right ends of DevOps.”

    Using ITSM With DevOps

    Here are some recommended engineering practices for using ITSM with DevOps:

    • Training to help DevOps professionals gain an understanding of ITSM, and vice versa, will help both teams find points of integration—not just in technology, but in process, as well.
    • Find ways to automate service change approvals implementation of known, good changes to allow for agility, iteration and innovation. At the same time, hold back high-risk, high-impact changes that might need more than just an individual contributor to sign off on before they make it in front of a customer.
    • Self-service requests, automation flows and IT orchestration are now the front door to IT service management actions. These can and should be happening in the background, automagically created, maintained and measured by our DevOps tools.
    • Embrace Agile service management to take Agile values and apply Scrum-like development practices to process design and improvement. Aim to make more frequent incremental changes rather than design and implement a total end-to-end process in the traditional way.

    What This Means

    This blog is part three of a series to explain how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covered how DevOps can be applied to enterprise apps and software services. You can read part one here and part two here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 2)

    Engineering Applications for DevOps (Part 2)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:
    • Factors to consider when deciding whether an application is a good candidate for DevOps.
    • Best practices for engineering designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog is part two, and covers best practices for engineering designs for DevOps. You can read part one here.

    Poor design practices severely limit the potential for improvements provided by DevOps. As stated in my blog post Design for DevOps—Recommended Engineering Practices, “Without good design practices, a DevOps implementation has no hope of delivering on its promise of accelerating innovation with high quality and scale.”

    Following DevOps-recommended engineering practices for application design is important for legacy, greenfield or brownfield products, platform projects, feature enhancements or repairs. It is important for enterprise three-tier applications or multilayer products. And it is true whether the developers are using Agile, waterfall or ITIL development processes. In every application design case, designers face tough challenges and are expected to balance conflicting goals.

    Designers are expected to rapidly drive down the work backlog yet produce quality products that avoid costly rejections and rollbacks. In addition, there is pressure to increase the percentage of effort spent on creative content rather than corrective content and do so with limited time and resources. Balancing these conflicting goals is a tall order. If not supported by recommended engineering practices, the overall design experience is a cauldron of stress and a fast route to burnout.

    Recommended Engineering Practices

    The following recommended engineering practices improve the design experience and the products of that design in a way that is streamlined for the DevOps pipeline:

    • Designers must thoroughly understand customer use cases.
    • Usability, reliability, scaling, availability, testability and supportability are more important than individual features! Quality over quantity is recommended. Designs that anticipate actual customer use are the most successful.
    • The culture needs to support designers.
    • Leaders must support the designers with motivation, mentoring and training.
    • No designer can be expected to know everything. It is OK for designers to make some mistakes if lessons are learned and improvement quickly follows. Continuous monitoring and quick remediation are examples of good DevOps practices that help minimize the impact of any mistakes.

    Design Coding Practices

    Design coding practices are critical in the following ways:

    • Products are architected to support modular independent packaging, testing and releases. In other words, the product itself is partitioned into modules with minimal dependencies between modules. In this way, the modules can be built, tested and released without requiring the entire product to be built, tested and released all at once.
    • Where possible, applications are architected as modular, immutable microservices ready for deployment in cloud infrastructure, in accordance with the tenets of twelve-factor immutable non-monolithic apps, rather than monolithic mutable architectures.
    • Software code changes are pre-checked using peer code reviews prior to commit to the integration/trunk branch.
    • Software changes are integrated in a private environment together with the most recent integration branch version and tested using functional testing prior to committing the software changes to the integration/trunk branch.
    • Developers commit their code changes regularly—at least once per day.

    Good Design Practices Support QA and Ops

    Good DevOps design practices support QA in the following ways:

    • Design practices consider and understand the QA process.
    • Software code changes are pre-checked with unit tests prior to commit to the integration/trunk branch.
    • Software source code changes are pre-checked with static analysis tools prior to commit to the integration branch. Static analysis tools are used to ensure the modified source code does not introduce critical software faults such as memory leaks, uninitialized variables, and array-boundary problems.
    • Software code changes are pre-checked with dynamic analysis and regression tests prior to commit to the integration/trunk branch to ensure software performance has not degraded.

    Good DevOps design practices support Ops in the following ways:

    • Design practices consider and understand delivery and deployment pipeline processes.
    • Software features are tagged with software switches (i.e., feature tags or toggles) during check-in to enable selective feature-level testing, promotion and reverts.
    • Automated test cases are checked in to the integration branch when code changes are checked in. Evidence that the tests passed are included with the check-in.
    • Tests are conducted in a pre-fight test environment that is a close facsimile of the production environment.

    Supporting Tools

    The following supporting tools are needed to realize design for DevOps best practices:

    • Elastic infrastructure that can be easily orchestrated, created and released, as needed, to support designers’ tasks on-demand with minimal delay.
    • Design, code management, monitoring and test tools are readily available and scalable with minimal delay.
    • Monitoring tools that track application process performance and report the results to designers in easily consumable formats without delay.

    Applications for Which DevOps Does Not Apply

    “If you build it, they will come,” made for a great movie idea in Field of Dreams, but the reality is that many DevOps journeys led to nowhere despite the excellence of the people, processes and technology involved. As indicated above, applications that will not benefit from faster lead times, incur insufficient costs to justify investment in DevOps or that are too rigid to accept change are not good candidates for DevOps.

    What This Means

    This blog is part two of a series explaining how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps, and covers topics including:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covered practices used to engineer designs for DevOps. You can read part one here.

    For more information on how to engineer applications for DevOps, refer to my book Engineering DevOps.

  • Engineering Applications for DevOps (Part 1)

    Engineering Applications for DevOps (Part 1)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:
    • Factors to consider when deciding whether an application is a good candidate for DevOps.
    • Best practices for engineering designs for DevOps
    • DevOps applied to enterprise apps and software services
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity

    This blog covers factors to consider when deciding whether an application is a good candidate for DevOps.

    Factors to Consider for Engineering Applications

    The extreme worldwide hype around DevOps products and services seems to suggest that DevOps is good for everybody and applies equally well to every software-based product or service in every organization. Here is my experience with that assumption: It is not true!

    On the other hand, I have heard people say that DevOps does not apply to applications for which it absolutely does! Before you go a step further in your DevOps journey—or decide that your application does not qualify for DevOps—you should consider carefully how well your application is engineered to become a model for DevOps implementations.

    Best Practices for DevOps Determination

    The following are important factors you can use to decide whether an application is a good candidate for DevOps:
    • The application will benefit from faster lead times.
    • Leaders with responsibility for this application are open to collaboration and will be sponsors of change.
    • Team players that are associated with the application team (product owners, Dev, QA, Ops, infrastructure, security, PM) are open to collaboration and change.
    • The application is currently using or planning to use service-oriented, modular architectures or microservices.
    • There are at least ten people associated with the application including product owners, Dev, QA, Ops, infrastructure, security, PM. Smaller projects with smaller teams may not show sufficient impact or justify substantive DevOps investment.
    • The application is expected to undergo changes for more than a year. Less time would not likely yield return on the investment required for DevOps.
    • The application represents a good level of business impact and visibility but does not involve an extreme amount of risk for the business.
    • The application experiences frequent demands for changes from the business.
    • Not all tools used with the application need to be replaced to implement a DevOps toolchain. The costs of changing out all tools can negatively affect the ROI and business case for DevOps transformations.
    • Efforts to build, test or deploy releases of the application are substantial and could be reduced significantly by automation.

    It is not necessary for an application to perfectly fit all the factors, but the more closely it fits the more likely your application will benefit from using DevOps and will, therefore, be worth the cost to implement a well-engineered DevOps practice.

    DevOps can be used with all types of software applications and services. However, not all applications and services should be engineered to use DevOps. The factors that determine whether to engineer DevOps for an application have more to do with business factors than technology differences between different types of applications. Applications must be engineered in specific ways for DevOps to operate effectively.

    What This Means

    This blog is part one of a series explaining how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors to consider when deciding whether an application is a good candidate for DevOps.
    • Best practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity

    This blog covers the first two topics: factors to decide whether an application is a good candidate for DevOps and best practices to use when engineering designs for DevOps.

    For more information on engineering applications for DevOps, refer to my book Engineering DevOps.