Tag: devops approaches

  • Design for DevOps – Best Practices

    Design for DevOps – Best Practices

    Without good design practices, a DevOps implementation has no hope of delivering on its promise of accelerating innovation with high quality and scale.

    Anyone who has been a designer knows that good design for DevOps is easier said than done. My blog, “7 Pillars of DevOps,” identified seven essential best practices areas that make up successful DevOps. One of the seven pillars is Design for DevOps. The blog explains that the pillars are not silos, but rather foundations that need to be kept in balance. It also can be said that a good design goes a long way to make up for shortcomings in the other pillars.

    This is true for any number of designs, including designs for legacy, greenfield or brownfield products. It is true for platform projects, feature enhancements or repairs. It is true for enterprise three-tier applications or multilayer products. And it is true whether the developers are using Agile, Waterfall or ITIL processes.

    In every case, a designer faces tough challenges and is expected to balance conflicting goals. Designers are expected to rapidly drive down the work backlog, yet produce quality products that avoid costly rejections and rollbacks. In addition, there is pressure to increase the percentage of effort spent on creative content over corrective content and do so with limited time and resources. These conflicting goals constitute tall responsibilities. If not supported with best practices, the overall design experience is a cauldron of stress and fast route to burnout.

    The following best practices improve the design experience and products of design in a way that is streamlined for the DevOps pipeline:

    Designers must thoroughly understand customer use cases

    Usability, reliability, scaling, availability, testability and supportability are more important than individual features! Quality over quantity is recommended. Designs that anticipate actual customer usage are the most successful.

    The culture needs to support designers

    Leaders must support the designers with motivation, mentoring and training. No designer can be expected to know everything. It’s OK for designers to make some mistakes if lessons are learned and improvement quickly follows. Good DevOps practices including continuous monitoring and quick remediation are examples of practices which help minimize the impact of any mistakes.

    Design coding practices are critical:

    • Products are architected to support modular independent packaging, testing and releases. In other words, the product itself is partitioned into modules with minimal dependencies between modules. In this way the modules can be built, tested and released without requiring the entire product to be built, tested and released all at once.
    • Where possible, applications are architected as modular, immutable microservices ready for deployment in cloud infrastructures, in accordance with the tenets of 12-factor immutable non-monolithic apps, rather than monolithic mutable architectures.
    • Software code changes are pre-checked using peer code reviews prior to commit to the integration/trunk branch.
    • Software changes are integrated in a private environment together with the most recent integration branch version and tested using functional testing prior to committing the software changes to the integration/trunk branch.
    • Developers commit their code changes regularly, at least once per day.

    Good DevOps design practices support QA:

    • Understand the QA process.
    • Software code changes are pre-checked with unit tests prior to commit to the integration/trunk branch.
    • Software source code changes are pre-checked with Static Analysis tools prior to commit to the integration branch. Static analysis tools are used to ensure the modified source code does not introduce critical software faults such as memory leaks, uninitialized variables, and array-boundary problems.
    • Software code changes are pre-checked with dynamic analysis and regression tests prior to commit to the integration/trunk branch to ensure the software performance has not degraded.

    Good DevOps design practices support Ops:

    • Understand delivery and deployment pipeline processes.
    • Software features are tagged with software switches (i.e. feature tags or toggles) during check-in to enable selective feature level testing, promotion and reverts.
    • Automated test cases are checked-in to the integration branch at the same time code changes are checked-in. Evidence that the tests passed are included with the check-in.
    • Tests are conducted in a pre-flight test environment that is a close facsimile of the production environment.

    Supporting tools are needed to realize Design for DevOps practices:

    • Elastic infrastructures that can be easily orchestrated, created and released as needed to support designers tasks on-demand with minimal delay.
    • Design, code management, monitoring and test tools readily available and scalable with minimal delay.
    • Monitoring tools that track application process performance and report the results to designers in easily consumable formats without delay.

    Design for DevOps Summary

    What do you think? Do you agree the above are important best practices for DevOps design? Are there others you recommend?

    While DevOps implemented with all seven pillars provides a strong foundation for long-term enterprise business improvements, it is important to understand DevOps is not an island. Enterprises implementing DevOps should be aware that DevOps interoperates with other IT systems and practices. Enterprises are well-advised to choose tool-agnostic IT partners that can provide solutions that best suit the needs of each unique enterprise and can integrate and evolve DevOps together with all of their IT systems.

    — Marc Hornbeek

  • 7 Pillars of DevOps: Essential Foundations for Enterprise Success

    7 Pillars of DevOps: Essential Foundations for Enterprise Success

    As indicated by the “2016 State-of-DevOps Report,” multiple compelling enterprise business performance improvements provided by well-built DevOps solutions are driving IT transformations for development and operations across many industries.

    While some enterprises have implemented DevOps to gain near-term efficiency benefits, DevOps is becoming a competitive essential for all enterprises to implement DevOps. Enterprises that fall behind risk long-term success in their markets. Yet, many organizations are struggling to get DevOps working effectively. What is the secret to success with DevOps?

    A solid foundation for DevOps requires mastery of seven areas that we at Trace3 refer to as “The Seven Pillars of DevOps.” These pillars are all necessary to support effective DevOps delivery pipelines together with end-to-end orchestration of all pipeline stages and workflows. If any of the pillars falter, the DevOps pipeline will fail to produce high-performing benefits of the most successful enterprises.

    So what are these essential seven DevOps pillars?

    1. Collaborative Culture: End-to-end goal alignment and cooperative workflows are essential. No amount of technology will suffice if Dev, QA and Ops leaders and teams do not cooperate.
    2. Design for DevOps: DevOps prefers products that are designed in modular fashion using microservices, 12-factor apps and containers—those are easily integrated into high-performance DevOps. That is not to say legacy products cannot benefit from DevOps, but modular, immutable architectures are most readily suited to take full advantage of DevOps.
    3. Continuous Integration (CI): Merging forward quickly with competent integration practices that minimize rollbacks, interruptions and costly delays in the pipeline.
    4. Continuous Testing (CT): Relevant quality testing at speed requires multiple strategies that ensure efficient coverage throughout the pipeline with minimal failures delivered to production, yet quick enough to not bottleneck the pipeline.
    5. Continuous Monitoring (CM): Real-time lifecycle intelligent active monitoring and analytics are essential to accelerate real-time decisions at each stage of the pipeline and avoid bottlenecks.
    6. Elastic Infrastructure: Infrastructures that are resilient and elastic perform much better. DevOps applies to most types of infrastructures including private data centers, special purpose bare-metal systems, virtualized functions, containerized packaged applications, private and public clouds and hybrid cloud environments.
    7. Continuous Delivery (CD): Modular delivery packaging preparation and deployment solutions with orchestration of virtualized and containerized applications are preferred over monolithic mutable applications.

    While DevOps implemented with all seven pillars provides a strong foundation for long-term enterprise business improvements, it is important to understand also that DevOps is not an island. Enterprises implementing DevOps should be aware that DevOps interoperates with other IT systems and practices. Enterprises are well-advised to choose tool-agnostic IT partners that can provide solutions that best suit the needs of each unique enterprise and can integrate and evolve DevOps together with all enterprise IT systems.

    We believe DevOps is an essential part of enterprise success. Keep a look out for future blogs describing best practices for each of the seven pillars of DevOps in coming months.

  • Measuring the Cost of Service Creation & Adoption

    Measuring the Cost of Service Creation & Adoption

    Adopting and implementing DevOps isn’t an overnight project. It takes time and effort to determine how it best fits into your organization—and what benefits your organization will realize as a result.

    Creating an enterprise-class DevOps service is typically done in one of three ways, depending on the organization, its leadership and its employees. Each has its pros and cons, and each has a number of factors to consider.

    A Fully Centralized Model

    In this model, tooling decisions are done in a central service provider group for the entire organization. Tools are managed centrally and standards are created centrally. DevOps automation creation (at least the templates or frameworks) is done centrally. This model is highly advantageous for organizations with a wide variety of technology platforms (think large population of legacy apps) and use many technology components for their apps, such as multiple types of databases, middleware and operating platforms, for example.

    Not many large organizations are ready to scrap what they have in the application portfolio and start over using the cloud and containers and develop in a singular methodology. Using a centralized model, companies can continue to improve upon their years of investment in legacy applications and adopt a DevOps continuum without starting over from scratch.

    This also is the lowest-cost model, as tooling is standardized and administered from a central point and licensing can be negotiated to maximum effect. In addition, labor (generally the highest ongoing cost of any service creation and implementation) is minimized by centralizing it. Training costs also can be minimized by establishing a centralized training program for all impacted audiences. Companies can change their incentive programs to achieve hiring objectives and business results as priorities change—something that isn’t practical in a decentralized or federated model.

    A Fully Decentralized Model

    In a fully decentralized model, tooling decisions are determined and managed by each development team. This has the advantage for shops with small numbers of applications or singular methods of development—fewer applications are better candidates for starting over with cloud or container integrations and can take advantage of the newest technologies from the starting point, if needed.

    Shops such as Etsy or Facebook seem massive, but from a user point of view, they really only present a single application that needs to be maintained and improved over time. Instances like this may favor a decentralized model because the actual number of application candidates may be highly limited—sometimes only one.

    This model typically represents the highest costs, as repeatability and standardization can only be measured at the development team level. As such, this is practical for a very limited number of applications, or technologies; if those numbers grow, the strategy quickly becomes untenable. In addition, this strategy has an uncanny history of pulling development resources to do the DevOps automation engineering that needs to be created and maintained. Typically, the highest skilled developers are assigned these types of duties, and thinking in terms of templates and frameworks is rarely their first approach. Developers tend to be expedient, and believe they will always be there to make sure it works. This also can reduce the potential for repeatability and predictability (raising costs).

    A Federated Model

    In a federated model, tooling decisions and standards are done centrally, but management and operations of the tools are performed by a limited number of subservient organizations (often by line of business). Centralized templates may or may not be viable depending on the variability of the business units they serve. The chief advantage of this approach is the segregation of tooling and reporting by business unit (typically) without having to build that kind of segregation in to tools that were not designed that capability in mind.

    This model also has the advantage of increasing the speed of adoption, as several parallel efforts can take place at the same time and spreading engineer resource demand across multiple organizations. It typically makes the most sense in larger organizations with multiple business units whose application portfolios that would benefit from an onboarding to DevOps.

    What’s more, a federated model may be the most cost-efficient. Labor costs will be inherently higher as functions are replicated from business unit to business unit; however, that replication will result in increased adoption rates and project completions (you get what you pay for). If business units are able to leverage centralized templates for each technology type, adoption rates become an order of magnitude faster (or have that potential). Tooling costs still can be negotiated centrally, even though management will occur in the business units. Again, there is hardware replication from business unit to business unit, but the ability to segregate users, permissions and operations down to the business-unit level can save tremendous labor costs if development was the only other option to create that capability in tools not designed for it.

    Adoption Costs

    Measuring the cost of adoption requires focus, no matter which method you employ to create the DevOps service. Applications moving to DevOps are said to be onboarded (where the original build, deploy, test and release automation is created). To successfully onboard an application requires education of the sponsoring executives, training of the impacted developers and testers, and coordination with the change and release managers.

    Collection of meta data about the behavior of application components is no small task, requiring frequent follow-ups and action items to reach completion. The effort to onboard applications to DevOps becomes the first cost for estimation and tracking as the application portfolio is attacked.

    Even after onboarding of a given application is complete, costs continue. The terms “continuous delivery” and “continuous integration” should also infer that application development methods “continuously” evolve and may adopt new technology components. Significant changes may occur that require revisiting the DevOps automation to adapt to these ongoing revisions and updates.

    While this work is rarely as extensive as the initial onboarding, it is measurable. Work of this nature is usually referred to as BAU (business as usual) and represents the second significant cost of DevOps services that must be measured and tracked. If no thought is given to this work effort, inevitably the team assigned to do onboarding will be distracted by the BAU requirements until they reach the point where no new applications can be completed.

    Staffing cost estimations can be presented at the component level for both onboarding and BAU efforts. For example, if one DevOps engineer can onboard 20 components per week (assuming familiar technologies within a given portfolio), and if the general ratio of components per application is 4:1, then a given DevOps engineer can successfully onboard five applications per week. If the engineer is never distracted with BAU, this can continue until the entire portfolio is onboarded. However, my experience has shown that for every 20 applications completed with onboarding, the BAU work effort can absorb nearly a full DevOps engineer. So if you start with a team of 10 people, you will onboard only 200 applications total before the entire team is required to focus on BAU work effort and has no more bandwidth to continue onboarding new apps. Understanding this demand, and learning how efficient your particular engineering team is, will be key to managing costs while completing the onboarding efforts of large application portfolios.

  • Step-By-Step: How DataArt Processes Application Backlogs, Converting Legacy Apps with DevOps

    Step-By-Step: How DataArt Processes Application Backlogs, Converting Legacy Apps with DevOps

    DataArt designs and custom builds software for industries such as travel & hospitality, media, IoT, financial services, and healthcare. “We build new products and services, improving existing solutions, re-engineering legacy systems, and providing expertise in technological and industrial domains,” says Anton Krasikov, Project Manager, DataArt.

    How DataArt Uses DevOps

    DataArt quickly delivers high quality software in support of operations performance. DataArt’s DevOps process enlists version control systems like Git and build systems such as Ant, Maven, and Gradle. “Almost every project relies on using these build systems inside CI servers like Jenkins, TeamCity, or TFS,” says Krasikov.

    DataArt’s development process also requires a lot of unit and integration testing using frameworks like TestNG and Junit. “There are also projects where we prefer TDD or BDD approaches, with the use of specific frameworks such as Selenium for UI testing,” says Krasikov. By using Puppet, Vagrant, and Chef, DataArt can swiftly deploy the development environment in the CD pipeline. DataArt applies these select DevOps development tools using methodologies, which could include Kanban or Scrum (Scrum was most frequently mentioned by DataArt for this story).

    Previously, DataArt relied on manual processes or scripting tools for build systems, source code, and unit tests. “It took a lot of management effort to tie everything including development, quality assurance, and operations together,” says Krasikov.

    Challenges to Adding Backlogged Applications to Those You Develop Using DevOps Approaches

    In the instance where DataArt returns to work on existing systems that include some legacy applications, it is possible to use agile development in maintaining these as they are, but excessive software dependencies could make it hard to align the software releases / release cycles.

    “Legacy software may not have good test coverage or a proper build system, so we usually take steps in a DevOps direction, if not to achieve continuous delivery, at least to implement proper continuous integration,” says Krasikov.

    There are challenges to adding backlogged apps to a DevOps development process. “These kinds of non-DevOps applications usually suffer from several issues, or in worst case scenarios from all of them,” says Krasikov. Backlogged apps waiting in a pre-DevOps state and hungry for transformation may have any or all of the following symptoms, according to Krasikov:

    *many dependencies between system modules
    *no automated environments
    *no continuous integration
    *no test automation
    *no build system or inefficient usage

    Transforming an Application, Step-By-Step

    To add apps to DevOps approaches, says Krasikov, start by analyzing the existing source code, system dependencies, and system environments. The steps that follow come from an example with a recent Java project that DataArt did.

    A) Select a source code control system such as Git.

    B) Do refactoring and perform unit tests. “These go hand-in-hand, so the main business logic was covered with tests. It does not make sense to rush to 100% test coverage, but we tested the main bits. This also allowed us to reduce system complexity and decouple system components,” says Krasikov.

    C) Build system – “The original application used huge and confusing Ant scripts to produce deployable packages. After we completed the refactoring, we split the application into modules with Maven as the build and dependency management system,” says Krasikov.

    D) Integration tests – “We introduced these as a separate test automation suite to ensure that application modules work correctly as a whole,” says Krasikov.

    E) Continuous Integration – DataArt set up a Jenkins server for multiple build jobs, building with each commit and targeting deployments to their intended environments with overnight integration testing. “CI allowed us to identify non-working builds early to save time on investigations,” says Krasikov.

    F) Environment setup – Finally, DataArt implemented infrastructure as code. The new development environment uses Puppet with Vagrant to check out source code, spin up a VM box / template, and deploy fast, according to Krasikov.

    DataArt runs some steps in parallel for additional time savings, for example moving to virtualized provisioned environments along with source code migration, according to Krasikov.

    DevOps Helps DataArt Speed Projects to Their Customers’ Delight

    By increasingly converting backlogged apps to DevOps development approaches, the enterprise benefits by achieving project goals and delivering a working software iteration with each pass through the DevOps processes. “The high level of automation reduces the risk of human errors and helps us to fail early in the development cycle. With continuous delivery, it is much easier for our clients to see the immediate impact of their ideas,” says Krasikov.