devopsy.kinsta.cloud

Category: Doin’ DevOps

  • SecDevOps is the Solution to Cybersecurity

    SecDevOps is the Solution to Cybersecurity

    Anybody paying attention to world news will notice that the threats and risks perpetrated by cybercriminal actors, in many forms, is a serious problem on the rise, affecting individuals, organizations and nations daily. The cybersecurity threat space is indeed alarming and growing rapidly. The cybersecurity solution space is struggling to keep up. After all, there is no such thing as 100% security.

    What is the Solution to Cybersecurity’s Problem?

    In my opinion, the industry has long underestimated the problem of cybersecurity, which has resulted in solutions that do not sufficiently stand up to the cybersecurity threat space. What is clear is that solutions must come from rapid software changes that detect and plug security holes as fast as they are found, and continuously deploy innovative defensive solutions that keep bad actors off their guard. To do that requires Agile and DevOps continuous delivery practices coupled with a security-aware mindset.

    What is a Security Aware Mindset?

    To put things in perspective, it is useful to consider the mindsets of design, QA, operations and security.

    Design mindset – create a product or service that customers want, now. The Agile framework addresses this requirement.

    QA mindset – verify a product or service works the way customers expect it to, when it is released, for all production configurations available at the time of the release. The DevOps framework addresses this requirement.

    Ops mindset – ensure a product or service continues to work the way customers expect, for all production configurations, for the life of the product or service. The site reliability engineering (SRE) and ITIL frameworks address this requirement.

    Security mindset – defend valuables for organizations that create or use a product or service against unintended and malicious actions, by people and organizations, both inside and outside of the organizations, for all production and non-production configurations, for as long as there is something of value to protect. The DevSecOps framework was created to address this requirement; as we have seen, this does not yet offer sufficient safeguards for the growing cybersecurity threat problems.

    The security mindset has multiple additional vectors of thought required compared to the design, QA and ops mindset. This helps us to understand why the cybersecurity problem space is bigger and requires more comprehensive solutions.

    Not long ago I considered security assurance to be a subset of testing practices. In 2016, in my blog, “Seven Pillars of DevOps – Essential Foundations for Enterprise Success“, I did not break out security as a separate pillar. At that time, this type of thinking was typical of developers. Security was someone else’s job. Perimeter defenses were deemed sufficient.

    Soon after, I realized that way of thinking was naïve on my part. By the time that I co-authored a subsequent blog, “Nine Pillars of Continuous Security Best Practices,” I added a separate pillar for security specific practices, and also added security practices within each of the other pillars.

    In parallel, DevSecOps became a thing in the DevOps world. The idea of integrating security practices into DevOps makes a lot of sense, because DevOps is the answer to continuous delivery of software, and security needs continuous delivery of defensive solutions. However, simply adding security to DevOps is not a sufficient solution, because DevOps was conceived to address specific problems associated with design, QA and Ops – not security. Simply adding security as another metric for releases is not enough. What is needed is a re-think of how DevOps can be used to deliver continuous security, as indicated in my book, “Engineering DevOps.” This is the same idea behind what is referred to as SecDevOps.

    In my opinion, it is now time for SecDevOps to gain favor over DevSecOps when it comes to integrating security practices with DevOps practices. This represents a much more significant shift than simply putting security first in the name. With SecDevOps, security concerns within the entire development organization and at every stage in the DevOps value stream, are explicitly prioritized over other types of concerns. This makes sense, given the much wider blast radius of security events versus other quality concerns that DevOps was originally designed to address.

    With SecDevOps, security tools and resources are weighted at the highest priority level. SecDevOps and secure coding practices are required training for application developers. Threat modeling is part of the value stream. Security tasks that are required to mitigate security risks are given the highest priority in backlog planning. At each stage in the DevOps pipeline, security concerns rise to the top of the priority list. Release and deployment readiness are weighted at the highest level. In short, SecDevOps explicitly plugs some important security holes that are not covered by DevSecOps. With SecDevOps, security becomes continuous, not just an add-on to DevOps.

    What This Means

    By explicitly plugging some important security holes that are not covered by DevOps or DevSecOps practices alone, it makes sense for SecDevOps to now emerge as the preferred framework to close the gaps between the growing security problem space with the evolving security solution space. SecDevOps offers the required security-first mindset and solves the need for rapid software changes that detect and plug security holes as fast as they are found, and continuously deploys innovative defensive solutions to keep bad actors off their guard.

  • DevOps Practices Gap Assessment

    DevOps Practices Gap Assessment

    Using DevOps to continuously deliver software has proven value for organizations that are on a path towards digital transformation and continuous improvement. To get the maximum value from DevOps, it is important to use recommended best practices, inclusive of people, processes and technologies. A gap assessment is a great way to efficiently evaluate an organization’s practices for DevOps, and determine a strategy for improvement. Gap analysis provides valuable inputs for formulating a strategy and roadmap to improve a topic including DevOps.

    My SlideShare, Gap Survey, Assessment and Analysis For DevOps, explains how my Nine Pillars Gap Assessment Tool greatly speeds up the gap survey, gap assessment and gap analysis process for DevOps.

    It is important to clarify a few definitions for this blog, because I have found that different people and organizations use these terms differently. In this blog, and in my consulting work, I use the following definitions.

    • Gap survey is a discovery tool used to collect information about the current state of practices for a topic.
    • Gap assessment is a process to determine differences (gaps) between current practices and recommended practices for a topic. Gap assessment is also used to refer to the entire collection of activities in this list.
    • Gap assessment workshop is a meeting in which the gap assessment is validated prior to gap analysis.
    • Gap analysis is the process of determining priorities for reducing gaps found during a gap assessment.
    • Gap assessment tool is software that helps perform gap surveys, gap assessments and gap analysis.

    For those that want to dig deeper on this topic, additional definitions and contextual information related to the way I do gap assessments are clarified in my book Engineering DevOps.

    I define a seven-step process for gap assessments. The steps are as follows:

    1. Pick a topic to analyze. The same processes and tools can be applied equally well to almost any topic.
    2. Determine recommended practices for the topic. Something that is worth doing a gap assessment for, such as DevOps, will normally be fairly complex with dozens, if not hundreds of practices. Categorize the practices into groupings (I call the groups “pillars”) to make the list of practices more understandable and meaningful. To do this will require some research and work. For example, I have a database of practices and pillars for DevOps, continuous testing, DevSecOps and SRE and some other topics such as DevOps.
    3. Enter the practices from step two into a gap assessment tool. The tool can be a simple survey tool you create, a commercial survey tool or something custom-made.
    4. Determine who needs to be included in the gap survey.
    5. Collect data using the gap survey.
    6. Perform a gap assessment.
    7. Perform a gap analysis.

    For the remainder of this blog, I explain the above seven steps for DevOps.

    Step one: Pick a Topic to Analyze
    In this blog, the topic is DevOps.

    Step two: Determine Recommended Practices for DevOps
    For an example DevOps assessment, I used the practices under the nine practices categories described in a blog that I co-authored called Nine Pillars of DevOps Best Practices. The nine practice category pillars are:

    • Leadership
    • Collaborative culture
    • Design for DevOps
    • Continuous integration
    • Continuous testing
    • Continuous monitoring
    • Elastic infrastructure
    • Continuous delivery
    • Continuous security

    Step three: Enter the Practices from Step Two into a Gap Assessment Tool
    A gap assessment tool, pre-loaded with a sample of DevOps practices, in a file called DevOps Assessment, can be downloaded for free from one of the resource pages found on my website Engineering DevOps. You can edit the practices categories and add/delete practices from each category if you prefer to make changes.

    Step four: Determine Who Needs to be Included in the Gap Survey
    For the gap analysis to be comprehensive, people in roles that are affected by DevOps practices need to be surveyed, or at least represented, to ensure their perspectives are included. The following are example roles that typically are included.

    • Business leaders – because they influence culture and budgets for DevOps.
    • Developers – because they need to design in accordance with DevOps practices.
    • Project owners – because they influence product work priorities.
    • QA testers – because they need to test in accordance with DevOps practices.
    • Ops – because DevOps practices affect operations.
    • Security – because DevOps affects security practices.

    The survey can be conducted for an individual application, a group of applications, or all the applications in the enterprise. However, it is important that the gap assessment and gap analysis be performed on the organization segment that is being targeted for improvement.

    Step five: Collect Data Using the Gap Survey
    A gap survey should allow each surveyed person to enter an importance level score, a practice level score and comments for each practice. All this information is essential for the gap assessment and gap analysis. In the gap survey included within my gap assessment tool, survey respondents are asked to score the importance of each practice as one of 0=not relevant, 1=not important, 2=nice to Have, 3=important, 4=very important or 5=critical. Practice level choices are 0=not sure, 1=rarely, if ever, 2= sometimes, 3=most of the time, 4=always or 5=We are really good at this. Comments that are relevant to qualify or explain the scores entered for each practice should also be entered, especially if there is any doubt or ambiguity.

    Step six: Perform a Gap Assessment
    The gap assessment process requires all the practices scores collected from the surveys to be collected and assembled to calculate an aggregate set of scores. Gap scores are calculated using a formula that weights each practice level score with the corresponding importance level score. A visual representation helps to identify practices areas and individual practices that have the highest, most important gaps.

    No matter how professionally written the practices and score definitions are, it is not unusual for some people to misunderstand and to enter scores that they would not have otherwise. For this reason, it is important to ensure that the data collected is validated before conducting the gap analysis. The preferred approach is to conduct a gap assessment workshop with key representatives from each role that participated in the survey. During the workshop, the scores for each practice that have a high deviation between survey responses are discussed and, if necessary, adjusted.

    Step seven: Perform a Gap Analysis
    The gap analysis process involves extracting the high gap practices and tagging and ranking each of them against solution categories determined by a consultant or topic expert.

    The results of the gap analysis indicate where solution strategies and implementation roadmaps need to be focused to reduce the most important gaps.

    What This Means

    While it is true you can easily stand up DevOps tools quite quickly, this alone will not realize the benefits of following effective and safe DevOps practices. Those who are serious about DevOps will do well to learn from others, and be serious about using recommended DevOps practices. This blog explained a gap assessment approach, which uses my nine pillars of best practices for DevOps. My free DevOps gap assessment tool facilitates getting a gap analysis for DevOps and priorities to improve DevOps practices for an organization.

  • DevSecOps Practices Gap Assessment

    DevSecOps Practices Gap Assessment

    DevSecOps practices are essential to deliver software safely and securely withinDevOps value streams.

    To get the maximum security protection from DevSecOps, it is important to use recommended best practices, inclusive of people, processes and technologies. A gap assessment is a great way to efficiently evaluate an organization’s practices for DevSecOps and determine a strategy for improvement. Gap analysis provides valuable inputs for formulating a strategy and roadmap to improve a topic including DevSecOps.

    My SlideShare Gap Survey, Assessment and Analysis For DevSecOps explains how my 9 Pillars Gap Assessment Tool greatly speeds up the gap survey, gap assessment and gap analysis process for DevSecOps.

    It is important to clarify a few definitions for these purposes, because I have found that different people and organizations use these terms differently. In this blog, and in my consulting work, I use the following definitions:

    • Gap survey is discovery tool used to collect information about the current state of practices for a topic.
    • Gap assessment is a process to determine differences (gaps) between current practices and recommended practices for a topic. Gap assessment is also used to refer to the entire collection of activities in this list.
    • Gap assessment workshop is a meeting in which the gap assessment is validated prior to gap analysis.
    • Gap analysis is the process of determining priorities for reducing the gaps found during a gap assessment.
    • Gap assessment tool is software that helps perform gap surveys, gap assessments and gap analysis.

    For those that want to dig deeper on this topic, additional definitions and contextual information related to the way I do gap assessments are clarified in my book Engineering DevOps.

    I define a seven step process for gap assessments. The steps are as follows:

    1. Pick a topic to analyze. The same processes and tools can be applied equally well to almost any topic.
    2. Determine recommended practices for the topic. Something that is worth doing a gap assessment for, such as DevSecOps, will normally be fairly complex with tens, if not hundreds, of practices. Categorize the practices into groupings (I call the groups “pillars”) to make the list of practices more understandable and meaningful. Doing this will require some research and work. For example, I have a database of practices and pillars for DevOps, continuous testing, DevSecOps and SRE and some other topics such as containers.
    3. Enter the chosen practices into a gap assessment tool. The tool can be a simple survey tool you create, a commercial survey tool or something custom-made.
    4. Determine who needs to be included in the gap survey.
    5. Collect data using the gap survey.
    6. Perform a gap assessment.
    7. Perform a gap analysis.

    Let’s take a look at the above seven steps for DevSecOps.

    Step 1: Pick a Topic to Analyze

    For our purposes here, the topic is DevSecOps.

    Step 2: Determine Recommended Practices for DevSecOps

    For an example DevSecOps assessment, I used the practices under nine practices categories described in a blog that I co-authored called 9 Pillars of Continuous Security Best Practices. The nine practice category pillars are:

    • Leadership
    • Collaborative culture
    • Design for DevOps
    • Continuous integration
    • Continuous testing
    • Continuous monitoring
    • Elastic infrastructure
    • Continuous delivery
    • Continuous security

    Step 3: Enter the Selected Practices into a Gap Assessment Tool

    You can find and download a free gap assessment tool, pre-loaded with a sample of DevSecOps practices in a file called DevSecOps Assessment, from one of the resource pages found on my website, EngineeringDevOps. You can edit the practices categories and add/delete practices from each category if you prefer to make changes.

    Step 4: Determine Who to Include in the Gap Survey

    For the gap analysis to be comprehensive, people in roles that are affected by container practices need to be surveyed, or at least represented, to ensure their perspectives are included. Some example roles that are typically included:

    • Business leaders – because leaders influence culture and budgets for DevSecOps.
    • Developers – because developers need to design in accordance with DevSecOps practices.
    • Project owners – because they influence product work priorities.
    • QA testers – because they need to test in accordance with DevSecOps practices.
    • Ops – because DevSecOps practices affect operations.
    • Security – because DevSecOps practices affect security practices.

    The survey can be conducted for an individual application, a group of applications or all the applications in the enterprise. However, it is important that the gap assessment and gap analysis be performed on the organizational segment that is being targeted for improvement.

    Step 5: Collect Data Using the Gap Survey

    A gap survey should allow each surveyed person to enter an “Importance Level” score, a “Practice Level” score and comments for each practice. All this information is essential for the gap assessment and gap analysis. In the gap survey included in in my gap assessment tool, survey respondents are asked to score the importance of each practice as one of 0=not relevant, 1=not important, 2=nice to have, 3=important, 4=very important or 5=critical. Practice-level choices are 0=not sure, 1=rarely, if ever, 2= sometimes, 3=most of the time, 4=always or 5=we are good at this. Comments that are relevant to qualify or explain the scores entered for each practice should also be entered, especially if there is any doubt or ambiguity.

    Step 6: Perform a Gap Assessment

    The gap assessment process requires all the practices scores collected from the surveys to be collected and assembled to calculate an aggregate set of scores. Gap scores are calculated using a formula that weights each practice level score with the corresponding importance level score. A visual representation helps to identify practices areas and individual practices that have the highest, most important gaps.

    No matter how professionally written the practices and score definitions are, it is not unusual for some people to misunderstand and to enter scores that they would not have otherwise. For this reason, it is important to ensure that the data collected is validated before conducting the gap analysis. The preferred approach is to conduct a gap assessment workshop with key representatives from each role that participated in the survey. During the workshop, the scores for each practice that have a high deviation between survey responses are discussed and, if necessary, adjusted.

    Step 7: Perform a Gap Analysis

    The gap analysis process involves extracting the high gap practices and tagging and ranking each of them against solution categories that are determined by a consultant or topic expert.

    The result of the gap analysis indicates where solution strategies and implementation roadmaps need to be focused to reduce the most important gaps.

    What This Means

    While is it clear that DevOps offers immense value for software deployment, the adherence to best practices for DevSecOps is essential to reduce risk and assure security. Each organization is different, and has different security postures. The specific practices, security tools and metrics that are appropriate may vary according to a risk assessment for each organization. This blog explained a gap assessment approach, which uses my nine pillars of best practices for DevSecOps. My free DevSecOps assessment tool facilitates a gap analysis for DevSecOps and priorities to improve DevSecOps practices for an organization.

  • Unified Enterprise DevOps Platforms Eliminate Bottlenecks

    Unified Enterprise DevOps Platforms Eliminate Bottlenecks

    Nearly all enterprise digital transformation projects include moving software to DevOps processes and platforms. The effort, cost and time it takes to be successful with these transformations depend on a lot of factors, but one thing stands out: as indicated in my white paper Unified Enterprise DevOps Platform Why? How? And What?, “An enterprise with many variations of DevOps platforms and tools is at best chaotic, and at worst, is a gross impediment to overall organization agility – undermining the very purpose that DevOps is meant to serve.”

    In my recent blog Why a Unified Enterprise DevOps Platform is Necessary, I indicated that “The aggregate set of platforms and tools is like a Frankenstein’s monster, with many incompatible parts resulting in an awkward looking product that is hard to control, replicate and evolve.”

    Success with DevOps requires frictionless interplay between the people, processes and technologies that need to cooperate for continuous deliveries to happen effectively. The existence of different DevOps solutions creates organizational barriers to unification, because different groups become dependent on their local variations.

    A Unified Enterprise DevOps Platform Accelerates Time-to-Value

    Culture barriers, process problems and DevOps platform “wars” all are factors impacting the progress of digital transformations. A unified enterprise DevOps platform would accelerate time-to-value, and realize efficient, safe operation for digitally transforming enterprises and organizations.

    The following are general requirements for a unified enterprise DevOps platform.

    ● A new, unified approach that supports end-to-end capabilities for continuous delivery of software, as a managed service, for multiple streams of pipelines.
    ● Unified metrics that show business value, costs of the value stream and timing for end-to-end continuous delivery flow across multiple pipelines.
    ● Unified capabilities including visibility, orchestration, integration, security, governance, traceability and management of flow for application value streams across multiple pipelines.
    ● Capabilities to unify continuous testing/QA, troubleshooting and debugging.
    ● A new, unified approach to simplify and accelerate toolchains that support new cloud-native and cloud-adapted applications, infrastructures, machine learning, commercial off-the-shelf and open source software.

    A DevOps Platform Blueprint

    In my SlideShare DevOps-as-a-Service Values, I described a four-layer blueprint for a unified enterprise DevOps platform that satisfies these general requirements. The main elements in the blueprint are:

    ● Stakeholders – There are three types of stakeholders: business, appdev users and DevOps admin teams.
    ● Service access layer – Consists of portals and APIs for business, appdev and DevOps administration users and access security.
    ● Service layer – consists of user services and DevOps admin services.

      • User Services include things that appdev users need to perform their application development and testing work. This includes capabilities for no-code automation and orchestration, repos for code, artifacts and data, multi-CI/CD pipelines and sandboxes, test data, configuration, continuous testing/QA, service level objectives (SLOs) for business and appdev users and troubleshooting and debugging.
      • DevOps admin services include things that a DevOps team needs to administer and manage the platform. This includes capabilities for platform administration, the ability to import pipelines into the unified platform, integration with enterprise service catalogs, integration with enterprise issues and change request systems, SLOs for DevOps administrators and capabilities to implement governance policies-as-code.

    ● Resources layer – includes resources that the services need to run including application stacks, infrastructure resources, tools, infrastructure code and topology data.

    One of the most important benefits of a unified platform is the ability to unify SLOs for each class of stakeholders. The SLOs need to be chosen to best fit the requirements of each application.

    Business value can be demonstrated by the following SLOs:

    ● Productivity metrics.
    ● ROI data tracking.
    ● Agility of services – Lead time for changes and frequency of releases.
    ● Reliability of services – Change failure rate; time to restore services.
    ● Governance compliance and security.

    Appdev user value can be demonstrated by the following SLOs:

    ● Platform availability.
    ● Service response time.
    ● User experience scores.
    ● Change request response time.

    DaaS team value can be demonstrated by the following SLOs:

    ● Scope of applications – number of apps supported, number of users per month.
    ● Number of requests served per month.

    Without a unified enterprise DevOps platform, all these SLOs would have to be implemented multiple times, once for each type of DevOps platform.

    To realize DevOps-as-a-Service for enterprises with multiple related pipelines, each pipeline needs to be configured to work within a unified model where common tools process each application and microservice continuous delivery task in parallel. Unified control and monitor points are needed – and should be exposed by the unified platform – to realize SLOs in a unified way. Without a unified platform, all of these SLOs will need to be implemented separately, for each platform, and then another system will be needed to aggregate them into a common view.

    The SLOs will indicate where the biggest bottlenecks are for a particular pipeline. In many enterprise pipelines, the biggest bottleneck is testing, troubleshooting test failures and debugging to resolve the failures. It is a myth that continuous testing is just the execution of automated tests. The whole range of “testing things” includes test planning, test resource scheduling, test design, test implementation, test environment management, test execution, test results reporting, test result analysis, troubleshooting, debugging, fix retest and test management, and all can be sources of bottlenecks. These “testing things” are all sources of bottlenecks for every stage of a DevOps pipeline, for all Three Ways of DevOps. Without a Unified enterprise DevOps platform, all these testing things must be addressed separately, in accordance with the different pipeline platforms.

    Testing, troubleshooting and debugging tasks are a major source of bottlenecks for most DevOps value streams. A unified platform improves continuous testing/QA, troubleshooting and debugging as follows:

    ● Continuous testing/QA – Orchestration of test environments, execution of tests and reporting test verdicts is unified.
    ● Troubleshooting – Failures are displayed in a consistent format, including trends correlated to pipeline and application log data, to simplify analysis of test failures.
    ● Debugging – Causes of failure can be isolated with intelligent analysis of troubleshooting results, and the platform can quickly install the fixes and orchestrate the test environment to verify the fix is working.

    These testing, troubleshooting and debugging advantages have a major impact on the ability to accelerate digital transformations.

    What This Means

    Digital transformation of many enterprises and organizations are being seriously impeded by having too many DevOps platforms and tools. A unified enterprise DevOps platform resolves the technical problems of broken, leaky, “Frankenstein” toolchains and processes. Enlightened leaders that understand the importance and the broad impacts of unified enterprise DevOps platform are wise to recognize the sense of urgency surrounding a migration to a unified platform, and take action to stop the proliferation of too many platforms.

    To realize a unified enterprise DevOps platform in an enterprise takes more than simply buying and installing a new platform and telling everyone to start using it in place of what they are used to. With multiple entrenched DevOps platforms in place, there are practical and political considerations that must be overcome. Introducing any changes across a large group of applications in an enterprise can be chaotic, and will often be more costly and take too much time if the changes are not well-coordinated in accordance with a carefully crafted strategy and implementation roadmap that leaders and team members are aligned with.

  • Continuous Testing: The Quest for Quality at Speed

    Continuous Testing: The Quest for Quality at Speed

    The DevOps goal to reduce lead times by accelerating software deliveries presents a serious challenge for assuring the quality of the software that is delivered. As I indicated In my SlideShare post, “The Quest for Quality at Speed,” testing activities span end-to-end software value streams and often account for more than 50% of bottlenecks.

    The answer to these DevOps testing challenges is referred to as continuous testing (CT), which is more than simply automating tests. My blueprint for continuous testing identifies eight pillars of practices, each with strategies to reduce bottlenecks. In this article, I enumerate the strategies for each of the eight pillars of continuous testing and explain how to approach implementation for greater adoption of CT across an organization.

    Continuous Testing Pillars

    Continuous Testing Tenets

    There are five tenets which are guiding principles that enable continuous testing:
    • Shift Left: Conduct each test as early in the pipeline as possible.
    • Fail Early: Arrange the tests so the most likely problems are found during the earliest possible stage in the DevOps pipeline.
    • Fail Often: Run tests frequently and with many different conditions.
    • Test Fast: Arrange tests to run in quick cycles.
    • Be Relevant: Focus on the most important tests and results.

    Leadership and Culture

    Leadership and culture practices have a major impact on influencing behaviors and habits of everyone involved in testing across the DevOps value stream. Bottlenecks to CT include inadequate funding of testing due to lack of quality vision and goals at the executive level, and bottlenecks caused by distrust and lack of cooperation between Dev, QA and Ops. Executive sponsors need to clarify the definition of quality for the organization, and emphasize quality’s importance. Executives can communicate the vision and goals for quality and establish incentives that foster a culture of collaborative behavior.

    Test Strategies and Plans

    Test strategies and plans are needed to guide testing approaches for each product/service pipeline. Bottlenecks to CT include confusion caused by lack of product/service test strategies, and unplanned testing activities and resources are not included sufficiently in planning process. It is important to establish test coverage standards, test case standards, gate criterion and planning process that include testing requirements. CT strategies include making sure test coverage requirements are defined, all types of tests required are identified, allocate tests that are required at input gates, indicate test results are required at output gates and enforce that test tasks are included in planning backlogs and sprints.

    Test Management

    Test management is needed to coordinate testing resources and test schedules. Bottlenecks to CT include poor utilization of test resources due to lack of visibility and schedule bottlenecks due to lack of coordination of test schedules. It is important to have a database of tests and test resources, and test scheduling tools. CT strategies include version management of tests and resources, an inventory of test resources and dynamic test scheduling.

    Test Automation

    Designing, creating, selecting and executing tests in an automated fashion to get to a verdict as fast as possible is the primary strategy to accelerate testing. Bottlenecks to CT include slow test creation, slow test execution and slow time to realize verdicts. It is important to have effective automation strategies for test design, test creation, test selection, and test execution. CT strategies include test design templates, advanced test creation tools (E.g., TDD, keyword, behavior, model-based), dynamic test selection, tests designed for fast verdicts and horizontal and vertical scaling tests.

    Test Tools

    Test tools are designed to automate specific types of tests and integrate with DevOps platforms through APIs for orchestration, execution and reporting. Bottlenecks to CT include test tools that are not designed to scale, limited test configurations and lack of complete APIs, which make integration with DevOps platforms inefficient and error-prone. It is important to have scalable test tools with good APIs. CT strategies include limiting the selection of test tools to ones that are DevOps-ready or DevOps-native and that have well-defined, stable APIs.

    Test Environment Management

    Scheduling and setting up resources into the needed topologies for running different types of tests for developers and testers can be a major bottleneck if not managed efficiently. Bottlenecks to CT include lack of test resource scheduling, which causes conflicts for resources, manual test setups, which are time consuming, causing long wait times for testing and delayed retesting. It is important to have test resources scheduling, test environment orchestration and resource utilization management. CT strategies include test environment inventory tools, test environment sharing tools, test environment orchestration tools and monitoring test environment utilization.

    Test Results Analysis

    Reporting and analyzing test results can be a major bottleneck after test execution is completed. Bottlenecks to CT include poorly written tests which can report confusing results and that take a lot of time to analyze. It is important to have design standards for test results and test results reporting standards. CT strategies include test verdict design in test cases, common test reporting tools and test analysis tools.

    Steps To Implement Continuous Testing

    Start with one or two services and a small set of critical CT practices, then expand to more services and more advanced CT practices.

    A gap assessment is a great way to determine the current state of CT practices, and will help focus the implementation plan. Given a list of practices from each of the pillars in the continuous test blueprint, a cross-functional team of developers, testers and ops people can assign scores to measure importance and current level of practice to generate a gap score. The combined set of gap scores will give guide and direct current priorities for the CT plan.

    For example, beginning CT priorities could be to establish a strategy sponsor, define quality goals, obtain an inventory of test resources and requirements, put in place version management for tests and test standards, tool selection and test refactoring to match the standards.

    An intermediate CT goal could be to complete tool integrations with DevOps platforms, automate test scheduling, orchestrate test environments and optimize test results reporting.

    With maturity, advanced CT goals could be to put in place advanced test creation strategies, accelerated test execution, automated test environment management and optimize test results analysis.

    What This Means

    Testing activities are typically a major source of bottlenecks that slow the quest for quality at speed. Continuous test strategies that accelerate all aspects of testing can have a major impact on speeding up lead time for continuous delivery of quality software. Implementation strategies that start by applying specific continuous test strategies to a targeted small number of applications, and then expand to more advanced strategies applied to more applications is an effective approach to CT adoption across the organization.

  • How lulu Became a Lemon: A Digital Transformation Fail

    How lulu Became a Lemon: A Digital Transformation Fail

    Many people sometimes have a tough time figuring out what exactly digital transformation is and why it’s so important. I want to tell you a real-life story of how a once-respected brand, lululemon, has failed the digital transformation journey and, as a result, may fail in a new retail landscape.

    The story starts innocently enough with my desire to get my wife a lululemon gift card for her birthday. She has gotten her vaccines and is returning to the gym, so I figured it was a great time to get new workout clothes. I purchased a gift card on March 15 to be delivered via email a few weeks later on her birthday. It was a simple digital transaction that for any company that has its digital transformation act together would be a no-brainer. Well, this is where lulu turns off the tracks and turns into a digital lemon.

    On the day of my wife’s birthday, I had planned that she would wake up and see how thoughtful I was in getting her the gift card. She didn’t say anything about the card, though, so I finally asked if she received the gift card. She didn’t know what I was talking about. I had her check her spam folder, nothing. Nope, no gift card.

    Thus began a day that would see me reach out and communicate with the lulu team no fewer than six times. It seems that lulu uses a third party for digital gift cards. Each time I reached out to a lululemon “educator” (that is what they call them, and did I get an education), I was told a different story:

    1.  A simple apology and card being emailed, give it an hour.
    2. Sorry, it must be that email address, do you have another email address? Give it an hour.
    3. Hmm, seems our third-party gift card provider is having problems today.
    4. We located your card, it was sent to: tlaynguyen@gmail.com  could I check that email? (I would love to, but that isn’t an email address I have.)
    5. Well, we have no control over when the third party sends emails.
    6. Digital gift cards are non-refundable, even if you didn’t receive it. Sorry, can’t help you.
    7. You have blocked our emails from your computer.

    At this point I realized that lululemon, while maybe making great workout gear, is analog to the core. A simple digital transaction like this had them tied in knots, wasted hours of my time and so turned off my wife that she is looking elsewhere for her new workout gear.

    This past year has taught us that retailers that can’t hop on the digital transformation train are heading for the precipice. It seems as though lululemon may have already gone over the deep end. The company better hope the malls fill up quickly because in a digital world, lulu is truly a lemon.

  • Languages and DevOps: To The Cloud!

    Languages and DevOps: To The Cloud!

    Cloud and DevOps are almost a match made in heaven. Cloud started truly gaining steam in the age of DevOps and Agile, and has adapted – or grown up in it – well, depending on your viewpoint. If not for the combined issues of data gravity and data (im)mobility, we might have gone even farther than we did into the cloudy world.

    While the breadth of possible cloud apps means a wide variety of languages is in use, Node rules this roost. Every conceivable language is used in cloud development – because, in the end, they’re just servers – but Node is the go-to language of today. The broad use and appeal of JavaScript make it an obvious choice for enabling full-stack development, and the pool of developers is larger than any other language today. Other languages are used, but not in the numbers that Node is. Python is the challenger, and a surprising number of PHP sites still exist, but forward-looking development aims at Node or Python. Considering the distance from the hardware, there is less call for C in cloud than there is in other areas of development, but it isn’t gone, since interfacing to the OS is also often done in C.

    Targeting cloud is not so much about language as it is DevOps tooling, though. All of the major Git companies have cloud-centric extensions (this is the core of GitOps) that raise the level of “fully automated.” Cloud environments, by their nature, provide APIs for infrastructure, and the Git vendors (and many, many others) have hooked into those APIs to make deployment an integrated part of DevOps at a level of detail that other environments (with the exception of containers) require a lot of work to match.

    The negative to cloud-centric development is lock-in, of course. Creating an entire DevOps toolchain and process that supports vendor X will not work directly with vendor Y. Tools can help with this issue, but the real solution is in our next installment – container development.

    The part of development that arguably benefits the most from cloud-centric development is testing. It is easier to spin up a test environment as-needed and spin it back down when testing is done – no need for a spare set of hardware or a stand-alone VM server; just redirect the DevOps ecosystem and spin it up, then account for data (scrubbing, copying, etc), and you’re off. Codify the steps for data, and you have a re-spinnable environment for test-on-demand.

    The part of cloud that is arguably the weakest is security. Let’s face it, far more of your infrastructure is publicly (or semi-publicly, if running in a VPC) exposed than in a data center. There are far more places to lock down, and a sometimes-bewildering number of options to secure. But the flexibility of pay-as-you-grow makes it an appealing option, and there are tools to help manage all those lockdown points. Just make sure that you’re not the weak point in the shared security model.

    You. Rock.

    Cloud makes a lot of things easier, and Node makes development of any step in the stack easier by using the same underlying language on the server that is used in browsers and in many mobile development platforms. Node “feels” more right for web development, at least because one connection, one call … even if it makes long-running processes require a bit more work.

    Whatever you choose, pay attention to how deeply invested in the cloud vendor’s proprietary interfaces you are, because, like all technical decisions, one day you or your replacement will have to get around those interfaces. And keep making the software the org needs.

  • Progress Unveils First Update to Chef Automation Portfolio

    Progress Unveils First Update to Chef Automation Portfolio

    Progress today announced it has enhanced its Chef Enterprise Automation Stack (EAS) and released a new version of Chef Infra Client that promises to make the programming language on which EAS relies more accessible.

    Sudhir Reddy, vice president of engineering for Progress, said the latest versions of both offerings are the first updates to the Chef portfolio since Progress acquired the company last fall.

    Chef EAS combines elements of the Chef portfolio in a way that makes it easier to develop and integrate DevOps workflows that span everything from provisioning infrastructure to deploying applications. The latest update to EAS adds the ability to automatically execute compliance audits as part of any Chef Infra Client run using Chef InSpec tools. At the same time, the latest update also simplifies the implementation of the patch management processes.

    DevOps teams can also view and manage Chef Infra Server details via the Chef Automate framework. Those views, when combined with Chef Infra Client Run Reports, provide insights into the various policies and Chef objects in use on a system.


    Finally, the Chef Workstation module includes the Chef Habitat command line interface (CLI) to further simplify workflows.

    Ultimately, Progress is moving toward creating a common user experience across all elements of the Chef EAS suite of tools, Reddy said. In effect, EAS is emerging as the control plane through which the entire Chef portfolio will be managed, Reddy added.

    Meanwhile, Infra Client 17 promises to make the Chef Language underpinning that suite more accessible. It provides a streamlined developer experience as well as pre-built integrations with various cloud services. In addition, Progress is adding support for various ARM processors used in cloud services, macOS Big Sur, the Apple M1 architecture, Windows 8 and PowerShell Core.

    Progress also enhanced its ability to securely capture metadata from the Amazon Web Services (AWS) public cloud as well as capturing data from Microsoft Azure clouds.

    An early pioneer of a framework for automating IT infrastructure-as-code management, Chef is still widely employed. However, rival open source frameworks have since emerged, which Chef eventually responded to by making its core platform also available as open source software as it changed its business model. Progress today also announced it will recognize external contributors to the platform with recognition awards. Progress reports that, in 2020, there were more than 6,000 contributions from over 350 contributors in the Chef community in support of Chef Infra, Chef InSpec and Chef Habitat.

    It’s still unclear how the battle for infrastructure-as-code framework management supremacy will ultimately play out. While many developers have clearly shown a preference for rival open source tools, Progress is positioning the Chef portfolio as a framework for IT organizations that need to automate processes at scale, including how application are deployed on infrastructure that is provisioned as code.

    Of course, many rival framework providers have similar ambitions. The challenge, of course, will eventually be convincing organizations that tend to employ a wide range of tools across different departments to finally standardize on one framework versus another.

  • Cooperative Gaming: Software Teams’ Secret to Success?

    Cooperative Gaming: Software Teams’ Secret to Success?

    Trust falls and team backpacking trips are out — cooperative gaming is in. Don’t think embarking on group Minecraft missions could help deliver great software products? Think again. A 2019 BYU study found a 20% increase in office productivity after just 45 minutes of gaming.

    Software engineering is a mentally taxing endeavor, and many developers are under mounting pressure to deliver. To destress, many software teams use gaming to unwind during work hours. Gaming, it turns out, has added benefits of improved morale, increased productivity and lowered ego around unofficial hierarchies within the office. Setting aside time to recharge with collaborative gaming could increase a workplace’s health and help software teams work together to deliver products more efficiently.

    I recently spoke with Max de Lavenne, founder and CEO of Buildable, ally to the workplace gamer, to see why he embraces gaming as a regular part of his software developer team’s weekly regimen. Buildable adopted gaming and saw a notable impact; de Lavenne shared how other teams could similarly introduce gaming to improve performance.

    Gaming in the Workplace

    Traditional business environments don’t typically embrace fun activities on the clock — there is a clear separation between work and life. But at Buildable, gaming is integrated into the weekly routine. The Buildable engineers set aside about 45 minutes mid-week for collaborative gaming, typically on Wednesday or Thursday. The Buildable team, made up of about 20 software engineers, favors Rocket League and a modded version of Minecraft.

    According to de Lavenne, gaming is a great way to foster collaboration and intuitive group thinking. “Getting recharged is really important,” he said. He encourages the team to adopt cooperative multiplayer games; no solo play. He also discourages violent and aggressive RPGs, like first-person shooters.

    Workplace gaming isn’t unique, in and of itself — many teams at other companies will often play Haxball during coffee breaks or get into a Call of Duty: Warzone match from time to time. Yet, few leaders have the stomach to sanction gaming as integral to the developer’s workflow and overall team dynamic.

    “Engineers collaborate best when they respect each other and have confidence in each other’s skillsets,” said de Lavenne. “They are individuals, but there’s a lot of groupthink in teams.”

    Benefits of Gaming

    Gaming has proven benefits. In fact, researchers at Kansas State University found that employees who spent one or two minutes during breaks in their day playing games such as Candy Crush on their phones reported being happier than their peers. Similarly, de Lavenne noticed improved mood and other positive outcomes from collaborative gaming at Buildable, including:

    • Better team morale: Having scheduled gaming breaks to look forward to each week keeps attitudes positive. Software engineering can cause fatigue, which managers need to monitor carefully. “People get tired, and there’s only so much the brain can do,” de Lavenne said .
    • More fluid communication: Collaborative gaming focuses a team’s effort on a collective activity free from monetary incentives, bonuses or other hidden agendas. Afterward, de Lavenne said he’s noticed more fluid communication emerge among the team.
    • Increased collaboration: There’s satisfaction in doing something together, and de Lavenne reports that gaming has redefined collaboration for his team in unique ways. Employees take the cooperative, creative attitude fostered from gaming experiences and apply it to more structured, scrum-type environments.
    • Increased productivity: Dedicating one hour for gaming per week equates to about a 2% loss of labor time. However, de Lavenne reported that the productivity gains afterward surpass that lost time and more.
    • Reduction of ego: In work environments, unspoken hierarchies inevitably exist. Interestingly, collaborative gaming can completely reverse those social dynamics. Seeing senior engineers at the bottom of a gaming leaderboard can be humorous and boost confidence for younger engineers.
    • Combating isolation: Throughout the pandemic, remote work has left many of us feeling lonely. For young engineers without families, a big chunk of social interaction may occur in the workplace. Games can help maintain good spirits by bringing teams together virtually.

    Tips for Encouraging a Healthy Gaming Culture

    The overall goal is to promote enjoyable, freeform shared experiences. To enable this, de Lavenne shared a few tips on integrating games into developer team workflows.

    First off, gaming should not be mandatory, as making it a required activity would somewhat defeat the purpose. Instead, gaming should be fun and completely voluntary. If people would rather get in some physical exercise or check Instagram, they should feel just as free to do so.

    Also, as much fun as gaming with the team could be, it’s helpful having the CEO removed from this activity, de Lavenne advised. This is important so that players don’t feel they’re constantly monitored. Yet, leaders may want to have a say in the type of game, as well as when it’s played.

    Furthermore, the game doesn’t need to be an immersive RPG — it could be as simple and accessible as Jackbox. Cooperative board games like Dungeons & Dragons or Gloomhaven could have just the same effect as digital games.

    Outside of encouraging collaborative gaming, leaders can take other actions to boost morale and productivity. One such action is leaving chat channels unmonitored, so that team members can engage with each other on whatever subject they want without oversight. Another way to affect morale is location, noted de Lavenne. As I’ve written previously, hiring interest is expanding outside of traditional tech hubs. Buildable, for instance, chose to headquarter in suburban McMinnville, Oregon, for its walkability, greenery and better work-life balance than in larger cities.

    Gaming Is Where It All Began

    Multiplayer video games, LARPing, and AR/VR environments not only offer an escapist retreat — they represent nostalgia. For many programmers, video games were a doorway to the computer science field in general.

    “I’ve rarely met a software engineer that never gamed,” de Lavenne added. Gamers often seek out the highest-performing machines. And chances are, those kids with limited resources built their rigs from scratch. “This inspires you to learn about hardware, and you start to care about speed, compile and software activities,” said de Lavenne.

    From Atari to Minecraft, gaming has been a shared passion among generations of programmers. Recognizing this affinity can unite a team. Because at the end of the day, “building good software is all about people,” said de Lavenne.

    Of course, software development takes extreme diligence, planning, and structure to deliver bug-free, user-centric products. But under intense pressure to perform and meet new digital innovation demands, software developers could get burned out without a bit of distraction. With gaming, “the team really comes alive,” de Lavenne said.

    So, when things get slow, check out what’s hot on Steam. You may, ironically, just find your next productivity breakthrough.

  • Languages and DevOps: Traditional

    Languages and DevOps: Traditional

    For quite a while now, all the cool kids have been going on and on about “full stack development,” “API first,” and a ton of similar methodologies/practices. And yet, a huge swath of applications are still targeted at a single system to do a specific job. There are fat clients, there are internal servers, there are standalone apps. All are still in development, and the hype machine doesn’t account for them much, though they’re pretty pervasive.

    We still have to maintain them, or write yet another. They’re not going anywhere, because there are some problems for which they are the right solution. So let’s talk about them.

    For clarity, I’m defining traditional as, “Targets a specific OS/hardware platform, and is not primarily web-based.” We’ll exclude mainframe, because the toolsets and integration with DevOps are still different enough to warrant their own blog.

    Online forums are full of people trying to re-target traditional apps (perceived Windows 10 issues caused many to want to move from Windows, and the growing Linux trend of making distros into beta – or even alpha – testers has many wanting to move from Linux), while others are just trying to keep them running as the world changes underneath their feet.

    Easy Integrations

    While these dev tools are the ones that Agile grew up around, that makes them well-integrated into Agile and DevOps. The feeds to/from compilers and various build tools are the same as other spaces we’re looking at (or, more precisely, the others are the same as traditional), so integration of AppDev is generally not an issue. Integration of Ops is harder, but exists. The various DevOps tools with an emphasis on operations (I like to call it devOPS as opposed to DEVops) are available to traditional development, and you can certainly deploy through them, but they’re like the second cousin to “full stack.” Good Enough rules the day, and a little work will be required. Thankfully, tools like Ansible and Puppet smooth the way a bit for deployment, not caring much if you’re deploying a traditional app or a web service, as long as you tell them what to do to successfully deploy.

    You got to move it, move it…

    Portability is increasingly becoming an issue. The rise and fall of “preferred” Linux distros, the costs of being a Microsoft shop … there are a lot of reasons to want to move your app around. The best first step to this (and the only step some will need, IMO) is containerization. Put it in a container, and your underlying hardware can run on whatever OS the corporation settles on, while your apps will run in containers on the OS they need. For most organizations, this will only be a stop-gap; a way to quickly move servers away from one OS and onto another, but the apps will have to move fully to the new OS, eventually. Microsoft has Windows Subsystem for Linux, which can help move from Linux to Windows, and Linux has Wine to help run Windows apps on Linux. Both have come a long way toward interoperability, but neither is perfect – or even easy. It will take a bit of work, but far less than rewriting for the target OS top-down. Tools like Cygwin are also available to get your Linux app running on Windows if you don’t want to use Microsoft’s solution. In the end, you are going to have to modify code, but you are porting an application, so no surprises there.

    Consider the future

    A place I worked in the late 90s/early 2000s had an OS/2 machine that was critical infrastructure. We were not allowed to touch the box. I think if the admins had their way, we wouldn’t have been allowed to look at the physical box. Of course, we virtualized it just as soon as we could in a step similar to my “containerize” advice above. Now is a good time to consider writing your apps with an eye toward portability, so you can remove this type of system from the equation. That RHEL 5, or Windows XP app should really be modernized or eliminated at this point. The tools for cross-platform development are out there, from Mono to Java to Xojo; platform-dependent bits can be isolated for rapid retargeting. It’s up-front work that pays off in spades when the time comes to change the targeted platform.

    You. Rock.

    Whatever your language of choice, in the end, it will likely be your build tool that links it to DevOps. Your biggest concern for DevOps will probably be portability in the near future. Meanwhile, keep rocking it. You’re doing the work that needs doing, even if it isn’t the part of DevOps everyone is currently buzzing about.