devopsy.kinsta.cloud

Author: contributor

  • Testing’s Role in Delivering Digital Transformation

    Testing’s Role in Delivering Digital Transformation

    “We have a quality philosophy now…but it took the rise of the middle child to make it happen.”

    • Ann Lewis, ExxonMobil

    Ann Lewis’ metaphor of testers being the “middle child” between software development and IT operations was present throughout Accelerate 2017, Tricentis’ annual user conference. Nearly every head in the room nodded as Lewis described how testers, like middle children, tend to get less attention than their professional or familial siblings, but she insisted that there’s never been a better opportunity for testers to receive the respect they deserve. Digital transformation, no matter how you define it, requires new commitments to quality, and it’s up to testers to claim be the change agents in the transformations that CIOs are tasked with delivering.

    Tricentis CEO Sandeep Johri and other CxOs who spoke at Accelerate echoed this sentiment. Johri pointed out that speed continues to be a primary focus of maintaining or gaining competitive advantage, and manual testing processes that prevent agility or DevOps success at the enterprise level must be automated going forward.

    What was also made clear is that scaling investments in automation is not the single key to success. The pressure CIOs are under today, especially around digital transformation, requires new investments in a number of different areas, many of which are only just being realized. While CIOs may understand the necessity of quality, it’s up to testers to get the buy-in required for the continuous testing required to deliver it at scale.

    Testers with an oft-ignored middle child mindset may find the concept of being recognized as a “champion” by their CIO or CEO unlikely. But, Todd Pierce’s impassioned session on “What CxOs really think about testing” was designed to inspire testers to change the way testing was viewed too often in the past.

    As a current Rock Health and Dignity Health board member, and former Bill & Melinda Gates Foundation chief digital officer, Pierce shared his peers’ all-too-common beliefs that testing takes too long, costs too much and constrains developer productivity. And, while it may hurt to admit, with manual testing that should be automated and insufficient testing time that results in bugs making it to production, it’s not surprising that CIOs would feel this way.

    Pierce explained that testers who adopt innovative, scriptless “high-precision testing,” are able to shorten their test cycle, reduce defects that make it to production and eliminate costly rework for themselves and developers alike. When high-precision testing is performed continuously throughout the software development life cycle (SDLC), those beliefs that testing takes too long, costs too much and impedes innovation lose all validity. “Everyone in this room should be the drivers of agility in the enterprise,” remarked Pierce.

    Emmet Keeffe at Insight Venture Partners hosted a CIO roundtable later in the day where four CIOs detailed their personal experiences with taking enterprises through digital transformations. Each CIO admitted that in the past they had underestimated the the time being spent on testing, and one somewhat bravely stated that he’d always found testing somewhat “boring.” One might expect a room of testers to storm the stage with charges of heresy, but, given that many testers have suspected that management felt this way for years, I got the feeling that many were simply impressed that someone who felt that way had admitted it.

    But the panel, Pierce, Lewis and countless other speakers at Accelerate also made sure that the attendees of their sessions understood one thing: As more organizations undergo digital transformation—and no matter what stage of digital transformation they’re in—testing has never been in a better position to drive positive changes that will have monumental impacts on the business.

    Whether CIOs have been tasked with delivering greater agility, DevOps shifts, increased security, enhanced user experience, reliability and performance or legacy app modernization, testing may not be the first organization that CIOs turn to for help. But, as Keeffe explained, testing leaders who truly understand the incredible opportunity they have to make those CIOs heroes will make themselves heroes in the process.

    About the Author / Noel Wurst

    Noel Wurst leads influencer relations for Tricentis, and collects stories around cultural shifts and evolving technology’s impact on the software testing industry. By attending global conferences, interviewing passionate software professionals, and working closely with product, engineering and customer success teams, Noel looks for stories that matter and innovative ways to share them with audiences that lead to business growth. You can follow Noel on Twitter and LinkedIn.

  • The Need for Speed Demands New-Era APM

    The Need for Speed Demands New-Era APM

    Applications have become the primary conduit by which customers experience a brand, making the application experience critical to business success. As such, continuous delivery and highly available systems that deliver optimal end user experiences at lightning speed have become top priorities for organizations. Modern IT is all about speed, efficiency and customer experience. Simply put, companies that develop new applications and services the fastest will win.

    That’s where containers and microservices come in. Containerized microservice applications enable organizations of all sizes to approach the ultimate goal of continuous integration and continuous delivery. This accelerates everything within the application life cycle, from initial development to component updates and new functionality.

    To match the development speed that microservices and containers enable, Ops teams need tools that can also operate hyper-fast, providing an immediate understanding of the impact of changes made, real-time analysis of vast amounts of data across applications and technologies, visibility across a range of languages and infrastructures, and knowledge of how the application is performing for end users.

    In fact, we are at a critical inflection point in application performance management (APM), where a generational shift is needed to keep pace with the new hyper-dynamic application architectures consisting of microservices, containers and hybrid cloud environments. To fully understand the impact of this shift, we can examine the genealogy of APM.

    APM Evolution

    The first generation of APM, established at the end of the last century, collected statistics and metrics out of system and application components, providing a first generation of user-defined transaction profiling by using instrumentation technology. APM 1.0 required a lot of effort to implement and maintain, but helped to monitor and analyze the performance of newly developed 3-tier applications.

    The second generation of APM was driven by more distributed, service-oriented architectures (SOA) and the rise of more elastic, virtualized or cloud-based infrastructure. APM 2.0 added application topology discovery and could analyze distributed applications. User experience monitoring of web and mobile apps quickly became part of APM 2.0 as more businesses depended on the service quality of their applications. With a primary focus on application monitoring, not management, APM 2.0 required an even broader set of subject matter experts to configure and interpret monitoring data from dashboards and reports. Similar to APM 1.0, the second generation still required way too much human analysis and setup to fully understand application problems and derive actions.

    The continuous change in today’s modern dynamic applications makes it difficult, if not impossible, for those older tools to keep up. Throw in additional architectural challenges such as hybrid clouds, containers and orchestration (such as Kubernetes and Mesosphere) and the picture gets murkier, all in the name of high-speed delivery. Meanwhile, the microservice architecture and development strategy spawns wide technology diversity—each engineer can choose the programing language, middleware and databases best suited to code their individual microservice. Scale, complexity and constant change is the new reality for today’s dynamic applications.

    Wanted: A New Breed of APM

    In this environment, traditional IT monitoring tools won’t work. These new applications use a vastly larger set of technologies, must scale dynamically and are in a constant state of flux.

    In this effort to engineer for speed and efficiency, complexity and scalability have increased exponentially, requiring continuous monitoring and more visibility than ever before. As organizations embrace agile methodologies for development and containerized microservices for deployment, developers have shifted their thinking in technology choices. Developers now continually select the most optimal technology for their individual goals regardless of DevOps’s ability to configure, deploy and manage the specific technology.

    With several teams across the enterprise constantly publishing new software releases, understanding the impact of these new releases and changes to the overall environment has become a significant challenge. Today there is no single source of truth assisting communication between various organizations, resulting in what seems a wild west–like environment for IT. With so many groups and technologies in play, everybody seems to be speaking a different language, especially when it comes to delivering high-quality applications. In this environment, a new breed of APM  is needed that can provide a “Truth Machine”—complete with deep visibility across all application components in the technology stack.

    This new breed of APM must be automatic and dynamic to deal with the ever-changing world of applications running in containerized microservices. To deliver these critical capabilities, the tool must provide real-time visibility into dynamic, containerized, microservice applications, regardless of language or infrastructure. New thinking is required to meet the challenges of these complex, large and dynamic application environments; a new monitoring and management approach that can handle dynamism by:

    • Delivering real-time analysis of vast amounts of data across all applications and technologies.
    • Conducting continuous discovery and automatically mapping every component, host and container, (hosted on-prem, private cloud, public cloud), to the exact architecture of complete technology stack.
    • Visualizing the application architecture and delivering actionable suggestions that assure the operational health of systems and applications.
    • Providing real-time accuracy to visualize, structure and identify problems immediately.
    • Monitoring the complete set of diverse technologies that run in microservice environments.
    • Operating in a heterogeneous environment and automating deployment across nearly all programming languages.

    Artificial Intelligence for Next-Generation APM

    DevOps and IT organizations are struggling to match the speed of agile development, and this is creating exponentially more challenges around new application architectures. The only constant in these modern application environments is change. Whether change is due to microservice architectures, container orchestration or simply more agile delivery execution to effectively manage microservice applications, DevOps needs up-to-date visibility, a precise picture of the application’s structure and an immediate understanding of its health. To accomplish that, DevOps must be able to apply artificial intelligence to the act of monitoring modern dynamic applications.

    Rather than spending time configuring and reconfiguring monitoring tools, organizations must automate management steps with as little human setup and configuration as possible. To have complete control and visibility of their environments, regardless of language and infrastructure, organizations must use a modern application management solution that is automatic and AI-powered to deal with the ever-changing world of dynamic containerized microservice applications.

    About the Author / Pete Abrams

    Pete Abrams is a co-founder and the chief operating officer at Instana, the leader in application performance management for containerized microservices applications. Formerly vice president of customer centric innovation at AppDynamics, Pete has spent more than two decades in the technology industry focused on introducing new products to market and ensuring customer feedback and desired capabilities are incorporated into future product innovation. Connect with him on LinkedIn.

     

  • Keep Developers Happy and Velocity High

    Keep Developers Happy and Velocity High

    In today’s economy, the companies that prevail are those that can accelerate software delivery and increase developer productivity. Yet, as applications become more complex and codebases grow, companies are forced to implement more comprehensive quality safeguards, which slow the rate at which they can deliver new software. Delivering more incremental builds in shorter intervals can address this conundrum: Customer needs are serviced quicker, overhead for engineers is reduced, and risk is mitigated because changes released into production are smaller.

    That’s why high-velocity engineering teams are applying not only continuous delivery processes but also lessons in experimentation from established leaders including Amazon, Netflix and Facebook. These companies have made experimentation a foundation for their release processes, trying out major feature releases and redesigns within smaller groups before making them broadly available. By using new techniques such as feature flagging, rollouts and traffic splitting, experimentation is no longer just the future for marketing teams; it’s quickly becoming an essential practice for high-performing development teams as well.

    Today, discussions around velocity focus on key practices that developers can use to maximize the effectiveness of continuous delivery, software development and experimentation programs. Here are some key ideas to consider.

    Reducing Your Dependency on a QA Team Actually Makes You Faster (Shifting Left)

    Many organizations eschew the conventional wisdom and deprioritize the heavy lifting of writing comprehensive test suites, opting instead to throw them over the fence to a Quality Assurance team. This approach is almost always less effective and causes organizations to lose momentum instead of conserving it.

    Higher-performing teams actually “shift left” and empower developers to own many parts of the QA process. A solid foundation of tests creates less overhead later on and empowers engineers to release quality code into production faster. Companies can address customer and product needs efficiently, quickly and with more confidence than if they wait for full QA cycles to finish.

    But what if you don’t have the ability (or luxury) to rewrite tens of thousands of tests? The good news is that even without expansive test coverage, you can still take advantage of this approach using code coverage tools. These work by evaluating how much of your codebase gets executed during test runs. If the overall coverage decreases, it means a feature was potentially rushed and is untested and should not be merged into the codebase until the ratio is balanced out. This code-to-test measurement can even “block” code merges unless it meets or exceed the previous threshold.

    Prioritize the Build and Test Frameworks Themselves

    Teams that that prioritize the development of build-and-test frameworks—as opposed to individual tests—are rewarded with faster, more reliable builds. But it’s a struggle to build out reliable processes that can handle tens of thousands of tests in a performant way for each developer.

    Particularly with more complicated test suites, there’s a tendency for these tests to become brittle or prone to occasional failure because of all the moving pieces. Optimizely built a “test gatekeeper” that automatically disables tests that fail in this way, without the need for human intervention, and assigns them to an owner to be fixed. Remember: unreliable tests are sometimes worse than no test at all.

    Achieve Faster Release Cycles Through Experimentation

    Companies that want additional, real-time insights and fine-tuning for their application stack are shifting to feature flagging and feature rollouts to expand their capabilities.

    Feature flags are simple control-flow boolean values that allow developers to use a new, experimental process on their application by gating it behind a virtual on/off switch, and have the option to disable it if a problem is found, all without another code deployment cycle. An example could be a “down for maintenance” notice, or something more advanced like a new product homepage that doesn’t require a code rollout to update the content.

    Feature rollouts take this concept a step closer to real experimentation by placing users into two or more experimentation groups, allowing every new feature to be beta-tested before a full general release. Each new feature becomes an opportunity to run an experiment and collect immediate feedback on your development hypothesis.

    Finally, there’s traffic splitting, a concept similar to feature rollouts that transparently routes a small number of users to a new experimental version to identify any bugs before they reach a broader audience. The primary difference between feature rollouts and traffic splitting is the “all-in” nature of traffic splitting an entire application vs. a single feature.

    Worried that the automated and manual testing of your large product release didn’t catch everything? Send a small percentage of your traffic to the new application as a “canary build” and see what happens. Optimizely experiments with traffic splitting between different production environments (stable versus release candidate, for example) to a small number of users. Facebook, and others have utilized this process for years.

    Shift Left, and Enter a High-Velocity Future

    Thorough testing helps companies catch many issues before they reach customers, but it’s extraordinarily difficult to anticipate every possible usage or scenario for your product. Investing in these release strategies will help companies mitigate the risks of unintended bugs creeping into code or, worse, unwittingly turning their customers into the QA team.

    Our advice? Don’t let feature development take priority over the necessary steps to keep your organization from working lean.  Empower your developers to participate in the process early by shifting left, and watch developer productivity and velocity soar.

    About the Author / Brian Lucas

    Brian Lucas is Senior Software Engineer at Optimizely, responsible for the Build, Test and Release Engineering. Previously, he was CTO at Credible and Hoopla, and lead architect at Sling (Slingbox). He holds four patents. Connect with him on LinkedIn and Twitter.

     

  • Event Management: Let the Noise Wail Without Going Deaf

    Event Management: Let the Noise Wail Without Going Deaf

    In part 1 of the blog series “Putting the Ops in DevOps,” by James Moore, we discussed the awareness and anticipation of operational management and the best practices that can improve the app at every stage of its life cycle. In this blog, I would like to cover one of the event management discipline maturity and share some best practices.

    If you are an operations professional on a team that’s beginning to embrace DevOps methodologies, you probably are facing some significant challenges.

    You know the business is applying pressure to get to continuous integrations/delivery, where release cycles can be measured by value delivered in days or even hours rather than weeks or months. Your Dev teams have embraced agile development, and are producing (what should be) production-ready code in short iterations. Your combined orgs are working toward implementing an integrated delivery toolchain that can build, test and deploy a new release version at the touch of a button. Your Dev teams are leveraging cloud technologies and architectural patterns that enhance their agility, auto-scaling microservice architectures of unprecedented scale and modularity. They are producing, as a matter of best practice, highly instrumented code that provides a dense trail of data relating to the state of the applications deployed on runtimes that constantly report their own state. You may be responsible for physical and virtual compute, network and storage infrastructure that itself is monitored with tools that spew a continuous stream of state information.

    In other words, you find yourself faced with continuous change (both automatic and human driven) and a barrage of potential noise. If you are a seasoned operations professional, bitter experience may have lead you to see change as the enemy—as change tends to zero, then so does operational risk. Now, you may to have to deal with continuous change as an inevitable force for advancing your company’s business goals. How will you separate the signal of service status from the noise of service state? Doing this successfully will be the difference between knowing quickly that there is a problem with an application or service before your users notice and being told about it by your users. It may also be the difference between getting a good night sleep and unnecessarily responding to a page indicating a benign application state.

    Enter event and incident management. The discipline of event management has matured, and has continued to mature, over decades as a mechanism for determining the status of elements of a managed environment from the state. Best practice in event management consists of:

    • Filtering out events that are not likely to be service affecting. A trace message from an app log is not worth your attention, unless you need to view it as a part of a diagnostic procedure. A synthetic user transaction timeout may require immediate investigation.
    • Correlating events that are likely to be related, so you get one notification per true incident rather than, say, 20.
    • Enriching events with context, so if a single service instance fails in a redundant array of five instances, route the event to the operations console but don’t wake anyone up unless the service is affected.
    • Implementing X-in-Y policies. In a large and complex system, a single microservice-to-microservice HTTP timeout may not be a huge problem. But you may want to investigate if you start getting 20 per second.
    • Implementing runbooks for common failures. Collaborate with development on the definition of those runbooks. Tie those runbooks to incidents as they occur so that the first responder has a process they can reinstate a service or prevent an outage in the first place.
    • If possible, automating those runbooks. If the problem is a failed process, then restart it. A disk is full? Free up some space.
    • Leveraging analytics and machine learning to gain insights from the reams of data your tool collects. Can you learn from event history to suggest correlations or areas for improvement to ops efficiency?

    With rapid delivery of new features and capabilities, the Dev team should already be mindful of the operability of the software they produce. Shift awareness of the event management tools and logic to the left. Have development use the same management tools that Ops use in production, but in pre-production. And then learn from what they see in pre-prod to improve. This should be done in combination with systems failure testing. Perform the test and see how the operations tools responds. Ask yourself, “Would I have been able to fix this if it had happened in production? Would I even have known what to do?”

    As is obvious by now, successfully operating an application or service in a modern business requires unprecedented co-operation between development and operations teams. Event management as a discipline and the tools that support that discipline are as critical now as they have ever been, if not more so.

    Listen to my recent podcast where I spoke about the common operational challenges many DevOps teams are facing today and how the traditional IT Operations best practices could be leveraged for use in a DevOps methodology.

    Also, download this white paper, which includes best practices for DevOps transformation and improving event or incident management.

     

    About the Author / Dr. Kristian J. Stewart

    Kristian Stewart is Architect – Hybrid Cloud Event Management and Analytics at IBM. He currently leads architecture for IBM’s Netcool Event Management offering, and is part of the team providing as-a-service capabilities to IBM’s clients with Cloud Event Management. He has worked in Systems and Service Management for 18 years. He lives in England with his wife and two daughters, two cats, and five Raspberry Pis.

  • Building a Services and IoT Platform with DevOps, Part 1

    Building a Services and IoT Platform with DevOps, Part 1

    With the growth of the internet of things (IoT), expected to connect more than 50 billion devices (things) in next three years, building intelligent systems in a more connected world is presenting many challenges. Especially in building software for IoT, beyond hardware and network connectivity challenges there are numerous challenges in development, test, release, managing the infrastructure, monitoring the infrastructure and devices (things) and security.

    This article focuses on key components in IoT platform, how to select right set of DevOps tools for building those components, and creating a DevOps pipeline for the IoT platform.

    Building Your Own Services and IoT Platform with DevOps

    The main components in IoT solution comprises of device registration and identity, data ingestion, command and control, rules and actions for hot data analytics and cold data analytics, data storage, alerts and reporting.

    Device Registration and Identity: The devices/edge need to connect to each other and to a cloud gateway to send/receive the data, for which the devices have to be uniquely identified and registered at the gateway so that only authenticated and authorized devices/users can communicate to the cloud. Every IoT platform vendors has its own way of device registration and identity methods for storing device metadata such as AWS Things Shadow and AWS Gateway Rules Engine, Device twin and Azure IOT hub in Azure, GE Predix, etc.

    Data Ingestion: The devices communicate to the cloud/field gateway, which further ingests the data (device info and telemetry info, such as temperature data from sensors) into the cloud. You can use multiple options of PAAS services from Azure/AWS/GE predix, etc. for this component.

    Command and Control: These components cause the device to take an action and send messages to a device from the cloud, which can be built with PaaS services such as AWS Kinesis, Lambda functions, Azure functions, Azure Logic Apps, etc. or open-source technologies.

    Rules and Actions: These components act on specific device-to-cloud data, using the rules defined for processing hot data (where the data needs to be queried and analyzed within specified time interval) and cold data (where the data can be stored for delayed querying and further analysis). These components can also be built with existing PaaS services or using open-source tools such as Apache Storm, Map Reduce, Hive, Data Lake, etc. for data streaming and analytics.

    Data Storage: You need to store multiple types of data ranging from device metadata, telemetry data from specified set of devices, hot data and cold raw data, which needs to be stored and scaled up based on demand.

    Alerts/Notifications: The analyzed data needs to be visualized to derive more useful business metrics and outcomes, which can be integrated to external systems or other business applications to take preventive and predictive actions. Customized dashboards and data monitoring tools needs to be integrated to the solution to enable monitoring, alerts and notifications.

    With increased complexity in IoT development, the large number of devices to be monitored and large volumes of data to be analyzed and to manage complex infrastructure, DevOps becomes essential to the IoT development life cycle.

    Selecting the Right DevOps Tools for an IoT Platform

    Selecting the right DevOps tools for an IoT platform can help overcome challenges in creating/pushing applications and firmware updates to large scale of devices, as well as monitoring and infrastructure provisioning on-demand.

    With PaaS services used for IoT development, there is less code development but more configurations and rules-based development for developing key IoT components.

    Also, the teams developing device software and application development may be distributed, but the releases need to be streamlined with complete traceability and auditability. You need to manage a scalable infrastructure to support an IoT platform.

    Given below are some of the DevOps Tools used in most of IoT developments to help DevOps teams develop, build and release an IoT platform seamlessly.

    • Version Control: Git, for distributed teams and maintaining repositories for device development, application development and infrastructure automation development. Other tools are Clear case, SVN or TFVC.
    • Continuous Integration: Jenkins, VSTS (Visual Studio Team Services) and Bamboo to create jobs to do build, unit test and code quality analysis for the application, device software and infrastructure scripts, with the support of large collection of tools plugins and usage of different build tools such as gradle, maven, msbuild, etc., as well as supporting multiple technology stacks.
    • Infrastructure Provisioning and Automation: Ansible, ARM (Azure Resource Manager) and AWS cloud formation templates to automate the infrastructure in multiple cloud vendors and use tools such as Terraform or Spinnaker to automate the provisioning of infrastructure across hybrid clouds and maintain a common infrastructure blueprint, which enables the porting/migration of infrastructure workloads across clouds.
    • Infrastructure Testing: When PaaS services are used for developing certain components in an IoT platform, testing those PaaS components is a bit different—PaaS services have less code but more configurations and endpoints from dev/test team perspectives. Testing these components will involve endpoint testing, whether rules are created properly, how data flows between the different components and robust infrastructure verification. These infrastructure testing scripts can be written using Python/groovy or Vagrant to simulate the environment and test, or by using testing frameworks such as Pester for PowerShell scripts. You can extend your test solutions to leverage the REST APIs/CLI commands, exposed by different PaaS services, and these test scripts can test the PaaS services too, with simulated test data and generate the test result reports.
    • Continuous Deployment: VSTS Release manager, Jenkins, Capistrano, CA Nolio, etc. can be used to create the deployment workflows for deploying device firmware updates, application updates separately with control to release on multiple environments, gated checks and streamlined deployment process, with feedback enabled.
    • Test Automation: Selenium, Jmeter, security testing tools such as Veracode, EMMA, CA Lisa, Inspec, Robot framework and performance test tools are used for test automation. Testing in IiT is complex, as it needs an end-to-end testing of multiple devices interconnected. Most of the other device development may not be in your control, except your own device development, so to test an end-to-end functionality you need to have robust test cases and simulated test devices and test environments. This is where containers can help in test automation—a simulated device as a Docker container can run in one test environment and testers can use service virtualization tools such as IBM Rational Test Virtualization Server, CA Service Virtualization, Parasoft Virtualize or HP Service Virtualization to test their end-to end functionality when other dependent components/devices are being developed or not available.
    • Security and Code Quality Analysis and Security Testing is key in IoT development. As more and more devices are interconnected, the risk of data breach/data privacy is high; hence, a shift left of security testing during the development phase as part of the CI pipeline is inevitable. Tools such as Veracode, EMMA and Inspec can be used to run security analysis and identify security vulnerabilities and compliance issues. Unit testing tools such as junit, xunit, nunit or mstest can be used based on the technology stack. Code quality analysis tools such as SonarQube, coverity, cobertura and Fxcop are widely used for code quality analysis in IoT application components development.
    • Containerization and Container Orchestration: You can deploy your application as Docker containers, which have less footprint in the physical/hosted infrastructure and runs only with required application framework, libraries and dependencies. These containers/cluster of containers are managed with container orchestration tools such as Kubernetes, Docker Swarm, AWS Container Services, Azure Container Services, etc. As mentioned above, simulated test devices can run in docker containers, which enables shipping/moving the containers across any cloud and making the test environments ready within minutes.
    • Monitoring: Monitoring in IoT involves monitoring the devices; infrastructure VMs/clusters; telemetry data; the build, test and releases; and billing, among others. Metrics such as cluster usage, CPU usage, network availability, response time, number of devices connected and devices updated/not updated can be configured and viewed in monitoring tools including AWS Cloudwatch, Azure Application Insights & LogAnalytics, Stream Analytics, New Relic, Prometheus (open source), Nagios, Coscale, SysDig, Dynatrace and GE Predix Monitoring. The tools can be selected depending on the target platform on different clouds or on-premises, technology stack and other factors.
    • Cloud and Cloud Management: Cloud providers including AWS, GE Predix, Azure and GCP provide out-of-the-box features and support for IoT platforms with starter kits, IoT device SDKs, protocols, gateways customization and more. Cloud management tools such as Right Scale cloud management platform, OneOps, etc., can be used for managing the infrastructure workloads; bringing in central governance, access control and security; monitoring billing and optimization; managing security and standards; and streamlining and orchestration of IoT platform development.

    Depending on the technology stack, the DevOps requirements and business needs, the right tool needs to be selected to streamline IoT platform development with feedback flowing between the different stream of teams involved in IoT development, and to bring in standardization in security, governance and tools, metrics to measure and to better manage IoT platform.

    Release, Rollout Firmware/Application Updates

    Rollout of firmware updates for an IoT platform is more challenging compared to application releases; with device updates, we have to deal with network latency, issues with connectivity of devices, security issues, different sets of devices, different deployment channels, deployments done over the air and different sets of protocols used by different manufacturers.

    There is additional complexity involved when there are new additions to devices or when supporting legacy devices, and this needs incorporating integrated end-to-end testing as part of deployment pipeline, to test the devices and applications from a user-centric approach.

    In IoT development, there are multiple development and deployment pipelines for firmware updates, server-side code, mobile apps and multiple devices delivering to multiple endpoints. Hence, the release cadence of IoT components varies, as application code updates can be released monthly, whereas firmware updates to devices can happen every three to six months—hence, a well-defined process must be followed for regularly integrating the code.

    Continuous testing involves infrastructure testing, performance testing, end user testing and functional testing, all of which must be integrated with the release pipeline, where deployments can happen to simulated test environments, provisioned as environments as code (environment with all runtime dependencies needed to run the application like Docker containers), rather than simply infrastructure as code.

    DevOps teams need to keep track of releases and monitor all these deployments, ensure environments are stable, trace back the releases with the device firmware updates and track the defects. The DevOps teams should pull the release packages of different components from multiple pipelines and prepare main and customized builds to integrate and deploy the components together.

    The following are the main steps in CI/CD pipeline for IoT platform development.

    1. Code checked into version control: Check in the code for application development, device/firmware development, PaaS services configurations, infra automation scripts and templates to the version control, pointing to the defined repositories.
    2. Continuous Integration: Build jobs created in a CI server such as Jenkins/VSTS/Bamboo executes build compilation, unit testing, code quality analysis, automated reviews, environment configuration preparation, running security testing analysis, creating deployment packages or baked-in container images, for deployment.
    3. Triggers Deployment: The build agents generate the packages for firmware updates application updates and notifies release agent to trigger deployment.
    4. The deployment workflow can call a service/worker job, which in turn can connect to the cloud gateway services, to notify firmware updates.
    5. The cloud gateway picks up the firmware updates and notifies devices for devices updates. The devices will the pull the updates and installs the updates and further send telemetry data.
    6. Multiple testing environments in cloud can be integrated to the release pipeline to simulate the firmware updates.
    7. In case of application rollouts, there will be canary releases to gradually roll out to different environments in different regions after functional testing/regression testing.
    8. Testing: Security testing, firmware updates targeting to specific devices, performance of the device during the updates, device identity and memory usage, etc., needs to be addressed during deployments in IoT. Once the application is deployed to the test environment, functional/regression testing occurs and generates the test reports.
    9. Monitoring metrics such as memory usage, performance of the device, network traffic, latency, devices connected, reliability, etc., needs to be monitored, as these are of prime importance to decide on how to implement remote updates and monitoring without affecting the performance of the devices and these metrics has to be notified to the Dev/QA/DevOps teams as part of release pipeline execution.
    10. On-demand environment provisioning scripts/templates must be integrated with deployment pipeline to provision the test environments, add more devices as per the scaling metrics.

    Establishing a CI CD pipeline well ahead at the start of development and following mature DevOps practices such as security testing, infrastructure testing, containerization, monitoring, cloud and DevOps governance and feedback-driven development are the keys to help address the challenges in building your own services and IoT solutions.

    About the Author / Lavanya Subbarayalu

    Lavanya Subbarayalu is Senior Architect working with Technology Office in HCL Technologies. She has expertise in IOT, Azure, DevOps consulting & Microsoft technologies. She is associated with DevOps COE, working on design and Development of DevOps solutions and consulting tools. Connect with her on LinkedIn.

  • The Future of Continuous Testing: Fail Faster

    The Future of Continuous Testing: Fail Faster

    In the early days of Agile and Lean development practices, a very noticeable gap appeared between dev teams and ops and QA. Development was accelerating at a lightning-fast pace, but testing processes could not keep up, slowing down the entire life cycle and preventing organizations from achieving continuous delivery. Traditional testing methods, which rely heavily on manual processes and frequently updated GUI tests, proved too cumbersome and inadequate for our new lifecycle timelines.

    The era of testing as an afterthought is dying out, and continuous testing has emerged as the new standard. The idea behind continuous testing is pretty straightforward: Implement a systematic approach towards improvement that incorporates not only automation, but also all of the other tools and processes that will help reduce risk before software progresses to release and production. Organizations embracing this are rethinking testing as a continuous process that starts from the sandbox, rather than a precursor to release.

    Continuous Testing: Achieving ‘Fail Faster’

    At xMatters, we strive to a testing strategy that supports our goal of committing to production multiple times per day—one that we’ve been diligently moving closer to, having progressed from releases every quarter to every month to every week to now, twice per week. Our approach is to conduct unit tests at the team level, code quality, sonar and static analysis, and then deploy to production at limited percentages that we bump up in stages. Our overarching mantra is “fail fast,” because the faster and sooner you fail, the faster and sooner you can fix what’s broken, and the less resources and time you will spend troubleshooting and resolving issues. This, of course, translates to real cost savings.

    Correlating failure to actual events is the biggest hindrance to the continuous testing dream, as pinpointing root causes is an incredibly time-consuming process. This is where automation will take us to the next level, if implemented in a smart way. The future of continuous testing will be adaptive so as to determine the causes of failures faster. We need to automate as much as possible the process of predicting where code quality issues are and identifying the precise data correlations that will enable us to fix issues rapidly.

    Part of this adaptive testing approach will include automating intelligent regression testing. We don’t need to run a black box test in every instance, for example, which can take hours every time. We’ll save incredible amounts of time, money and developer resources if we design tests that can detect new changes and only test those. Smarter regression testing will result in precious time saved that ultimately impacts—and can even make or break—the speed of innovation.

    Another critical consideration to smarter testing is making sure to account for a comprehensive understanding of risk. We need to incorporate real business considerations into our understanding of risk and bake those into our tests as well. We should hold our software to not only standards of code quality and integration success, but also to meet real business expectations for areas such as security, compliance and reliability. Testing from the bottom up must be done in tandem with top-down analysis to account for the full spectrum of potential risks that have the potential to impact the bottom line. Machine learning systems can also be extended to learn new business applications as new functionalities are added and assist in creating test cases that serve to enhance the quality of the application.

    Developing these intelligent automation tools remains our biggest barrier to achieving the continuous testing dream, and while many organizations are gunning for it, most are still mired by manual processes. A global study conducted earlier this year by CA Technologies, for instance, demonstrated that while 75 percent of organizations identify continuous testing as important, just 20 percent report a high quality of test automation coverage.

    It’s worth noting that not every aspect of testing can be successfully automated. The vast majority of functional tests will almost certainly not be manual processes—but what about user experience testing? We have yet to conceive of an approach to this that can truly be automated. Because nothing beats a real human’s assessment of experiential quality, this will be one of the areas of testing that will likely remain manual. Thus, while our goal is to automate 100 percent, realistically this won’t be the case.

    The end goal here is not only to achieve the dream of continuous delivery, but also to eliminate distractions as much as possible. The more repeatable processes you can automate, the fewer distractions your team will encounter, and the more they can focus on creativity, strategy and building new features and capabilities instead of fixing old ones. This is what will ultimately determine your ability to innovate, as you’ll no longer have to sacrifice either time, scope or quality.

    About the Author / Deepa Guna

    Deepa Guna is Senior QA Architect for xMatters. She has been working in technology since 2003. Connect with her on LinkedIn.

  • The 7C Holistic Approach to DevOps

    The 7C Holistic Approach to DevOps

    The DevOps methodology was first adopted in 2007 with a bigger purpose of improving organizational efficiency by bridging the gap between software release and IT operations teams. This started as the core practice of continuous integration (CI) and continuous delivery (CD) with a prime focus on better quality of releases and more frequent releases. Now, DevOps is becoming the norm of the industry and is being embraced by multiple organizations as a digital transformation with a shared goal, involving all stakeholders, to serve the customers better with higher quality product release and more frequent releases.

    DevOps has evolved beyond release and deploy and now involves product owners, project managers and all aspects of software development life cycle. This is the key reason it has evolved as a holistic approach starting from the service team interacting with the customers to product release and production monitoring. In this blog, we will explain how the 7C approach of DevOps is a natural evolution of DevOps and how it helps different organizations to perform better.

    DevOps now spans across the entire organization involving customer service, marketing, product owners, project managers, program managers, and different levels of developers, QA, release or build teams and infrastructure team. As the main focus of DevOps is to serve the customer better and faster, it is important that all the stakeholders have better end-to-end visibility of operations, planning, integration, testing, monitoring, delivery and feedback. This is the 7C approach of DevOps, as illustrated in Fig 1. This requires an efficient integration of processes and tools to automate the seamless information exchange and execution. This approach also empowers all the stakeholders to better contribute in product’s success and efficient execution.

    Fig. 1: 7Cs of DevOps

    The 7C holistic approach to DevOps also requires efficient collaboration between all stakeholders, empowering them to contribute more. The 7C approach removes all silos of information and execution.

    Startups were fast to adopt DevOps as many of them were setting up things from scratch. But the latest industry trends show large enterprises embracing DevOps primarily with the focus on efficiency and better, more frequent releases.

    The DevOps domain includes lots of tools and many of them are open source. This has provided a toolset for engineers to tinker with or experiment. This tends to have problem of its own, as too many toolsets in the flow create silos of information and execution, which creates confusion and inefficiency. This is becoming a problem and that’s why solutions are required which provide better integration and execution across multiple tools. The right solution for this will be a platform with deep integration capabilities.

    A deep integration platform for DevOps will enable seamless integration and execution across different tools, even home-grown proprietary tools, to provide a holistic solution starting from customer initiation to production deployment and feedback. This approach is significantly important for large enterprises, as this enables them to have the flexibility of preserving their existing investments on tools and processes while also allowing them to selectively introduce new tools. Rather than trying everything new, large enterprises can manage changes while primarily focusing on efficiency, automation, collaboration, better quality of releases and more frequent releases.

    This 7C approach gives same visibility to all stakeholders and everyone knows what feature is being released when. This approach also helps to move information/execution in both left and right direction with powerful traceability features, analytics and dashboards. Deep integration of information and execution also reduces dependency on other teams and promotes self-service.

    Fig. 2 illustrates a sample DevOps flow where service team uses ServiceNow; software, product and project managers are using Jira for Agile management, developers are using Eclipse as their integrated development environment, GitHub is used for source code repository, QA team uses a test management tool for test and defect management, and finally the release team uses Chef and Jenkins to orchestrate build, release and deployment. In real-world examples, teams add more tools for code analysis, security and production monitoring. All such complex flows are supported within DevOps 7C holistic approach by utilizing the deep integration platform. The 7C approach is more productive as it allows faster detection, analysis and resolution of problems.

    Fig 2. Sample DevOps Flow with Deep Integration

    The 7C approach works well with both traditional waterfall software management as well as Agile software management—mainly because of better end-to-end visibility, efficiency and collaboration.

    To conclude, 7C approach of DevOps is the natural evolution of DevOps, empowering the team to achieve better efficiency and execute better. This empowers all the stakeholders in the organization by providing better end-to-end visibility and powerful means of collaboration. Involving all stakeholders also creates a better environment of trust and collaboration. With this 7C approach, DevOps is not limited to build and release but it has spanned across the entire spectrum of software development lifecycle. This is especially highly beneficial for large enterprises as it enables them to preserve their prior investments and enhance their existing flows. This gives them leverage to selectively try new tools and technology while preserving what is working well. This is a great approach to expedite the enterprise level digital transformation.

    About the Author / Devendra Joshi

    Devendra Joshi is Sr. Director, Customer Solutions and Products at Kovair. He is a software professional with over 20 years of experience in enterprise software, social media, healthcare, SaaS companies, and EDA. He has worked with startups and large enterprises in engineering management, and product and solutions management roles.

  • SMEs Can Benefit from DevOps, Too

    SMEs Can Benefit from DevOps, Too

    The DevOps approach has several clear-cut benefits with the value centered in several key business areas: improved market agility, increased quality and revised culture of cooperation and effectiveness in staff operations. By promoting and implementing a DevOps culture in your SME’s business, Dev and Ops teams align much closer together and can better share in-between knowledge. This lets you create products more fluidly and increase deployment rates. The time for evaluating the business idea, development, testing and moving the finished product into the operations department is reduced dramatically. Quality margins improve, and this feeds back into the ability of your teams to continually fine-tune the products you offer.

    Businesses see a point in adopting DevOps only if it is directly contributing to the company’s increased effectiveness, productivity and, of course, profitability. The only way that it is possible for small and medium enterprises (SMEs) is when their IT and DevOps teams are working in sync and streamlining the provision of tech solutions that solve the business needs. And only a successful and timely DevOps adoption inside your organization will help your company beat competitors and stay afloat as the development across the industry speeds up.

    In the last years, we’ve seen a tremendous application delivery shift in businesses. We’ve moved away from the old ways of having large-scale system implementations based on one single project and its continued maintenance. SMEs now perceive application delivery as a continuous evolution. DevOps plays a leading role, as this software engineering trend ensures that such a major shift is possible. It brings all of the different parts of the enterprise together: business, development and operations teams collaborate on a different, more in-depth level with a goal of automating processes. This helps eliminate the usual constraints and takes the time-to-market issues off the table.

    In the latest ECS-Digital research on adopting DevOps in businesses, 57 percent of SMEs have partially implemented DevOps practices in their daily routine. The key benefit of DevOps for them is improved inter-departmental collaboration. But for the 43 percent that haven’t yet adopted DevOps, the collaboration issues are among the last priorities on the list.

    Step-by-Step Guide to a High-Level DevOps Life Cycle

    According to IBM Cloud Director, the end goal of a relevant DevOps practice is to cut the development time and improve enterprise’s efficiency. A growing business has to start from the “business needs” of the company and go down the road to make it come to life. The goal of adopting DevOps in your company is to push your business needs into production at a quicker pace more reliably. The goal in any DevOps practice is to cut the time and increase the efficiency of processes. You need to move from the “business need” to the “it’s live!” stage.

    Business Needs are a DevOps Priority

    But does your business have to get involved? Of course, your SME leadership focuses on the “business needs” of your company. But when we look at organizations that have implemented proper DevOps practices, the internal structure of the company shifts to focus on delivering great business results across the whole life cycle of your product/services offering.

    DevOps also ensures that employees have the ability to correct their work direction during the whole lifetime of a project. Things and goals tend to change, competitors may roll out some features that may make some of your project’s parts trivial, and all of this makes the need to adjust your project accordingly. This is easily done when your business need is close to both the Development and Operations teams that are able to steer and drive this alignment of goals.

    Have a look at the infographic below that clearly shows how a DevOps delivery cycle looks like:

    The Clash Inside Organizations

    Measuring the impact of DevOps on the things that matter to business is vital. Most SMEs are experimenting with DevOps, and there are some within the company who want to prevent DevOps adoption. The culture within enterprises is shifting too drastically and dramatically, although the effects are mostly positive.

    Such a shift is impacting all operations, with software development impacted the most. SMEs have it easier to change and adapt compared to larger businesses, and this lets them move the workplace culture a step ahead of their larger counterparts.

    If we look at business, operations and developer teams from a historical point of view, it was never a goal to establish inter-connection between them. The teams started cooperating only when it was absolutely necessary to get the other team’s input. SMEs are still faced with such a separation, as most responsibilities are set in stone for each team and the individual employee. All business units are used to working in their closed-in niche, and there is a lack of will to leave the comfort zone and act outside the predefined roles. Such actions in the past could have been viewed negatively.

    While most businesses that want to implement DevOps have employees who want the business to prosper, it can be difficult to force people to act outside their duties. However, to reach the end goals of any business—namely, ensuring fast and quality product delivery—and solve issues on the go, people must interconnect their duties and reach outside their business unit. With proper DevOps application, such collaboration can be smooth and result in the improvement of cross-functions.

    DevOps Represents Change in Internal Business Structures

    Many SMEs are faced with internal structural issues that limit the company’s prospects: Engineers are not willing to be on-call, the Ops team has a negative attitude toward those who write code and executives are slow to embrace automation. Organizations must overcome the culture war to be able to approach the agility and productivity gained by a DevOps model. The faster they can get there, the faster these organizations can take the competitive edge away from traditional enterprises.

    DevOps culture looks very different from organization to organization. However, regardless of the business implementing a DevOps approach, the potential benefits are the same. From faster deployments to increased efficiency and cultural changes toward experimentation and away from rigid planning, a DevOps culture has the ability to transform an organization completely. Wholeheartedly adopting a DevOps culture has real implications on how well a business is positioned to take on the market. With many industries facing uncertainty and a host of upstarts disrupting the status quo, shifting to a DevOps culture can be an important weapon in repositioning your business.

    Empathy is the Core of DevOps Adoption

    One of the foundational ideas behind DevOps is mutual empathy. Having empathy from operations toward developers and vice versa, as well as empathy in other business-related and non-technical company units, remains vital. It’s not about patting your colleague on a shoulder. We are talking about the moral mandate of a developer who just creates their part of the software and doesn’t care about how it can be deployed in production. The goal is to push employees to think outside of their small task, and to take into account the outcome of their work: Is the developed app secure? How hard is it to deploy it? Is it easy to keep it running? etc. If those questions are not accounted for, the developer’s colleagues on the Ops side will pay the price if something goes wrong past the release.

    By ensuring that the SME has empathy for each other at its core, businesses will deliver better products/services. And doing just that is what DevOps aims to bring to companies. Until this idea is ingrained, DevOps will face a cultural war of sorts.

    Your Most Valuable DevOps Resource

    By considering to adopt the DevOps approach to running your SME, you can better rethink the ways your organization operates—how it deploys its teams across the enterprise and how often it addresses structural problems that appear over time. Now you can lay out a new foundation for your SME based on the culture of cooperation and experimentation. With proper guidance in the application of DevOps culture, you can address the waste of time, human resources and money in your business.

    Becoming a DevOps organization is no easy task and taking up on such a challenge requires faith that changing the existing processes will bring its benefits. Reworking and optimizing organizations is the new norm, regardless of their size, industry, and goals. The faith in DevOps culture is driven by the firm and quantifiable statistics, but the intangible benefits to corporate culture are just as compelling. Many businesses are looking at implementing DevOps, and even more, SMEs are in the active process of its application. When your company is delivering quality ahead of time, there is a strong chance your competitors will be left behind as you are getting the bigger piece of the pie.

    Key Takeaways

    • To stay current, businesses need to adapt to an ever-changing environment, respond to competitors and stay on top of the innovation cycle, increasing customer value via responsiveness to change.
    • Increased service quality depends on both the availability of the service and being able to restore the service to an error-free state. Thanks to fast feedback loops and high release velocity, service breakdowns can be removed much more quickly than in the past.
    • DevOps are built on the sound principle of waste reduction such wait times and process overhead due to continuous improvement. This helps to improve agility and has a positive impact on costs.
    • Fostering continuous learning and improving culture through efficient principles is what DevOps is all about. In a fast-paced, unpredictable environment, non-stop improvement is paramount for sustaining your competitive advantage. It is also important for attracting exceptional talent and keeping employee satisfaction at a high level.

    About the Author / Stephan Pushkarev

    Stepan Pushkarev is the Head of DevOps Practice at Squadex.com and CTO at Hydrosphere.io. Co-founded and managed engineering teams for eCommerce, IoT and Ad-tech companies. He is responsible for the full products stack: math models, infrastructure & operations, enterprise applications as well as recruiting, establishing engineering culture and delivery processes. Stepan combines strong technical and managerial skills, and entrepreneurial spirit. Connect with him on LinkedIn.

  • Continuous Testing: Crushing the Testing Bottleneck

    Continuous Testing: Crushing the Testing Bottleneck

    The practice of continuous delivery has been with us for many years, but it has started to see real adoption in the last couple of years. Continuous delivery is about automating and simplifying the process of delivering new features to the market at any given time, accelerating delivery and helping bring Dev and Ops together to make them jointly accountable for the release process.

    As continuous delivery is helping to break down barriers, culture and bottlenecks in the delivery pipeline, a new practice has evolved to help teams solve this most daunting challenge of the delivery pipeline: Quality.

    The practice is called continuous testing, which can be defined as:

    • Testing your change in an efficient way (so your tests are completing faster),
    • In an optimized way (so that you find issues as early as possible in the cycle),
    • Based on risk and priority (so you test what is important to your users and the business).

    Some may think of this as equivalent to test automation, but that premise would be wrong.

    Automated testing tools can execute tests repeatedly, provide reports on testing outcomes and show comparisons with previous test runs. This is a critical foundation for a continuous testing practice, but just a starting point.

    In a continuous testing practice, testing is not done just in the “testing” phase and not only by test teams. Testing is performed at the level of the build (for example, Unit Test and Security Test) and does not stop after the change has been deployed into the production, where testing can ensure great customer experiences. Here are some other key aspects of continuous testing.

    Continuous Testing Automates Test Environments

    Let me clarify what I mean by environment. Basically, you want to replicate the real-world situation much as possible for testing. A test environment contains the test cases to be run, the right test data and services available, and the right application version deployed in the right environment. The juxtaposition of these elements of an environment must come together automatically to ensure a continuous testing practice.

    Continuous Testing Amps Up Throughput

    A key premise of continuous testing is that you are continually maximizing efficiency. This is a must to keep up with the velocity and frequency of today’s modern software factory.

    You are doing testing earlier in the testing cycle (shift left) where problems are easier to fix and less expensive. You are doing more testing in parallel. You are automating the promotion and feedback during testing stages. And you are ideally able to prioritize the testing order based on the risk profile of the code so that you are testing higher risk areas first. All of this enables you to find issues quickly and roll forward if possible or roll back seamlessly.

    Continuous Testing Applies Intelligence

    Finally, continuous testing means you have the knowledge at your fingertips about the risk profile of a release so you can make more informed decisions. You should be able to see what was changed in the code and the implications. For example, was there a small change in the UI or a complex backend change that affected integrations and could break the APIs for the interface?

    You also need a detailed view of the quality level of the release, including which tests have failed, which tests have not been executed, whether you have tested the new or the risky areas. All this intelligence will help you to understand the quality profile of a release and make faster decisions about whether additional tests are needed or if a release is ready to go to production.

    Steps Toward Continuous Testing

    There is no perfectly prescribed way to put a continuous testing practice in place for your organization. For example, you may find that your biggest bottleneck of the moment is managing test data, so you should tackle that first. But here are five steps critical in your adoption of continuous testing:

    Step 1: Implement best-of-breed test automation tools—Functional, GUI, Performance, Security, etc.

    Step 2: Automate test environments with the right test data and virtual services to ensure real-world, always available testing.

    Step 3: Automate test requirements (you’ll find manual methods can no longer keep up).

    Step 4: Automate the automation—Connect your testing toolchain and stages. Automate promotion from one stage to the next. Automate feedback loops end to end.

    Step 5: Get smarter—Automatically align the right tests to the right app version. Automate testing sequence by risk profile of the code. Build release health profiles.

    About the Author / Uri Scheiner

    Uri Scheiner is a senior director of product management in the Continuous Delivery Business Unit of CA Technologies. He was previously a key architect and responsible for the product road map and vision at a leading release automation company Nolio, which was acquired by CA in 2012. Uri started his career as a software engineer at Mercury Interactive (acquired by HP in 2006) where he was responsible for developing several products for the DevOps and APM markets.

  • Maturity Models Help Measure the DevOps Journey

    Maturity Models Help Measure the DevOps Journey

    DevOps professionals often encounter clients who struggle with sorting through the “what”s and “how”s of where they are in their journey, and how they can either get started with—or improve upon—their current practices.

    As a project progresses, we often find that diving into a technical solution up front without taking a holistic view of where our clients are in the aforementioned journey will result in undesirable outcomes. That’s because no two projects or clients are alike, and there are a surprising number of dependencies within DevOps practices that must be satisfied before overall success is achieved.

    For example, a client who wants to jump right into a microservices-based architecture will have trouble without first implementing CI/CD practices and utilizing infrastructure as code. That’s where a measurement tool can come in handy, especially one that can help to identify a logical progression in these cases.

    Having seen more than a few clients facing this challenge, we set out to create a Maturity Model to provide an easy way to determine reasonable next steps in both helping clients improve practices and quickly assessing where they currently stand.

    To begin, we took a comprehensive look at the many ways we’ve helped clients with their DevOps practices, searching for common patterns that would help guide and characterize how an organization performed. Additionally, we examined our internal practices and procedures, using numerous projects we have done to self-evaluate the effectiveness of our teams and approaches. From there, we gleaned what we considered to be five distinct realms that constitute a comprehensive view of DevOps maturity and developed appropriate questions within the realms to evaluate performance.

    So who can benefit from engaging in a maturity evaluation?

    Anyone, really, at every level of an organization. To see where you are in your DevOps evolution, log on to answer a few questions and they receive easy-to-understand recommendations. We can discuss results in-depth with anyone who wants a more thorough understanding of where they are in their practice or more personalized advice. Or, just enjoy the exercise. Either way, the questions will help expose certain ways in which your practices can incrementally improve.

    About the Author / Matt Brewster

    Matt Brewster is DevOps Team Lead for Seattle-based Base2 Solutions. His proficiencies include deployment/build/test automation and infrastructure, systems integration and analysis, performance automation and analysis, infrastructure virtualization and containerization, aerospace information technology, software quality assurance and rapid prototyping. Matt’s DevOps/Infrastructure knowledge includes VMWare, AWS, Virtualbox, Docker, Vagrant, Ansible, Chef, Linux, Nginx and Jenkins.