devopsy.kinsta.cloud

Category: DevOps Practice

  • The CMDB Is Dead – Long Live the CMDB

    The CMDB Is Dead – Long Live the CMDB

     Many enterprise IT organizations have a configuration management database, or CMDB, in place. These systems are important for overall configuration management, governance and DevOps coordination. Yet, these legacy data repository systems can fall short in our dynamic IT environment. They require manual updating and can’t always support real-time decision-making. Furthermore, we have new requirements since the onset of widespread WFH workplaces. In this article, we talk about the past and current role of the CMDB, how it is evolving and how integrating it with other IT operations systems can modernize infrastructure monitoring and configuration management.

    History Lesson

    Remember the enormous fuss over Y2K? IT execs were concerned that the new century turnover would break their systems, which led to a secondary concern that they weren’t sure what was even on the network. The 2001 release of ITIL v2 best practice led to the notion of a CMDB, which would enable the tracking of every technology asset and its configurations and show the relationship between those assets. But getting all that data in one place proved to be a problem—and one that has persisted to this day.

    Is the CMDB Still Relevant?

    CMDBs have been around for a long time, and they still play a valuable role in the enterprise IT organization. In fact, a CMDB has never been more essential than now. The year 2020 has been chaotic for most IT organizations as employees quickly transitioned to working from home and are using a wide array of personal devices and networks, all of which need to be monitored, maintained and secured. As shadow IT persists and organizations move increasingly toward rapid deployment and release models that incur frequent, ongoing change, governance and control become critical.

    Contrary to what some may think, you can’t safely go faster without a level of control. This is a balance that DevOps practitioners must strike, and it entails a deep understanding of all elements within the IT environment and how they relate to one another. If the enterprise doesn’t have a CMDB (or something akin to it) to provide visibility into infrastructure components, configurations and relationships, the IT operations team is working at a disadvantage. Such lack of knowledge slows down developers—and nobody wants that. Even worse, without always updated, reliable infrastructure data, DevOps teams risk a catastrophic result with each software release.

    Modern CMDB Requirements

    Today’s CMDB is critical for incident management and must be supported by a mature, well-understood change enablement practice in multi-cloud and hybrid environments, yet legacy platforms aren’t designed to monitor systems residing outside of the corporate firewall. More importantly, modern applications are built on containers and ephemeral microservices that appear and disappear frequently. It is nearly impossible for IT staff to manually update the CMDB under such constraints, and the truth is, people weren’t typically doing this regular updating years ago. Today, the only viable way to maintain state and configuration data to support accurate, real-time decision-making is through automation.

    Most CMDBs on the market today lack automation; they still look and act much like a data repository. The appetite for heavy manual work is razor-thin, and IT organizations simply don’t have the people power to do it regardless. Simultaneously, the role of the IT operations professional has evolved dramatically—it is no longer as focused on installing, patching and rebooting systems. Rather, these individuals are now expected to manage applications and platforms in an end-to-end fashion, serving the business with their unique requirements spanning distinct business units and use cases. Therefore, the more we can use technology to reduce the load of traditional incident management tasks and enable operations to be a true partner with development, the better we can serve the business and end customers.

    So, What Now?

    Integrating modern and legacy tools is one practical solution for resolving these issues. Newer IT operations management (ITOM) technologies that incorporate artificial intelligence and machine learning (AIOps) to automate the discovery of assets and topology mapping can bring the CMDB into the modern age. An open ITOM framework for integrations can keep the CMDB always up-to-date, without manual intervention. This enables IT operations staff to focus on optimizing and improving infrastructure for reliability and performance, versus digging into one dashboard after another to make sense of the status quo.

    Understand the requirements and scope of your CMDB. Tracking the configurations of components that are not business-critical only expands on the effort and resources needed to successfully maintain the CMDB, creating waste in the process. Keep it simple and iteratively improve the CMDB over time. Most importantly, get started now. Even seemingly small, simple improvements can have a positive cascading effect down the road.

    IT environments will only get more complex—even though the end goal is simplicity, cost savings and efficiency. To keep pace, enterprise IT organizations will need to rethink the traditional CMDB. By integrating the CMDB with modern IT operations monitoring, management and automation systems, IT operations teams can finally have a reliable repository of data on all of the company’s on-premises assets and multi-cloud resources, while also gaining the ability to automate manual tasks that impair productivity and eventually remediate where appropriate with the right approach to AI and machine learning.

     

    This article was co-authored by Curt Thorin, a solutions consultant at OpsRamp. He has 20+ years’ experience in enterprise IT software and services. Curt can be reached online at LinkedIn.

  • What Is Resilience Engineering?

    What Is Resilience Engineering?

    Admitting things will go wrong isn’t easy for anyone or any team. But modern engineering practices have moved beyond this fear, giving birth to a new practice in DevOps and site reliability engineering (SRE) known as resilience engineering.

    Resilience engineering today isn’t thought of as a function. However, just as DevOps was a description of culture before it was a role and site reliability was an extension of operations before it was a focus, I wouldn’t be surprised if resilience engineering became a function in the new future. The first question most will ask however is, “Isn’t this just SRE?” The purpose of the term is to change the focus from simply reacting to incidents to developing long-term response strategies for them.

    Because the expectation in these environments is that things will break, resilience is the responsibility of existing DevOps and cloud operations teams. When applications and services do break, a “fly by the seat of your pants” response strategy will not work.

    A Focus on Frameworks

    Resilience engineering, while rooted in engineering practices, is largely focused on building strategies and a framework for their execution. This leaves the process of building resilience into a largely unestablished system in part because each system is unique. And, how you respond to issues in that system will likely be unique, even if the management plane that reports issues is not.

    The job of resilience engineering is to:

    Establish procedures, habits and decision trees

    When things break, fight is the only option. Operators and on-call engineers need to address issues in a systematic and repeatable way and do their best to remove emotion and fear from the equation. This not only helps triage and resolve issues, but it also makes sure the activity associated with the issue leads to meaningful insights in post mortems and future collaboration. Part of that is establishing habits and decision-making processes for those who are on-call. The processes help prioritize what to focus on, help catch details because details are critical.

    Be data-driven

    Resilience engineering must rely on data. This is another place where traditional SRE practices grow with a focus on resilience. In typical SRE environments, the focus is on the now, using real-time dashboards of the current state. When you think about resilience you are thinking about the past’s impact on the future, not the now. The only way to do that is to make sure the data supports it; thus, part of resilience engineering is making sure the data is there. Instead of data silos that most organizations have across their delivery chain. Resilience engineering should ensure that telemetry across the entire delivery chain is captured, correlated and shared. The reason this is critical is that what happens early in the delivery chain directly impacts incidents. That activity can be the source of answers, it can be the triggers for rollback, or it can be the clarity needed to prevent similar issues in the future. Without continuity between each stage of the delivery chain, it’s easy to miss correlated events that can lead to more systemic problems.

    Engineering out of reproducible incidents

    For most, the best part of resilience engineering is taking what is learned from previous incidents and finding ways to automate future resolution. Learning from data and having consistency in habit leads to the ability to create runbooks and automate remediation for known issues. Often incident response audit trails can read like playbooks for addressing issues of a particular type. When the resolution is not directly related to code and the potential of issues to surface again in the future is inevitable, being able to build intelligence to address it saves waking someone up at midnight and much shorter impact on customers.

    The Resilience Stack

    Organizations looking to embrace resilience engineering need to have a toolkit built for it. It’s pretty straightforward but worth highlighting. The resilience stack will include:

    1. Observability and/or monitoring tool.
    2. Incident response tool.
    3. On-call strategy documentation.
    4. Post mortem process and documentation.
    5. Documented processes including recommend response steps.
    6. Path to automation.

    For those with a relatively mature and automated environment, the next step is chaos engineering—embracing chaos as a way to get ahead of incidents before they happen in the wild.

    Get your markdown skills to maximum level. There is a lot of documenting that needs to happen with comprehensive resilience engineering. But these documents should not be shelf-ware—they should be living and ultimately lead to the implementation of automation or feedback to development.

    As for monitoring, observability and incident response tools, it’s not enough to simply implement them. Knowing how data will be collected, consumed and actualized is also necessary. So is establishing an on-call strategy with purpose, not just because having everyone on-call is the “cool thing to do.”

    It may seem obvious to say that organizations want system resiliency. What is not obvious is how to execute it. When organizations see the gaps (and are often embarrassed by them), they understand that resilience is a focus for either current functions or new ones in the future. That is why it’s worthwhile to talk about resilience engineering and what makes it effective.

  • VMware Expands Reach of IT Management Portfolio

    VMware Expands Reach of IT Management Portfolio

    VMware today announced a bevy of updates to its portfolio of IT management tools that is now at the heart of its overall approach to both DevOps and AIOps.

    VMware vRealize Operations 8.2 and VMware vRealize Operations Cloud, the on-premises and software-as-a-service (SaaS) editions of the same IT automation platform, provide deeper support for Kubernetes distributions, including the VMware Tanzu Kubernetes Grid, as well as tighter integration with application performance management (APM) platforms from AppDynamics, Datadog and Dynatrace.

    VMware is also providing tighter integration with its VMware vRealize Network Insight to bridge the divide between IT and network operations and tools to better monitor instances of VMware software running on the Amazon Web Services (AWS) public cloud.

    Enhancements to the capacity and cost management tools that VMware makes available as part of the platform include more visibility into daily virtual machine (VM) costs, enhanced metering capabilities and support for non-VMware vRealize Automation workloads.

    At the same time, VMware is making updates to VMware vRealize Automation and VMware vRealize Automation Cloud. The latest updates to the VMware automation platform add support for VMware Cloud Templates, a way to manage infrastructure as code that is compatible with configurations created using Terraform tools.

    Other enhancements include self-service provisioning of Kubernetes namespaces via the service catalog, including support for VMware vSphere with Kubernetes, as well as more granular role-based access controls and application programming interfaces through which policies for VMware NSX network virtualization software can be applied.

    Updates to VMware vRealize Log Insight and vRealize Log Insight Cloud add enhanced Kubernetes support, deeper integration with VMware Cloud on AWS and other usability enhancements.

    Finally, VMware is streamlining the onboarding process for VMware Skyline, a predictive analytics tool the company makes available as an extension of its operations management portfolio.

    Ken Lee, senior director of product marketing for VMware, said VMware vRealize Cloud Management portfolio is evolving in a way that makes it accessible to both traditional IT administrators and DevOps teams. Each function in the platform can be invoked via a graphical user interface or application programming interface (API), he said.

    At the same time, VMware is infusing machine learning algorithms across the portfolio to enable AIOps, eliminating the need for IT organizations to acquire a separate IT management platform to employ AIOps or embrace best DevOps practices, he said.

    While the number of IT organizations that employ virtual machine software from VMware numbers in the hundreds of thousands, the number of organizations employing its IT operations platform is still measured in the hundreds. The company clearly enjoys the inside track in terms of relationships with those customers. However, it’s clear that the number of IT organizations relying on tools other than those provided by VMware to manage their environments is significant.

    As IT organizations look to modernize those IT environments, however, there is clearly an opportunity for VMware to gain additional traction. The challenge is the race to embed AI and DevOps practices into IT operations platforms is already quite heated.

  • How To Build a Business Data Map

    How To Build a Business Data Map

    Recently on DevOps.com, Anais Baig provided a brief overview of the privacy process and capabilities that developers need to think about to comply with new global privacy laws such as Europe’s GDPR and California’s CCPA. Top of the list for complying with these laws? The need to build a business data map.

    It’s no exaggeration to say that without a comprehensive, accurate data map—or, in other words, a complete inventory of the personal data the business possesses at any given time—it’s impossible to comply with these modern privacy regulations. Developers will understand better than most the difficulty in creating and then maintaining this record.

    But it can be done, and today we’re going to dive into the requirements and some of the technical processes that go into crafting a data map.

    What’s in a Data Map?

    In simple terms, a data map shows the personally identifiable information (PII) your business possesses, why you possess it, where it lives and how it’s being used. To answer these four questions, you’ll need to audit the databases in your business. Be sure to include those that are part of third-party SaaS applications as well as your own proprietary data systems; figure out the fields of information contained in each. Then you’ll need to coordinate with your team to understand the business purpose for which each of those data points is being used.

    The bulk of the work that developers typically do is that first step: auditing the business data system and documenting the database structures. At the very least, these are the things that you’ll need to be able to say about every field of personal information you possess:

    Source: How was it collected?

    Use Case: What is the business purpose for which it is being used?

    Storage: Where is it being stored for additional processing, transformation or record-keeping?

    Access: What business units have access to the data?

    Retention: What is the schedule for deleting this data or renewing the user’s consent to possess it?

    Consent: Was this data obtained through explicit customer consent?

    If you’re a small business, sometimes a data map can be “visualized” with a simple spreadsheet, but it very quickly uses its utility as data infrastructure scales. Larger businesses often find it very useful to construct a data map like an actual map, with a visual representation of the data relationships that exist in the business. In this form, a data map most often contains nodes and links to show how different systems that contain any personally identifiable information link together in your organization.

    Here’s one built by Ethyca:

     

    As you can imagine, without automated tools it takes a good bit of grunt work to get to this point of elegant visualization. For those just getting started mapping data in their business, we recommend a hybrid mapping approach.

    The Hybrid Approach to Mapping

    The hybrid approach means using an automated baseline process to identify and record the individual personally identifiable information that your organization is processing into one central repository, and then using manual label analysis data analysis to annotate and verify entity relationships across applications in your stack.

    Label analysis involves using a data attribute’s name to decide which data dictionary element the attribute identifies. For example, an attribute named ‘email_id’ can be identified as ‘person.contact.email-address’.

    When the attribute’s name alone is not enough, the name of the entity to which the attribute belongs is included in the analysis. For example, the attribute named ‘name’ in an entity named ‘customer’ can be identified as a name of a person, whereas a similarly named attribute in an entity named ‘city_master’ can be identified as a name of a city.

    Sometimes a group of attributes occurring together can be identified easily. For example, attributes with names ‘lat’ and ‘lng’ can be identified as latitude and longitude of a location. Similarly, attributes with names ‘f_name’ and ‘l_name’ can be identified as parts of a person’s name.

    Data analysis lets you manually confirm the relationships posited in your label analysis by using actual data from your databases to test.

    The presence of certain values or patterns in the actual data can confirm certain mappings. For example, if the data for an attribute follows ‘foo@bar.com’ pattern, it can be confirmed that this attribute captures an email address.

    Range values captured in attributes of different tables can be checked to see if those attributes are referring to the same piece of information, such as a user_id, that can be used for linking the tables with specific relationships. Similarly, the absence of values in an attribute can confirm the opposite.

    If you’re not using an off-the-shelf data mapping solution, the label and data analysis processes can be arduous. But if done correctly, by the time you’ve completed the exercise, you’ll have a comprehensive understanding of how the data in your business links together, and you’ll be ready to start answering the compliance-related questions about each PII field you hold. You can also begin thinking about processes for maintaining the map’s accuracy as your data inventory flexes and evolves.

  • Oracle Launches Managed JSON Database Service

    Oracle Launches Managed JSON Database Service

    Oracle today unfurled as standalone Autonomous JSON Database service that it will manage on behalf of developers that require a document database.

    Gerald Venzl, a developer evangelist and distinguished product manager for Oracle, said the Autonomous JSON Database service is based on support for JSON data types that Oracle currently provides in its Autonomous Database service based on its relational database.

    Priced at a quarter of Oracle Autonomous Database service, Oracle is also targeting rivals such as MongoDB by also pricing Autonomous JSON Database service at roughly 30% less than rival managed database service, Venzl noted.

    In addition, because Autonomous JSON Database service is based on the same platform Oracle employs to deliver its Autonomous Database Service, developers can add support for a wide variety of other data types by upgrading to the Autonomous Database Service with a single click, added Venzl.

    That approach provides developers with more flexibility should their data requirements change, or they simply initially chose the wrong type of database for their application, said Venzl.

    Other capabilities provided by the Autonomous JSON Database include support for built-in machine learning algorithms and spatial queries; fine-grained access controls; a server-side procedural language; low-code development tools; support for ACID-compliant transactions with no time or transaction size limits, support for cross-collection joins and/or aggregations and a search index for JSON documents.

    Oracle also makes available an open source document store application programming interface (API) dubbed Simple Oracle Document Access (SODA) that makes it simpler for developers to employ programming languages such as Java, JavaScript and Python to build applications on top of a JSON data store. Autonomous JSON Database stores JSON documents in a native tree-oriented binary format that Oracle claims is optimized for fast reads simplified updates.

    It’s not clear to what degree IT organizations will embrace managed database services. Oracle is making a case for managed database services that essentially eliminates the need to rely on database administrators to manage the infrastructure the database is deployed on as well as any upgrades made to the database itself.

    Developers in recent years have embraced document databases in part because they are easier for them to provision. In most cases, developers download a document database that they at least initially manage themselves. The challenge IT organizations inevitably encounter is the number of document databases that need to be supported grows rapidly. Developers who are focused on building applications don’t always have the time required to manage fleets of databases.

    Oracle is, of course, making a case of managing a variety of classes of databases. As enterprise applications continue to evolve in organizations the number of data types organizations are supporting continues to expand. A managed service focused on only one class of database may not be as appealing as a suite of managed service options.

    Regardless of approach, it’s clear database management is getting more automated. The opportunity and challenge now are finding the best way to incorporate database management with a larger set of DevOps practices.

  • Pushing Beyond Agility: The Kanban Maturity Model

    Pushing Beyond Agility: The Kanban Maturity Model

    Kanban is not a novel idea. It is a well-practiced and known method for organizations to increase productivity and efficiency, reduce the lead time of processes and improve delivery flow. However, historically Kanban has been frequently misunderstood.  Many people think that Kanban is just a whiteboard with stickies in columns of to-do, doing, and done. Or just a visual representation of a board in their agile tool. This resulted in many organizations missing out on the full potential of Kanban.

    The Kanban Maturity Model (KMM) is challenging this idea by defining certain steps for growing the maturity of the organizations that use the Kanban Method. The KMM collects and summarizes a decade of experience applying Kanban across diverse industries, in businesses of all sizes. By codifying these experiences, it addresses two common problems experienced in Kanban and agile implementations: overreaching and plateauing. We see overreaching when organizations or teams learn about some interesting practices and try to take them on before they are ready for them. While the practices have shown to be valuable in evolved environments, when taken on too early they can backfire. Plateauing happens when improvement levels off as the original problem that initiated the change initiative is abated. While there have been some improvements, there is potential for significantly more improvement. The KMM addresses this by identifying a road map and coaching tools for taking teams or an organization to the next level of improvement.

    As defined by Teodora Bozheva, Kanban trainer and consultant, and co-author of the “Kanban Maturity Model” with David J. Anderson, “The Kanban Maturity Model provides guidance for developing cultural values and management practices that enable enterprise agility, better customer satisfaction and business outcomes. It is an organizational maturity model that helps organizations evolve their agility and resilience.”

    The KMM maps Kanban practices as well as cultural values against seven organizational maturity levels. At first sight, the KMM may look complex. However, by advancing through the different stages, organizations are helped in a variety of ways with a road map to achieve their improvement goals.  For example, it can help relieve staff from overburdening, make processes more predictable and sustain a balanced workflow. In addition, businesses can deliver faster to market, align around a shared purpose and strengthen organizational values.

    When used in conjunction with software technologies such as Kanbanize, a Kanban automation solution for Agile Project Management, the model helps users fully optimize the workflow across all business functions and bolster the full intent of Kanban. Kanban software solutions and the KMM are stronger together. When companies simply invest only in Kanban automation solutions, they lack the required guidance and resources to fully maximize such tools. Similarly, without the automation of processes, insights and metrics provided by Kanban software technologies, deeper organizational maturity on the KMM scale is difficult to achieve.

    KMM: Understanding the Maturity Levels

    Transitioning from individual gain to the long-term company goals, each level represents a different step in the organizational journey toward agility with Kanban. The seven levels are arranged based on the company focus, main values, how stable the business is and what benefits it receives from Kanban. It is important to understand that each level of the model includes transition and consolidation practices. The transition practices are key to getting past the plateau and helping move from one level to the next,  causing the company to reflect on what can be improved in the process. The consolidation practices are then pulled to achieve the outcomes that the maturity level describes.

    Level 0 – Oblivious

    If your organization is not aware of the need for a structured work process, you are at maturity Level 0. Most organizations at this stage do not have a systematic approach to organizing the way they work and have a difficult time applying Kanban on a team level. At this stage, organizations typically visualize work on a personal level using either a physical or online Kanban board.

    Level 1 – Team Focused

    At maturity Level 1, team members already have built the habit of using Kanban on an individual level. After individuals start to recognize the advantages of simple workflow visualization, it is time to transmit the benefits of Kanban from an individual to a team level.

    An indication that an organization has initiated the transition toward using Kanban at the team level is that the work from the personal Kanban boards is moved to a shared team Kanban board using per-person swim lanes. This way, team members will easily recognize their own tasks, but at the same time, have an overview of what others are doing.

    An important step at this stage is to define and visualize initial policies. These are norms or rules that the whole team agrees to follow for the sake of normal functioning and improving work processes.

    Level 2 – Customer-Driven

    So far, your team should have an understanding that using Kanban helps to boost collaboration and bring transparency to the work process. The next step is recognizing the workflow. In other words, the team starts to understand what and how things are done and realize the need for a defined work process. The goal is for workers to understand what is expected of them and what they can expect of their colleagues with a shared vision of what finished product should look like.

    However, at this stage, while the process, policy usage and decision frameworks are consistent, there is still inconsistency of the desired outcome.

    Level 3 – Fit-for-Purpose

    By the time a company reaches maturity Level 3, they get a deeper understanding and better definition of their processes, workflow, policies and decision frameworks. It is important that the whole team agrees on all of them and understands how they help them achieve better results.

    As you start to outgrow maturity Level 2 and begin the transition from a customer-driven to a “fit-for-purpose” process, an organization becomes better at meeting customer expectations in a sustainable manner, as well as balancing upstream and downstream flow. The steps for maturity should be dedicated to complete alignment of the organizational requirements and processes needed to consistently meet customer expectations with predictability, a fast and balanced workflow, actionable metrics and shorter lead time.

    Level 4 – Risk Hedged

    Maturity Level 4 introduces risk-hedging practices, data-driven decision-making frameworks and use of predictive models. At this level, organizations embrace a model-driven management approach to strategic planning that anticipates risks, forecasts outcomes and results in consistent economics. All these improve overall business economics and robustness against unforeseen events and exceptional circumstances.

    Level 5 – Market Leader

    At maturity Level 5, not only have design, implementation and service delivery become routinely “fit-for-purpose,” the business is now entirely “fit-for-purpose” from a shareholder and end user perspective. The focus of this level is now on continually optimizing the workflow for efficiency and improved economic outcomes without sacrificing quality, increasing margins, minimizing costs and optimizing the value of work done.

    Level 6 – Built for Survival

    The deepest stage, maturity Level 6, is when a business can be established as “built to last.”

    The business is capable of questioning and assessing:

    • Is the way we do things still competitive?
    • Are new technologies, processes, methods or means becoming available that we should be investigating or adopting?
    • Do we offer the right products and services? and if not, how should we change?
    • Are we serving the right markets? Do we have the capability to serve our chosen markets adequately?
    • Who are we as a company? Is our current identity relevant and appropriate, or do we need to reinvent ourselves?

    These questions are strategic concerns, and the answers can help establish capabilities and services that align with the broader company strategy and bottom-line.

    A Road Map for Business Resiliency

    Over the past few years more and more people have learned, understood and applied the Kanban Method correctly and demonstrated significantly improved results. Knowledge work of all forms, from operations to project management to product innovation, the use cases and impact of Kanban across industries and business functions are extensive but sometimes underutilized. The KMM provides a greatly detailed map on how to take your company from a shallow practice that helps sort out individual needs to a fully functional business model. When adopted in conjunction with Kanban automation software, the KMM enables organizations to evolve and improve processes, business outcomes and customer expectations, and drive business resiliency and robustness.

  • Functional Safety in Embedded Design Starts with Code Development

    Functional Safety in Embedded Design Starts with Code Development

    With embedded software likely to become more complex, now is the time to start thinking about functional safety

    As the world has become increasingly more dependent on embedded systems in functional safety markets—such as rail, automotive, aerospace and medical—ensuring the safety of the software in those systems has become even more imperative. That assurance must begin with software development.

    As we all know, software development is the point at which many vulnerabilities are introduced, potentially leading to future performance, security and safety issues. The challenge for many embedded software developers is that they are often dealing with highly complex environments. In general, these complex environments are built using C and C++ programming languages.

    While C and C++ provide developers with the flexibility and scope for the innovation that embedded environments demand, they also introduce risk. Code can have undefined or unspecified behavior, or may not behave the same way when run on different hardware. Even the most skilled developer may make decisions that inadvertently lead to errors.

    Embedded Systems and Functional Safety: Cultural Appreciation

    There are ways to reduce software development risks. From a cultural perspective, safety and security need to be baked into the software development process, right from the very start and continuing throughout the entire DevOps life cycle. Of course, this is a practice that should be happening across all software markets, not just within embedded design.

    At the same time, there needs to be a greater focus on enforcing compliance with functional safety standards. This practice requires strategies to ensure the safety and security of systems. In some industries, such as rail and automotive, the use of these standards is mandated as part of approval and compliance processes.

    Functional Safety Standards

    Functional safety standards require performing a hazard analysis and risk assessment of the system. Based on the results, safety functions are then implemented to reduce the risk to an acceptable level. Functional safety standards set out appropriate software development and verification actions to mitigate known risks.

    Some of the main functional safety standards in use today include IEC 61508, which covers electrical, electronic and programmable electronic safety systems. First published more than 20 years ago, IEC 61508 is also the basis for several other functional safety standards including ISO 26262 for automotive, EN 50128 for rail, IEC 62304 for medical devices and ISO 25119 for agricultural machinery.

    Similar to other standards, functional safety standards do not always receive a warm reception because there is a perception that they require additional work and cost. In fact, by requiring consistent, high-quality coding practices, functional safety standards can reduce time-to-market and cost because they introduce best practice efficiencies into the development process.

    Coding Standards

    Most functional safety standards require or recommend the use of coding guidelines. For example, in the automotive industry, ISO 26262 mandates the use of coding guidelines and highlights specific areas that must be covered, such as the use of a safe language subset and naming conventions.

    Typically, a coding standard is made up of the collective knowledge of many people and organizations, such as manufacturers, consultants and industry associations. This collective knowledge is then used to provide developers with a set of rules to ensure that the code they are developing is safe, secure and compliant. Coding standards also engender better quality coding practices by making code more consistent, readable and understandable within a team.

    Depending on the project, organizations often choose to implement an external coding standard either fully or partially. They may also decide to create their own in-house coding standard based on an external standard or their own experience.

    While MISRA C and MISRA C++ were originally created for the automotive industry, the coding standards are now widely used across multiple industries. AUTOSAR C++14 Coding Guidelines were developed to address the requirements of modern C++ development environments, including autonomous vehicles and in the future will be used as the basis of a new MISRA C++ standard.

    Another popular coding standard used across multiple markets is CERT (coordinated by the CERT division of the Software Institute at Carnegie Mellon University), which focuses on security. With increasing awareness of cybersecurity risks, the use of CERT in embedded software development is likely to increase.

    Successful Implementation

    Regardless of the approach, successful implementation of coding standards depends on several factors. As there may be resistance to the implementation, teams need to be involved in the selection process of a coding standard and supporting tools. By being included in the decision process, it helps the team to understand why the requirement is critical.

    In addition, while it is possible to carry out manual code reviews against a coding standard, any project of significant size will need an automated static code analysis tool.

    Static code analysis tools can detect thousands of defects quickly and accurately, which frees up the developers to consider the implementation of the design. These tools work by both analyzing an individual developer’s code changes before they are committed and then scanning throughout the entire project as part of continuous integration (CI). For that reason, static code analyzers harmonize well within DevOps-led development environments.

    With embedded software likely to become more complex—especially against a backdrop of AI, ML and IoT—now is a good time to start thinking about putting in place an effective functional safety strategy, which includes tools such as static code analyzers and coding guidelines.

  • 4 Steps To Achieve a 66% Reduction in Test Run Time

    4 Steps To Achieve a 66% Reduction in Test Run Time

    Here are ways organizations can reduce their test run times to improve speed and efficiency

    Consult any recent DevOps survey and the same theme emerges: Testing remains one of the biggest roadblocks impeding organizations’ collective efforts to deliver better software, faster. Unlike days past, however, when agile development was still in its relative infancy and investments in testing lagged other areas of the delivery pipeline, organizations today are increasingly aware of the critical role continuous testing plays in delivering quality software at speed, and are aggressively investing in the requisite testing tools and technologies.

    No, the problem today is not lack of awareness or tooling. The problem, quite simply, is that testing is difficult, and only gets more difficult as organizations look to drive automation at scale. Nowhere is this challenge more evident than in organizations’ ongoing struggles with test quality. A recent study showed only 24.37% of organizations pass at least 90% of their desktop tests and only 25.45% pass at least 90% of their mobile tests. Remember, the purpose of agile development is to accelerate application delivery and shorten the release cycle. Organizations can do neither if they constantly have to manually follow-up to determine the source of repeated test failures.

    Doing It Right

    Organizations must find ways to improve test quality and speed. A major reason so many tests fail is they take far too long to run. The longer it takes to run, the more likely it is to end with a failure. According to the aforementioned study, those that complete in two minutes or less are twice as likely to pass as those lasting longer than two minutes.

    With that in mind, let’s take a look at four key steps testers and developers can implement to reduce test run times.

    Script Atomic Tests

    There’s no more important best practice to create a better and faster automation experience than to run atomic tests. These focus on just one piece of application functionality and are much faster and easier to execute than longer tests assessing multiple pieces of functionality.

    Let’s examine what this looks like in practice. Suppose an online retailer needs to validate that users can log in, view their gift card balance, add items to their cart, proceed to checkout and successfully process a transaction. Even an experienced tester will often make the mistake of scripting a single test to validate those five functions. The far better approach, especially if the aim is to reduce run time and improve test quality, is to write five separate tests, each specifically focused on one piece of functionality. Running individually but in parallel, these five “atomic” tests will execute in far less time than one longer test.

    Run Tests in Parallel

    Of course, if those five atomic tests are executed sequentially—that is, one at a time, one after the other—they’ll take even longer to complete than the one screen flow test, negating any potential runtime benefit. That’s why running tests in parallel is an equally critical component of any strategy to reduce test run times. Consider the hypothetical example of a test suite with 100 atomic tests, each of which takes two minutes to execute. Run those 100 tests sequentially and they’ll take more than three hours to complete. Run them in parallel, and all 100 will complete in just two minutes.

    Remember, too, not to get thrown by the volume of tests in a given suite. It’s tempting to think that a suite with 10 long tests that each take 5 minutes to complete will execute faster than one with 100 atomic tests that each take 1 minute to execute. But even if executed in parallel, the suite with the 10 long tests will still take 5 minutes to run—five times longer than the suite with 100 atomic tests.

    Reduce the Number of Selenium Commands

    Having too many Selenium commands in your test script flies in the face of atomic testing and is one of the most common underlying causes of long and unstable tests. Every single command takes time to execute and represents a new opportunity for something to go wrong. Minimize the number of commands required to execute a test case and run time will shrink accordingly.

    Use Explicit Waits

    Another effective way to reduce test run time is to use explicit waits rather than implicit waits. Implicit waits set a default wait time between each step or command across a test script, such that the subsequent step only executes after the pre-defined amount of time has elapsed. Explicit waits, on the other hand, enable the next step in a script to execute as soon as the preceding step is complete. Though more complicated to implement, using only explicit waits can have a significant positive impact on test run time.

    Putting it Together

    The example below shows an unoptimized test suite with 311 commands run against a mobile device. This example took 1,265 seconds (or more than 21 minutes) to execute. If delivering better software faster is the aim, that’s not going to work.

    Applying the aforementioned tactics to the same test suite executed against the same device, however, drove the total number of commands down to just 127 and enabled the suite to execute in just 430 seconds (or just over 7 minutes), a 66% improvement in test run time.

    Though the prospect of reducing test run time may be daunting, by focusing on a limited set of key tactics and best practices, the end goal of shorter, more stable tests is indeed attainable.

  • Healthcare Increasingly Embracing DevOps

    Healthcare Increasingly Embracing DevOps

    Speed and efficiency rank high as reasons why among respondents to a survey of healthcare organizations that have adopted DevOps

    While the novel coronavirus pandemic has hurt many industries, few have faced the immediate difficulties that the healthcare industry has had to manage. While trying to deliver care during one of the most significant healthcare crises in generations, providers must also contend with all the challenges every other organization must contend with as they strive to keep their business technology systems up and running securely.

    To that end, a recent survey found that healthcare organizations are increasingly embracing DevOps. According to the study from Redgate Software, the “2020 Database DevOps Report for the Healthcare Sector,” 70% of respondents have adopted DevOps across all or some projects or have a proof of concept to use DevOps techniques. In fact, at 41%, more healthcare organizations have moved out of the proof-of-concept stage than other sectors such as financial services and government, at 36%.

    “We know that the healthcare sector is facing unprecedented demands from the pandemic and other regulatory struggles, and DevOps is the best way to tackle these issues,” Kendra Little, Redgate DevOps Advocate, said in a statement. “By then extending those practices into the database, the healthcare sector can not only improve the quality and efficiency of these systems but ensure patient data remains secure and maintain compliance practices.”

    There are a handful of motivations behind the shift. The increased risk of failed deployments/downtime was cited as the top reason healthcare organizations moved to DevOps by 24% of respondents, while 21% reported inability to respond to changing requirements and 20% cited slow development and release cycles as the top reason.

    When it comes to automation, the report found that the healthcare sector is adopting automation across the database development process more than other industries. The majority are managing to do so while remaining compliant, or so they say—58% of respondents claim they have version control measures for database code in place. The primary motivations to automate database delivery had to do with efficiency and risk reduction, with 27% wanting to free developer time, 22% wanting speed of delivery and 12% wanting to reduce data loss risk.

    “What’s surprising this year is the way North American respondents are surging ahead, with 55% saying that they deploy database changes daily, weekly or even on-demand,” Little said. “This indicates that they’re embracing Agile practices to better meet the needs of doctors, patients and remote services like telehealth that rely on these databases each day. We’ll be watching this space closely as the sector continues to change during the pandemic and in response to pressing compliance needs.”

    With the rush to telemedicine and electronic medical records, it is surprising that only 42% of respondents said they use the cloud in some capacity. That’s a number that’s sure to rise in the months ahead as remote access to medical services continues.

    As organizations everywhere strive to do more with less, and remotely, health care is particularly challenged. And as the industry continues to work under strain, more will likely continue to embrace DevOps and cloud computing as ways to meet their objectives and deliver care reliably.

  • CircleCI Adds to Salesforce CI/CD Orbs Collection

    CircleCI Adds to Salesforce CI/CD Orbs Collection

    CircleCI today announced it has extended its CI/CD support for DevOps teams building applications that extend the software-as-a-service (SaaS) platform from Salesforce.

    Company CEO Jim Rose said interest in building applications that complement the Salesforce platform is rising sharply in the wake of the economic downturn brought on by the COVID-19 pandemic. Rather than build entire applications from scratch, IT teams are advancing digital business transformation initiatives by leveraging the core suite of applications Salesforce makes available as a cloud service, he noted.

    Many of those applications are built by professional developers that have embraced best DevOps practices, Rose said.

    To reduce friction for those DevOps teams, CircleCI is now adding support for a Salesforce Apex orb. An orb is a set of pre-configured connectors that CircleCI makes available for its continuous integration/continuous delivery (CI/CD) platform. The company has created more than 1,600 CircleCI orbs. Salesforce Apex is a proprietary object-oriented language that Salesforce makes available to developers.

    The Salesforce Apex orb automatically deploys changes to a production application, runs unit tests that are displayed using the Salesforce Platform user interface and tracks version control on commit.

    CircleCI has also updated its existing Salesforce command-line interface (CLI) orb to tighten integration.

    Rose said that as the pace of application development continues to accelerate, more IT teams are applying best DevOps practices to how they build and deploy applications as part of an effort to make the whole organization more resilient to change. Others are simply looking to modernize existing DevOps workflows that currently run on legacy CI/CD platforms.

    In either case, Rose said most end customers would prefer providers of CI/CD platforms to focus on making DevOps teams more efficient rather than moving into adjacent market segments to increase the size of their total addressable market. Those latter strategies are, in many cases, trying to drive consolidation of the overall DevOps market to woo potential investors rather than focusing on customer interests, he noted.

    It’s too early to say how the DevOps platform wars will play out. The size of the overall market continues to expand as the rate at which mergers and acquisitions are occurring accelerates. Regardless of the outcome, more IT organizations are under pressure to deliver more robust software faster, said Rose, noting the Salesforce application developer community represents only one segment of that expansion.

    Each organization will, of course, need to chart its own DevOps course. It’s not uncommon for different teams with the same organization to have standardized on multiple DevOps platforms. There may come a day when the need to reduce costs may force those organizations to rationalize DevOps platforms. However, as long as the rate at which high-quality software is delivered continues to improve, just as many organizations may conclude it’s just as well to leave something alone that doesn’t appear to be broken.