devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • Understanding SaaS Security for DevOps

    Understanding SaaS Security for DevOps

    As Software-as-a-service (SaaS) and DevOps adoption grew, new teams were formed to address emerging security challenges. Traditional solutions weren’t built to detect the new vulnerabilities in the cloud and created excessive noise for an already stressed pool of resources.  

    The combination of emerging zero-day vulnerabilities and an avalanche of false-positives ends up increasing security problems and slowing down DevOps projects. The good news is that recent developments in security and visibility are now increasing speed and improving visibility. 

    Gartner forecasted spending on public cloud services would reach $396 billion in 2021 and increase by 22% to reach $482 billion in 2022. SaaS applications based in the cloud have exploded with the market growing 18% each year; 99% of organizations will be using one or more SaaS applications by the end of this year. 

    What does this mean for security, especially for the security of personally identifiable information (PII) and business data in the enterprise? SaaS data includes, but is not limited to, customers’ sensitive PII and payment information; increasingly SaaS applications also house mission-critical enterprise information including product plans, patent information, business and operational processes and human resource records. 

    The extensive data held in SaaS applications presents huge security risks and costs for organizations globally. The average cost of a data breach exceeded $4.2 million, the most in the 17-year history of the IBM Cost of a Data Breach report, and nearly half of the breaches involved compromised PII, the costliest record type to lose, at $180 stolen record—an increase from $146 per record in 2020.

    SaaS Offers New Ways to Do Business but Introduces Significant Risk

    Salesforce is one of the most recognizable and widely used examples of enterprise SaaS software. Recently, analyst Vernon Keenan explained how Salesforce DevOps required guardrails. “Some companies go too fast when it comes to SaaS, DevOps and security, but smart developers and implementers will respect some basic guidelines to keep their product safe.”

    The same is true of other SaaS software platforms and solutions.

    General purpose application security testing solutions aren’t built for SaaS software environments, and often slow down DevOps processes as well as lack the necessary visibility into SaaS environments. Instead, the patchwork of tools must be replaced with a continuous integration and continuous deployment (CI/CD) approach. Specific steps include:

    1. Check all local and remote libraries. Only checking configurations and access controls, which focus mostly on insider threats, misses application vulnerabilities from custom development or app downloads that could open up your SaaS services to external threat actors.
    2. Carefully and routinely check third-party software libraries. If you are only testing source code and ignoring third-party software libraries, you are only securing half of your software application attack surface. CVEs are publicly reported every day on commonly used open source software libraries, showing attackers a direct path to compromise those key components. Running a software composition analysis (SCA) regularly to check all your locally bundled and remotely referenced libraries is a key step in achieving a secure software supply chain.
    3. Don’t assume you’re protected from common vulnerabilities. Static application security testing (SAST) can often miss cross-site scripting (XSS) or SOQL/SOSL injection attacks on SaaS platforms. 

    By identifying security vulnerabilities faster and more accurately, new SaaS security tools can allow organizations to gain the promised benefits of DevOps with better visibility.

     

     

  • Dagger: Standardizing CI/CD is the Holy Grail of DevOps

    Dagger: Standardizing CI/CD is the Holy Grail of DevOps

    Continuous integration and continuous delivery (CI/CD) has become a hallmark of quick software release life cycles. Nowadays, many teams support CI/CD pipelines for their software, offering a repeatable pathway to build, test and deploy code. A CI/CD pipeline is necessary to support an iterative, rapid deployment approach—yet, maintaining one is not an easy task.

    Enter Dagger. Dagger is a utility that aims to streamline how developers create and manage their CI/CD pipelines. Dagger, led by founders of Docker, recently unveiled their public beta launch along with a Series A $20 million funding raise led by Redpoint Ventures. While many open source CI/CD projects already exist on the market, Dagger is unique in offering easier configurations, pre-built components and more reusable actions.

    Akin to traditional supply chain management, software supply chain management is an increasingly complex endeavor involving many third-party dependencies. Development teams truly require a set of standard building blocks to piece it all together. For this reason, Solomon Hykes, co-founder and chief executive of Dagger, told SiliconANGLE that “solving this problem is the holy grail of DevOps.”

    Below, we’ll look at the common stressors inherent in creating and managing modern software delivery chains. We’ll also take a peek into the Dagger beta to see what it’s attempting to accomplish.

    Issues Facing CI/CD

    Software development supply chains are growing larger and larger to support a complex network of software architectures. But DevOps engineers, who must automate this chain, are drowning in tool sprawl—and stitching these components together to make custom pipelines can become a big bottleneck.

    Especially as an organization expands its distributed microservices architecture into the cloud, it can be tricky to scale a repeatable CI/CD pipeline across various department and disparate tech stacks. This often results in many bespoke CI/CD pipelines. Even within the same team, nuances between codebases can complicate each CI/CD process and require engineers to rewrite scripts.

    Simultaneously, organizations are under intense pressure to deliver to meet new digital innovation demands. But, until DevOps can make the release process more efficient, software development automation risks introducing more toil than it takes away.

    Diving Into Dagger

    So, how is Dagger alleviating these concerns? Well, Dagger creators described it as a “portable devkit for CI/CD pipelines.” Using Dagger, DevOps engineers can use a standard set of building blocks to assemble their CI/CD pipelines and then run them on their chosen cloud. It’s like a better developer experience for CI/CD pipeline creation.

    So, how does Dagger work under the hood? Dagger is powered by Buildkit, the Dockerfile-agnostic builder toolkit. Developers then use Cue to specify how their code is run in a container and to link up actions. Cue is an open source configuration language developed at Google.

    According to the docs, it takes about five minutes to install Dagger and the necessary dependencies. Dagger also provides boilerplate code to integrate with pre-existing CI environments. In essence, Dagger doesn’t need to replace your CI tool—it adds “a portable development layer on top of it.”

    Then, there are the actions themselves. Dagger supports a growing library of actions that it calls Universe. At the time of writing, the Universe of actions is limited and is being staged in the main Dagger repository. These actions are reusable Cue packages. For example, this package details how to run and deploy the Nginx web server in Cue:

    package nginx

import (
    "universe.dagger.io/docker"
)

    And then,

    #Build: {
    output: docker.#Image & _pull.image

    _pull: docker.#Pull
    *{
        flavor: "alpine"
        _pull: source: "index.docker.io/nginx:stable-alpine"
    } | {
        flavor: "debian"
        _pull: source: "index.docker.io/nginx:stable"
    }
}

    Benefits

    CI/CD can certainly benefit from standard building blocks and more fluid wiring between tools. There are several potential benefits from Dagger’s proposition:

    • Unification: Unifying developer and CI environments could avoid headaches.
    • Localization: Dagger enables testing a closed-loop pipeline locally. This helps for debugging.
    • Reusability: Building blocks help avoid rewriting automation scripts repeatedly.
    • Multi-cloud: Easier ability to migrate a pipeline from one cloud to the other, thus avoiding vendor lock-in.
    • Compatability: Dagger can run in any environment compatible with Docker, meaning it’s probably workable with your current toolset.

    Final Thoughts: Lego Blocks For CI/CD

    Docker revolutionized the software industry by easing the process of managing containerized applications. And now, Docker founders Solomon Hykes, Sam Alba, Andrea Luzzardi and others at Dagger are tackling another meaningful problem within today’s complex software ecosystem. Given the success of their previous endeavors, it’s not too difficult to imagine a network of reusable CI configurations emerging with an excitement similar to Docker Hub.

    Of course, Dagger is still in its early stages and there’s much development ahead. It will undoubtedly be an interesting project to follow in the years to come. Dagger is being built as an open source project on GitHub. The company has also opened a Discord channel to engage with the community.

  • How to Get Started With Low-Code/No-Code

    How to Get Started With Low-Code/No-Code

    Software development is complex. It can take years of practice to become an experienced developer. But the good news is that you can build many applications without ever learning to code.

    Low-code/no-code development is the practice of using a visual interface and drag-and-drop elements to build applications for business processes without coding.

    These types of tools can be used by non-technical users to build many types of applications; these can then be integrated with other software systems. By learning low-code/no-code development, you will be able to create software for yourself or your business much faster than if you were to learn how to code the traditional way.

    In this post, we’ll cover why you should learn no-code development, along with some great learning resources and communities you can join to get started with low-code and no-code development.

    Low-Code/No-Code is Gaining Popularity

    The internet is a very different place today than it was even a short 10 years ago. In the past, websites simply aggregated information to an audience on the web. Today, websites are designed to engage and encourage visitors to take action, from filling out sales forms to education and training to shopping and scheduling appointments.

    Fortunately, the tools available on the market today have made it possible for individuals without coding experience to build beautiful websites, useful business applications and productivity and automation enhancements to workflows using no-code tools. In fact, 26% of executives say that no-code, low-code development platforms are their most vital automation investment.

    Forrester Research believed that by 2022, the market for no-code and low-code development platforms will increase to $21.2 billion. Another report from Gartner estimated that by 2024, 65% of application development activity will be credited to no-code development.   

    No-Code is For Everyone

    While there is still a place for code and professional developers, no-code development gives individuals a way to build applications and boost their productivity and their effectiveness faster, cheaper and better than ever before. In addition, it relieves the burden on IT, software development and DevOps teams who can offload the work of creating applications to the actual users of those applications.

    No-code also allows for collaboration of citizen developers with other cross-functional teams in the business, while developing apps that can solve for their unique challenges.  

    Better Agility

    No-code can also accelerate digital transformation. The most important advantage of no-code development is speed. On average, it takes a few minutes to build a simple app or a website using no-code tools. When you’re developing more complex projects such as an entire business management system, this speed can be a major competitive advantage. It lets you quickly test your ideas and iterate until you get the results you want.

    Since most development is done with a drag-and-drop interface that is built visually using pre-built modules, you can create apps faster. Usually, even testing is automated, which further speeds up development time. In fact, no-code platforms can reduce development time by 90%.

    Reduced Cost

    Forrester reported that when one company passed up hiring two IT developers and implemented low-code tools instead, they saw an increase of about $4.4 million in business value from the resulting apps over three years. Naturally, since low-code/no-code technology removes the need for and additional cost of developers, it helps you cut those costs as well. 

    Increased Productivity

    People who make the switch from traditional coding to no-code development often say that it feels like a new world. It’s fast, easy and efficient—and you can create beautiful results with relatively little effort. 

    It is estimated that 71% of organizations that leverage citizen development could speed up app development by at least 50%. And 29% of companies noticed delivery speed doubled. These numbers suggest that citizen development and no-code development can increase productivity tremendously.

    No-Code Apps Are Customizable and Scalable  

     If a company wants to add a new feature or change existing ones, many no-code apps are flexible enough to adapt. The majority of no-code app development platforms allow you to add new features and customize your app. Low-code apps, on the other hand, are slightly less flexible and often have predefined components that make it difficult to customize the app.

    Plus, because of the drag-and-drop elements, no-code app development platforms allow you to build an app quickly. You don’t need coding skills to create an app. Instead, you will rely on prebuilt modules and templates.

    Getting Started With Low-Code/No-Code 

    There are lots of great resources out there to help you learn how to build your apps and workflows without code. Here are some of our favorite places to start!

    1. No Code HQ

    No Code HQ is a great resource for learning how to build your website or app without the need to code.

    It is a community of no-code builders, where they showcase their newest projects, share news and experiences and engage with and learn from each other. You will get tutorials about no-code development, templates and tools to start your no-code project.

    2. Nucode 

    This community has more than 5,000 no-code builders and experts. They learn and share experiences and showcase the progress of creating no-code products.

    3. No-Code Makers

    This forum is for no-code experts to learn, interact, share and collaborate with other no-code community members and creators. They regularly update the community with news about no-code tools and experiences. 

    4. Nocoders Academy

    This is another space for no-coders that arms you with valuable information. You will find many tutorials, learning materials and over 200 free resources on various tools. Soon, the site said there will be a formal traning academy with mentors and tutors. 

    5. NoCode.Tech 

    This is a directory of over 200 tools and strategies implemented by developers and no-code makers. Even a novice can learn how to build no-code software and receive community support. 

    The idea of learning to code can be overwhelming, even for experienced designers. This is why no-code is so appealing to so many people—it is essentially instantaneous, which allows you to create projects without worrying about the coding process. Have fun!

  • The Missing Link in Continuous DevOps Cycles

    The Missing Link in Continuous DevOps Cycles

    Delivering software quickly, reliably and safely is at the heart of technology transformation and organizational performance regardless of industry. As part of overall technology transformation, many companies are recognizing the benefits of applying DevOps methodologies to areas outside of software development, such as general product development, IT expansion and scaling and new business processes. However, some organizations—and even software development teams—may be missing a crucial step in the product development life cycle. 

    The Missing DevOps Link

    DevOps methodologies are nothing new. In fact, part of the reason organizations are adopting DevOps practices for initiatives outside of software development is because of its tested and proven success when it comes to developing and deploying new processes and features. The exact steps involved in DevOps processes can be debated by reasonable people, but there are three generally agreed-upon factors involved: Continuous integration (CI), continuous deployment (CD) and continuous release (CR). 

    Continuous Integration

    CI refers to a strategy of constantly updating products with new features and code on a regular basis. The cadence of the updates varies from weekly to monthly, but the strategy remains the same: Detect problems early before they can snowball and impact further development down the line. 

    Continuous Delivery

    CD is related to continuous integration but is focused on the delivery stage. CD is executed when new features and development updates are deployed to production. The most critical factor of CD is consistent management of the product’s end environment. To maintain the consistent functionality of a product in the end-user environment, it must be regularly deployed and tested to ensure performance doesn’t suffer as a result of product updates and new features.

    Continuous Release

    CR is the final step before the product development life cycle starts over again. CR refers to the practice of continually releasing new product updates, features and capabilities to end users. The tricky part of CR is that you must execute CI and CD flawlessly to ensure the end user doesn’t experience any bugs or issues caused by an update. 

    But what’s missing? The answer lies in continuous exploration (CE). 

    Why CE is Critical to Product Development Success

    CI, CD and CR are often-discussed product development strategies, but they don’t tell organizations where to start with the DevOps life cycle. To begin developing new products or features, organizations must first identify what exactly they’re building and for whom. As I often say to my product team, you need to get the kitchen ready before you can cook a great meal, but you need to know what meal you’re making before you start cooking the wrong thing.

    CE refers to the continuous exploration of a company’s markets, target audiences and customers to understand their requirements and preferences. This may sound like the job of a marketing department, not product developers, but the two should work together in a coordinated fashion that helps to achieve both teams’ goals. For example, product development teams don’t know what new product features their customers love, which ones they hate or which ones they really need. On the other hand, marketing teams don’t know how to market products if they don’t understand the ins and outs of the new features and how they can be of service to customers. 

    Even departments within an organization that don’t develop products can benefit from CE. For example, if an organization is pursuing a cloud migration of on-premises software, they will likely hire an expert consultant or vendor to help them execute their cloud vision rather than developing their own cloud environment.

    However, simply moving all of the on-premises systems and data to the cloud at once could be a catastrophe. What if you migrate systems and processes to the cloud and they suddenly stop working? Unless you’ve performed the migration at a gradual pace by following CI, CD, CR and CE strategies, you don’t know what, specifically, is causing the system error in the cloud.

    In the same vein, by performing CE internally, organizations can determine which processes and on-premises systems would benefit most from cloud capabilities and execute the migration accordingly. This not only prioritizes technology transformation initiatives based on the needs of the business but may also let the business know which on-premises systems may no longer be needed in a cloud environment.

    Many organizations interested in improving customer experience (CX) and internal processes by leveraging DevOps strategies would benefit greatly from leveraging CE. Products and businesses processes should be designed to meet the needs of customers and drive bottom-line revenue. So, it stands to reason that the first step in meeting the needs of customers is understanding those needs. This cannot be done without CE as the foundational building block of product development life cycles. 

  • GitLab Updates CI/CD Platform to Improve Dev Productivity

    GitLab Updates CI/CD Platform to Improve Dev Productivity

    GitLab has made available a 14.9 update to its namesake continuous integration/continuous delivery (CI/CD) platform that adds epic-linking, integrated security training, rule mode for scanning result policies and a revamped design, among other capabilities. The updated environment makes it easier to discover the state of the application development projects, GitLab said.

    Cindy Blake, director of product marketing at GitLab, said these updates are part of an ongoing effort to embed technologies within the CI/CD platform that will make applications both more secure and compliant with various mandates before they are deployed in a production environment.

    For example, DevOps teams can now take advantage of a user interface through which they can use a rule mode to design and edit policies without the need to access YAML files directly.

    GitLab has also partnered with Kontra and Secure Code Warrior to provide content that will drive security training functionality. This content can be accessed via the GitLab platform whenever a vulnerability is discovered.

    The overall goal is to make it simpler for organizations to shift left because expecting every developer to become a security expert is not especially realistic, noted Blake. The ability to automate many of the security tasks that developers face is a way of showing empathy for the daunting challenges they face.

    In general, as application development becomes more complex, GitLab is moving toward making its CI/CD platform more accessible to a wider range of organizations, including those that might only be starting to embrace DevOps best practices. The epic-linking capability, for example, makes it easier to identify “related”, “blocking,” or “blocked” relationships to better track and manage epic dependencies across GitLab groups.

    The two biggest challenges organizations face today are securing their software supply chains and improving the overall productivity of the developers. In most cases, those two goals are tightly coupled; every minute a developer spends on security is one less minute they spend writing code. In fact, many organizations are finding that, in addition to salary and benefits, many developers are using the level of friction they will encounter when building and deploying applications as a guide to decide which organizations they want to work for. As a result, many organizations are now looking to modernize their application development and deployment environments with an eye toward attracting the best developer talent.

    At the same time, of course, organizations are looking to improve the productivity of the developers they do have. In the age of digital business transformation, there are many more application development initiatives being launched. The challenge organizations face is that hiring and retaining developers remains a major challenge.

    It’s not clear how eager organizations are to replace their existing CI/CD platforms. However, the pressure to improve the overall developer experience is mounting at a time when organizations are increasingly dependent on software. Arguably, going forward, the ability of any organization to compete will be dependent on the productivity of application development teams. Platforms and tools that get in the way of enabling organizations to build and deploy more secure software faster will be discarded.

  • Positioning ML Devs and Teams for Success

    Positioning ML Devs and Teams for Success

    Intelligent applications are (by their very nature) complex. While conventional software basically consists of one thing (code), intelligent software involves code, models and data. As previously discussed, three distinct fields—DevOps, MLOps and DataOps—have evolved to govern each of these interconnected disciplines. 

    Moving through the ML life cycle quickly and efficiently requires collaboration between teams in each of the above disciplines. Organizations should strive to make this collaboration as frictionless as possible to position their ML efforts for success. What follows are four places where we believe organizations should focus: 

    • Infrastructure
    • Team structure
    • Tools 
    • Project ownership and KPIs

    Build Dedicated Infrastructure for Each Pillar

    Infrastructure software is not one-size-fits-all across the three pillars of code, model and data. For example, DevOps teams have historically played the infrastructure role in software development, supporting application or product services. Think of regular software engineering teams who are supported by DevOps, producing different kinds of application services. App infrastructure is so baked-in to software development that most of us take it for granted.

    Just as regular software services require DevOps, data and model services require corresponding infrastructure. Data infrastructure is typically concerned with managing tools (such as Airflow installations, Spark clusters and feature stores), building data pipelines and ensuring that data services remain smooth and operational for data consumers. 

    On the other hand, model services are chiefly concerned with creating, running and organizing models in such a way that others can use them efficiently. Since the AI/ML life cycle is incredibly complex, a host of factors need to be considered (e.g., hardware, parallelization libraries, model packaging formats, etc.) to help data scientists and engineers do their jobs efficiently. Robust ML infrastructure along with a dedicated ML infrastructure team goes a long way in helping organizations scale their AI.

    Optimize Team Structure

    Best practices for ML and AI are continuously evolving, and organizations have tried a host of team structures over the years. We have noticed two popular structures for intelligent application development teams emerged in recent years:

    • Pods

    Pod structures involve smaller teams—usually a data scientist, a data engineer and an application software engineer—working together closely to deliver an intelligent feature or product. Due to their tight-knit nature, these teams are positioned to iterate extremely quickly, which often leads to better products and features. However, the closely-knit nature of pods also comes with a downside, as knowledge sharing between pods with different objectives can be difficult.

     

    • Center of Excellence

    In a center of excellence (or COE) structure, all data scientists or data engineers are “pooled” and then assigned to different products or features depending on project requirements and circumstances. In contrast to pods, COE structures tend to share knowledge very easily. However, this arrangement can lead to slower iteration cycles because team members are not dedicated to particular products or features. 

    In our opinion, both pod and COE arrangements can be effective depending on the realities of an organization. Choosing an optimal team structure depends on how the rest of the organization is structured and how reporting relationships are set up. 

    Select Appropriate Tools

    To select the best possible tools for your organization, choose those that foster collaboration and uphold safety across code, model and data teams. 

    The right tools help teams work quickly and efficiently. Teams should be able to discover pertinent information for themselves without having to pull others off of whatever they are working on. For example, a data science team should be able to discover what data access or learn what features exist. Similarly, application teams should be able to discover what models exist, what versions are the latest or how and when to use models. The more that you can make your code, model and data self-serve, the faster teams will be able to build products that are ready to be put into production. 

    Furthermore, proper tools make debugging faster and easier. If you’ve been around ML-enabled features or products for any length of time, you’ve probably been bitten by undeclared dependencies and apps breaking due to upstream changes. In an intelligent app, finding the root of a problem can be extremely difficult because the root cause can exist in any of the three pillars or any combination of the three. Robust tools provide logs and information to help discover what went wrong more easily.

    Finally, governance should play a key role in tool selection, as safety should be a chief concern for ML teams. For example, consider a model that predicts the demographics of a user based on their app usage. While these predictions might be considered acceptable in certain applications, if a model is being used to make financial recommendations or approve credit applications, use of certain demographics may violate various consumer protection laws. Since a software engineer using this model might not be aware of the nuances related to its use, baking in safety checks on model and data use becomes vitally important. 

    Establish Ownership and Determine Appropriate KPIs

    Whether we are working at a company or in an academic lab, our goal always remains the same: build the best product. 

    But “best product” can mean a lot of different things depending on the use case. For example, sometimes teams may be willing to trade a bit of accuracy if a model fails in predictable ways (e.g., in health care applications.) By contrast, an e-commerce recommendation system can optimize for overall model accuracy even if a model isn’t quite as stable. As a result, choosing the appropriate KPIs for model optimization requires careful attention.  

    Similarly, choosing appropriate metrics that make sense for the product and business usually requires a high-level view of a number of factors — user expectations, revenues, user complaints, etc. In our experience, product owners are usually in the best position to make these decisions. KPIs for each of the pillars supporting the intelligent app should be based on key metrics determined by product owners. While these can range, we’ve seen the following KPIs work well:

    • Code — Number of releases, number of bug fixes, number of features that have shipped, mean time to resolution
    • Model — number of model iterations shipped, model quality in production, model performance SLAs 
    • Data — frequency of data refreshes (e.g., every day or every month?), number of data quality issues, speed of access to data

    The key point here is to drive all of these based on the product KPIs and to focus on the agile principle, which is how quickly are you moving and how many deploys are happening. 

    Know what metrics your team is looking to optimize so that you can invest your resources in the right spot. 

  • Lapsus$ Shames Okta/Sitel | Bitcoin Nukes Climate | EU DMA E2EE FAIL

    Lapsus$ Shames Okta/Sitel | Bitcoin Nukes Climate | EU DMA E2EE FAIL

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Okta and Sitel under fire over Lapsus$ hack, Greenpeace and others call for bitcoin change, and Europe still hates encryption. (more…)

  • What Donuts Teach Us About DevOps and Delivery Risk

    What Donuts Teach Us About DevOps and Delivery Risk

    Today, most DevOps teams do not think too much about their approach to risk. Usually, the typical attitude focuses on what to add to reduce risk further. In other words, “Can we add more tests? Can we deploy more carefully?” More tests, more processes and less risk are all obviously good things. Or are they?

    As a programmer, I’ve seen a lot of projects where there are so many tests and so many processes that they actually become counterproductive. Those ‘overheads’ presumably do reduce risk, but nobody knows by how much. I call this attitude ‘blind risk avoidance.’ This line of avoidant thinking usually builds up over time as we react to failures. It’s a very natural reaction.

    But when risk avoidance becomes your standard modus operandi, your delivery cycle gets slower and slower over time. Your team starts to feel they are trapped in molasses but, as time goes on, they not only gradually train themselves to tolerate this but to actively justify that it’s a good thing. In the meantime, nobody pays attention to the fact that your software is losing a competitive edge and that you’re being out-innovated and out-executed by others. Nobody pays enough attention to engineering team attrition; losing people as they drop off, one-by-one to move to more modern software development teams with less baggage. The ultimate conundrum lies here: If you move too fast, you break things. But if you move too slowly, your business and your team are doomed.

    So what is the optimal approach for DevOps teams to take when it comes to risk? The truth is, neither extreme is desirable. Blind risk avoidance leads to a failure of value delivery. On the other hand, blindly ignoring risk means crashing and burning.

    The ideal balance lies in not avoiding risk entirely or ignoring the reality of risk. Instead, a sensible middle ground can be found in taming risk.

    Moving forward, DevOps teams and developers must focus on taming risk to deliver software confidently with data-driven actions, speed and quality. 

    Automation Can Reduce Operational Overhead

    Taming risk should become a larger part of DevOps culture. One of the most famous ways this approach has been adopted is by Google’s site reliability engineering (SRE) team. 

    Google’s SRE team embraces two key principles when it comes to taming risk: Automation for reduced operational overhead and structured and rational decision-making for better, data-driven choices.

    To explore this mindset of risk-taming at Google, let’s dive into how the SREs use automation to slash waste, save time and reduce repetitious, low-value tasks and tests. 

    At Google, SREs are software engineers with an extremely low tolerance for repetitive, reactive work. (Aren’t we all?) But here’s the difference at Google: Thanks to a workplace culture of taming risk, operations that do not add value to a service are avoided or automated away.

    The vast majority of DevOps teams hold the same beliefs as Google: That automation cuts down on waste and saves everyone time.

    We all know automation allows for lower operational overhead, frees up software engineers to proactively assess and improve the services they support and boosts team morale. No one wants to spend their time or energy on repetitive, low-value tasks. Automation frees up DevOps teams to expend their energy on what matters most: Continuous quality.

    Bottom line? Automation is integral to the idea of taming risk. Automation of repetitive, low-value tasks is the perfect way to embrace a mindset of taming, not avoiding or ignoring risk entirely.

    Data-Driven Decisions

    Another way that Google’s SRE team tames risk is through structured and rational decision-making based on data. 

    Data-driven decisions are crucial to the success of any software engineering team, project or mission. Without proper data and feedback, there’s no way for software engineers to make the best, most calculated decisions and mitigate risks. 

    A data-driven decision methodology can—and should—be applied to DevOps testing cycles. 

    Another key aspect of data-driven decisions that speaks directly to the concept of taming risk? Service level objectives (SLOs).

    A service level objective is an element of a service level agreement that measures the performance of the service provider. SLOs are created (and agreed upon) between the service provider and the customer and help to mitigate disputes between providers and customers.

    Now, let’s take the concept of SLOs and apply them to engineering and QA teams.

    In traditional organizations, engineering and QA are forced to interact intentionally with a healthy amount of tension. One team produces software and the other team tears this software down. While this kind of necessary and healthy conflict between teams can lead to better results, the inherent push and pull between engineering and QA can create an excess of interorganizational infighting.

    Engineering teams will always advocate for more velocity or blind ignorance of risk, above all else. On the flip side, QA will always advocate for quality, or blind avoidance of risk, above all else.

    Of course, neither side is entirely correct (nor incorrect).

    Instead, the solution to this tension lies somewhere in the middle: The taming of risk, not ignorance or avoidance of risk. This is where an SLO, or the numeric target that represents ‘somewhere in the middle,’ comes into play. For engineering teams focused on velocity and QA teams focused on quality, creating an SLO—a happy medium to work toward with regard to risk—will create a more balanced workplace culture. In short, this approach can be considered a taming of risk.

    ‘Donut’ Slow Velocity

    Now, we’re finally going to get to the donuts. To demonstrate how embracing a mindset of taming, not avoiding, risk can create better, happier and faster DevOps teams, let’s take a look at Krispy Kreme Donuts.

    Krispy Kreme churns out millions of delicious, hot, glazed donuts every single day. However, during production, it’s inevitable that one or more of those donuts will be flawed. (I know, you’re asking, “Is there really such a thing as a flawed donut?!” For the purposes of this analogy, let’s say yes, there is.)

    One common donut defect occurs when the donuts bunch up on the production line and clog it.  Now, let’s assume this common error is caused by the speed of the line. If the Krispy Kreme team attempts to solve this problem by pushing the line even faster, tolerance gets smaller and the risk of even more smashed, defective donuts only rises. That’s certainly not a solution to this issue.

    On the other hand, if the Krispy Kreme team slows down the donut production line speed, there will be a smoother—albeit slower—operation.

    Is Krispy Kreme better served by slowing down its entire production line and sacrificing its velocity to eliminate the risk of a potential defective donut? Of course not! 

    If that’s not true, then is Krispy Kreme better served by cranking up their production line speed to go as fast as possible, regardless of the risk of defective donuts? Again, of course not!

    For sweet success, Krispy Kreme must embrace a mindset of taming risk. By baking careful measurement and an iterative feedback loop into their donut production process, Krispy Kreme is better able to create delicious, high-quality donuts at an optimal speed, producing some flawed donuts. This balance of speed with quality creates the best economic outcome and the mindset and measurement allow for a constant evolution to risk-taming in the future. As the production line changes, the optimal speed also needs to evolve.

    DevOps teams should take this same approach. Tame the risk without sacrificing speed and quality. ‘Donut’ slow velocity to fix a few minor issues!

    If you’re still not convinced, here’s another way to think about taming risk that is decidedly less delicious than warm, glazed donuts: Taxes. 

    The IRS balances velocity and accuracy during tax season as they process millions of tax reviews. While suspect returns are flagged for audits, the IRS simply cannot catch every single flawed tax return. It’s not economical or even possible to run detailed audits on every single problematic tax return, every single year.

    So, instead, the IRS tames its risk with automation and data-driven decisions. The IRS focuses on creating and using methods to detect patterns that correlate with flawed tax returns. Then, once these data-driven problem returns are detected by automation, experts can devote their precious time and mental energy to tackle the biggest of the problematic returns. 

    Again, this approach to taming risk creates an inherent balance between quality and speed and allows for an evolution of the process as economic activities and laws evolve.

    Whether you love eating donuts or doing your taxes, the bottom line is clear. Most organizations need to balance potential risks with maintaining speed, quality and team morale. DevOps teams are no different. 

    How to Tame Risk in DevOps Teams

    So how can we, as DevOps teams, tame the risk?

    The basic idea here is to do smarter testing, not just more testing. The good news is that if you have a reasonable level of automation, the data coming out of your existing automation can make your testing smarter. Approaches like predictive test selection can let you spend your computer hours more meaningfully, allowing you to improve speed and quality at the same time. Approaches like flaky test insights can let your precious engineering resources focus on the most impactful effort to improve the quality of tests.

    The Power of Taming Risk

    What is the right balance between speed and quality? This is the constant question DevOps teams face. But, here’s how I answer this riddle: Taking calculated risk is the most logical and rational approach in the workplace.

    It’s easy to avoid risk. Anyone can skirt around potential pitfalls by avoiding new solutions and sticking to the same old methodologies. But it’s so much more satisfying to take a calculated risk rather than ignore or avoid the risk outright. 

    If you want to improve the velocity and the happiness of your DevOps team, it’s time to embrace a new mindset: Tame the risk, gain control.

  • How DevOps is Revolutionizing Embedded Development

    How DevOps is Revolutionizing Embedded Development

    There’s something curious happening on the desktops of embedded developers around the world. From home offices to cubes to labs, in lamp-lit rooms late at night and sun-lit rooms early in the morning, developers are doing something different. They’re beginning to implement DevOps practices in their embedded development.

    Something quite serendipitous has occurred to make this possible: The parallel rise of cloud-native computing and the explosion of internet of things (IoT) devices and edge technologies. A world with billions of embedded devices connected to the internet all the time, every minute of every day, requires a new way of designing, testing, deploying and maintaining these devices. At the same time, cloud-native computing has introduced continuous delivery (CD) and continuous integration (CI), among other methods, to enable automated software builds that are reproducible. This is creating a revolution in the embedded world.

    Until now, embedded software was written and deployed and then remained static in perpetuity because it was isolated. Not anymore.

    The Slog, Security Demand New Thinking

    Embedded software developers have traditionally faced a monumental slog to build projects. Typically, embedded developers have had to build an entire platform to build new software, including the kernel, the distribution and more. And all of that software would need to get (re)built anew every single time something changed. Everything had to be checked in. Everything had to be rolled out. There was a version control infrastructure team, a CI team, a QA team—and they all had to work together and be aligned to meet the production schedule.

    This approach to developing was incredibly slow, costly and clumsy and in no way architected for today’s world of connected devices, IoT and the edge. Consumer product makers (home appliances, for example) simply don’t have the resources to provide frequent updates. Additionally, consumers are not used to frequent updates on consumer devices that are not their phones or laptops. With the rise of modern applications, it is just no longer acceptable to build static software that is never changed or updated.

    With billions of devices connected to an always-on network, large-scale security vulnerabilities have never been a bigger concern or consideration among companies that have millions of nodes on a network. Today, companies need to be continuously connected to all of their nodes and devices to properly maintain and secure them.

    Enter DevOps and Embedded Innovation

    DevOps has already transformed the way cloud-native applications are developed. Software gets built often, but it also gets deployed often. That’s why web apps are constantly being upgraded. Now, embedded is poised to take advantage of this software version control system that automates software builds and that is reproducible. Bringing DevOps to the embedded software ecosystem is about providing a set of tools or a platform that does the version control, CI and CD all in a single package—a platform to simplify the workflows for companies that typically have to build all of this themselves now can deliver opportunities for tremendous efficiencies and innovation.

    Consider the airline industry as it was 60+ years ago. At that time, the airlines built their own planes from start to finish. Today, Boeing and Airbus build all airlines’ planes. Of course, each airline can customize their fleet from the overall flying experience to the interior branding, but everybody is leveraging the same core technology because airframes are really hard to build and the differentiation happens on top of that foundation.

    Similarly, embedded software developers can focus on the customization of their product, not the time-consuming build that provides no differentiation. This is the concept behind the cultural shift we see in embedded software: Buy the airframe and customize it rather than build it yourself. Also inherent in the DevOps model is a distributed development practice, where different team members in different roles can be anywhere in the world and still work together seamlessly.

    The pandemic certainly created more demand for this model, but distributed teams aren’t going away and are more prevalent than ever in technology development. Many embedded software developers are still working in the same way they were in the 1980s and 1990s. With DevOps practices and platforms in place today, embedded software developers can focus on the value-add of their application. By taking all the complexity out of building it yourself, companies today can simply leverage easily consumable and reusable tools to achieve what cloud developers have been doing for years.

  • Harness Acquires ChaosNative to Meld Chaos Engineering, DevOps

    Harness Acquires ChaosNative to Meld Chaos Engineering, DevOps

    Harness this week announced it acquired ChaosNative as part of a plan to more deeply integrate chaos engineering with DevOps workflows.

    At the same time, the company added a Harness Service Reliability Management (SRM) and Harness Security Testing Orchestration capability to its continuous integration/continuous delivery (CI/CD) platform.

    Chaos engineering, as a discipline, refers to experiments on a software system that test its resiliency in the face of an unplanned outage. The goal is to deliberately make resources unavailable to see how the application environment either adjusts or whether it fails.

    ChaosNative’s LitmusChaos platform, which manages these chaos engineering tasks, was donated to the Cloud Native Computing Foundation earlier this year.

    Scott Sanchez, chief marketing officer for Harness. said the company plans to shift chaos engineering further left as part of an effort to enable developers to test the resiliency of their applications before they are deployed in a production environment.

    It’s still early days where adoption of chaos engineering within DevOps workflows is concerned. Many IT leaders’ performance is measured based on the stability of application environments. The idea that they should deliberately break something as part of an effort to make an application more resilient requires a shift in mindset, noted Sanchez.

    However, once IT leaders start to appreciate how chaos engineering actually improves the availability of applications it’s only a matter of time before the practice becomes more widely implemented, he added.

    In the meantime, Harness is looking to automate more of the lower-level tasks that today’s DevOps teams often manually manage. Harness SRM enables DevOps teams to automatically track service level objectives (SLOs) that are attached to specific pipelines within the Harness CI/CD platform.

    The Harness STO module, meanwhile, makes it simpler to orchestrate security testing across multiple pipelines.

    In general, there is now more focus on the productivity of DevOps teams as the number of applications that are being built simultaneously increases. The challenge, however, is not just the number of applications but also the dependencies that exist between them.

    Harness is making a case for a CI/CD platform that automates DevOps tasks by employing, for example, machine learning algorithms to identify which applications are most likely to fail before a build is completed. The goal is to reduce the amount of time DevOps teams waste waiting for builds to complete only to discover that the application almost immediately failed.

    The time, effort and cost of maintaining a DevOps platform will naturally vary as the overall environment becomes more complex. However, as DevOps processes become more automated, the platforms themselves become more accessible to a wider range of organizations. Today, the effort required by a DevOps to set up and then support a CI/CD environment is often considerable.

    Nevertheless, as more organizations rely on custom applications to drive digital business processes, it’s only a question of when and to what degree DevOps best practices will be more widely adopted. After all, the total cost of application downtime is only going to rise in the age of digital transformation.