devopsy.kinsta.cloud

Category: DevOps

  • Continuous Delivery Challenge Looms Large in 2022

    Continuous Delivery Challenge Looms Large in 2022

    As 2022 kicks off, it’s becoming apparent that the major DevOps challenge organizations will face is mastering best practices that enable continuous delivery.

    A global survey of more than 19,000 software development professionals conducted by the Continuous Delivery (CD) Foundation found 44% of respondents use either continuous integration or deployment. However, less than one in five (18%) use both practices to automate the entire software delivery life cycle. Continuous integration is currently the most popular DevOps-related activity (32%) followed by continuous deployment at 29%, the survey found.

    The survey also found that only 14% of respondents said, on average, that they can go from code checked-in to successfully running that code in production in one day. Just 10% said they routinely deploy code to production each day. Nearly a third of respondents said they still manually approve code deployments to production.

    Tracy Miranda, executive director for the CD Foundation, said it’s clear there is a need for a better understanding of continuous delivery as a distinct DevOps discipline that stands apart from continuous integration processes. While the two DevOps practices are complementary, she noted that continuous delivery requires a level of engineering expertise that many developers either lack or are simply not interested in gaining.

    In fact, the survey found more than a quarter of respondents (26%) are not involved in DevOps-related activities at all. At the same time, however, the survey revealed that 80% of respondents that work for companies with two or more employees are involved in DevOps to some degree. Nearly 60% of enterprise developers that participated in the survey employ a continuous integration/continuous delivery (CI/CD) platform, as opposed to 48% of developers working in small businesses and 42% of contractors. However, it’s clear from the survey results that much of that activity is focused on continuous integration rather than continuous delivery.

    The survey revealed, for example, that less than a quarter of respondents (23%) are involved in infrastructure provisioning and management at all. That compares to nearly a third (32%) that are involved in application and infrastructure monitoring. Only 20% are involved in creating automated tests, Collectively, the survey results make it clear there is still a long way to go in terms of shifting responsibility for DevOps practices further left toward developers.

    One reason continuous delivery is so challenging is that automating deployments across platforms has never been especially easy. Each platform tends to be customized to the point where it is difficult to automate application deployments. In effect, each platform is its own unique type of “snowflake.”

    However, with more attention focused on the integrity of software supply chains in the wake of a series of high-profile breaches, Miranda noted that automating software delivery will play a major role in achieving automation goals by reducing the need to rely on manual processes that are more easily compromised.

    At the same time, more organizations are starting to embrace Kubernetes clusters—this can simplify automation of software delivery across a set of platforms that all share a common application programming interface (API), she noted.

    The growing complexity of application development projects, coupled with increased reliance on software to drive digital business transformation initiatives, will also accelerate the adoption of continuous delivery best practices, added Miranda.

    One way or another, application deployment in 2022 is shaping up to be a lot more automated than it was in previous years. The only thing left to be determined now is the degree to which that goal can be achieved.

  • NPMs Sabotaged as OSS Sustainability Crisis Continues

    NPMs Sabotaged as OSS Sustainability Crisis Continues

    A long-simmering debate over the sustainability of smaller open source projects moved beyond the theoretical when two widely used open source node packaged modules (npm) were deliberately sabotaged this week, allegedly by a primary contributor.

    Colors.js is an npm that has been downloaded more than 3.3 billion times, with more than 19,000 projects depending on it. Faker, meanwhile, has been retrieved 272 million times, with over 2,500 projects depending on it. Colors.js enables organizations to print colorful text messages on the console, while faker is used to generate fake data for testing applications. Developers that pulled a recently published Colors.js version found their applications caught in an infinite loop, printing ‘LIBERTY’ ‘LIBERTY’ ‘LIBERTY’ followed by a sequence of gibberish non-ASCII characters. Functional code, meanwhile, was also removed from Faker.

    Ax Sharma, senior security researcher for Sonatype, a provider of an open source software security platform, noted these actions occurred in the wake of the series of zero-day vulnerabilities that impacted the widely used Log4j logging tool for Java applications. Sharma first reported the update issues with Color.js and Faker.

    The small team of contributors that work on the Log4j project found themselves creating multiple updates to the package to address vulnerabilities of varying severity on the common vulnerabilities and exposures (CVE) list. There is some debate, however, over how many of those vulnerabilities warranted a CVE listing given their severity, said Sharma.

    That issue has now emerged as a flashpoint for contributors to smaller open source projects. These contributors contend that larger organizations are taking advantage of their efforts without making any substantial contributions to a project in return, much less compensating any of the contributors for their time and effort. The recent updates made to the npms are, essentially, a protest statement, explained Sharma.

    It’s not clear whether other contributors to small open source projects might follow suit, but the debate over the sustainability of these projects has become heated. Many contributors to open source software assume that the organizations who use the free software they created should assume responsibility for securing it. That “user beware” approach to security is understandable for contributors that are not compensated for their efforts. However, when asked to patch open source projects used by billion-dollar organizations—and do so on an urgent, emergency basis—the resentment among those volunteer contributors rises sharply.

    Fortunately, some effort is being made to address these issues. The Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation, raised $10 million to help maintainers embrace best practices and better protect open source projects from malicious code. Google has pledged $1 million to help open source developers adhere to National Institute of Standards and Technology (NIST) guidelines in response to the Biden administration’s recent executive order on cybersecurity. Administered as a pilot program by the Linux Foundation, that effort is part of a larger $10 billion commitment that Google previously made to open source security.

    White House national security adviser Jake Sullivan also recently sent a letter to major software companies and developers inviting them to discuss initiatives to improve open-source software security. The first step is a one-day discussion this month hosted by Anne Neuberger, the deputy national security advisor for cyber and emerging technology. In the letter, Sullivan specifically noted that while open source software has accelerated the pace of innovation, much of it is maintained by volunteers. This is now a key national security concern, he noted.

    It’s not clear how this emerging open source sustainability crisis will play out. However, DevOps teams might want to consider how dependent they are on open source projects whose contributors might have issues with how their work is being exploited.

  • GitLab Revamps Observability Strategy After Acquiring Opstrace

    GitLab Revamps Observability Strategy After Acquiring Opstrace

    GitLab plans to make observability a more integrated element of its continuous integration/continuous delivery (CI/CD) platform after acquiring Opstrace.

    Opstrace’s open source project exposes a horizontally scalable application programming interface (API) created by Cortex for Prometheus, an open source monitoring platform being advanced under the auspices of the Cloud Native Computing Foundation (CNCF). It then makes use of a Loki API to collect log data.

    DevOps teams can also point an existing instance of Prometheus to Opstrace or any instance of Fluentd or Promtail software they are using to collect log data, with support for a wide variety of other APIs, such as the Datadog agent, planned. A command line interface (CLI) is provided to set an instance of Opstrace that creates a controller through which the collecting of log data is managed.

    Kenny Johnston, senior director for product management for GitLab, said as part of an ongoing effort to make observability more accessible, GitLab plans to integrate Opstrace by default with GitLab Monitor for organizations that use either the self-managed version of GitLab or the software-as-a-service (SaaS) platform.

    Opstrace currently collects logs and metrics and support for distributed traces is planned. The goal is to extend the overall scope of observability capabilities that GitLab already provides within its CI/CD platform, noted Johnston. The goal is to make it simpler for DevOps teams to employ observability capabilities and interrogate data to optimize workflows without acquiring a separate observability platform, he added.

    GitLab is betting that, as Prometheus continues to gain traction, more DevOps teams will prefer to extend its reach into the realm of observability rather than having to manage a separate platform.

    Although observability has always been a core tenet of DevOps best practices, achieving it has proven to be a challenge. At best, most organizations can continuously monitor IT environments based on predefined metrics that identify when a specific platform or application is performing within expectations. Observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without having to rely solely on a limited set of pre-defined metrics that have been created to monitor a specific process or function.

    The rate at which DevOps teams will embrace observability will naturally vary. In 2022, the biggest obstacle might not be the technology as much as it is simply understanding what queries can help DevOps teams better understand the root cause of an IT issue before there is a major disruption. In the long term, it’s expected that machine learning algorithms will leverage the data collected by observability platforms to automatically identify issues that might lead to a disruption long before it actually occurs.

    One way or another, the ability to discern events even within the most complex IT environment is steadily improving. Whether that’s accomplished as an extension of monitoring or by embracing true observability may not be all that relevant—as long as the ultimate goal is achieved.

  • Intel Expands Scope of OneAPI Toolkit Initiative

    Intel Expands Scope of OneAPI Toolkit Initiative

    Intel has made available a 2022 edition of its oneAPI toolkit through which it makes processor extensions accessible to DevOps teams.

    James Reinders, chief evangelist for Intel software products, said the goal is to make it simpler for DevOps teams to expose extensions to any company’s processors in the runtimes developers use.

    At its core, the oneAPI toolkit is a collection of APIs that Intel created for x86, graphics processing units (GPUs), application-specific integrated chips (ASICs), field programmable gate arrays (FPGAs) and other classes of processors.

    Reinders noted that developers have never had as much ready access to so many different processor classes before. The challenge organizations face, however, is making it simpler for developers to take advantage of those capabilities in a way that doesn’t require them to write low-level code, he added.

    The oneAPI 2022 toolkit adds additional capabilities including the world’s first unified compiler implementing C++, SYCL and Fortran, data-parallel Python for CPUs and GPUs, advanced accelerator performance modeling and tuning and performance acceleration for artificial intelligence (AI) and ray tracing visualization workloads. Collectively, Intel said it has made 900 new and enhanced features available via the oneAPI toolkit in the last year, including deeper Microsoft Visual Studio Code integration, support for Microsoft Visual Studio 2022 and Microsoft WSL2 for Linux development on Windows.

    As IT continues to evolve it will become more routine for developers to invoke heterogeneous classes of processors within the same application, said Reinders. Those capabilities will drive an explosion of software development innovation through the rest of the decade and beyond, he added.

    Intel is especially focused on making it easier for developers to invoke inference engines that take advantage of processor extensions to run AI workloads, Reinders said. Going forward, most applications will take advantage of machine learning algorithms that run inference engines optimized by Intel via extensions to its processors. As those workloads are constructed, integrating AI models within DevOps workflows will become the next major challenge, noted Reinders.

    It’s too early to say how simplified access to heterogeneous processors, also known as XPUs, might change the application development landscape. The one thing that is clear is IT organizations now have more options than ever. The issue now is finding a way to invoke all those capabilities in a way that doesn’t overwhelm developers. DevOps teams will play a key role in deciding which processors and associated extensions to make available to their development teams.

    In the meantime, the battle for processor dominance will only intensify in the months ahead. DevOps teams will need to decide how comfortable they are using processors manufactured by a single company like Intel or whether they want to employ a mix of processors provided by multiple vendors. Regardless of what they decide, it appears APIs will play a critical role in making it easier to benefit from advances in processor technologies.

  • Crafting an Agile Enterprise Architecture

    Crafting an Agile Enterprise Architecture

    Digital transformation, business agility and enterprise architecture are frequently heard terms in this era of digital technologies and rapid change. The appropriation of digitalized applications, solutions and processes has enabled organizations to increase efficiencies. In turn, this allows them to respond to shifting customer needs and trends in a more flexible and agile manner. 

    Digital transformation, the rise in mobility and a remote workforce have created a shift in the business model paradigm. The adoption of a more agile approach has been gathering steam, and businesses that are building agility into their processes are seeing the payoffs. The results are greater productivity, faster time-to-market (TTM) and increased consumer satisfaction.  

    The Culture of Agile 

    Digital transformation succeeds in a business environment that embraces a more collaborative and flexible philosophy. Moving from simply doing agile business processes to being agile across the board requires a shift in the corporate mindset. When a business develops a truly agile architecture, everyone in the enterprise is involved in delivering solutions; from business operations and tech to marketing, finance, compliance and security. The agile culture moves from a company-oriented one to a people or customer-oriented culture, embracing the tenets of collaboration, co-creation, ownership, flexibility and communication.

    Short-Term Cycles and Fluid Planning

    Agility is redefining enterprise architecture from the traditional schematic of centralized governance, lengthy development cycles, scheduled quarterly reviews and reliance on a waterfall delivery method. Instead of long-term operating and strategy plans, agile enterprise architecture involves short-term cycles and fluid planning that is customer and market-centric. The language changes from top-down project mandates and management to continuous product development, experimentation, testing and monitoring within collaborative product teams. 

    Essentials for Crafting Your Agile Enterprise Architecture 

    The blueprint for a truly agile architecture requires fundamental shifts in the business dynamic. There are three essentials that stand out as requirements for attaining agility across the entire enterprise. 

    • Collaboration – The bedrock of successful enterprise-wide agility is collaboration. Innovation will flourish when it is decentralized, and isolated silos give way to cross-functional, agile and self-organizing teams. An isolated IT team leads to delayed projects, overrun budgets, productivity that is hard to measure and disconnects between business and operations. Every department must be involved in supporting and achieving key business goals. 

    Teams containing a mix of business line and IT professionals accelerate development and delivery, greatly reducing time to market. Based upon a shared customer-centric goal and vision, there is shared ownership of outcomes and a deeper level of engagement throughout the enterprise. Daily communication and collaborative feedback nurtures creativity, problem-solving, and drive continuous integration and continuous development. Collaborative input from colleagues, inside and outside the organization, is essential for enterprise, technology, and solution architects. 

    • Current state mapping – Adopting agile practices and principles throughout an enterprise entails potentially massive change. Without a clear roadmap for navigating the process, a business can flounder. Instead of achieving a leaner simplicity, there is increased complexity and a muddied vision. Before a business can successfully embrace an agile architecture, the current state of operations must be fully understood. Essentially, a blueprint of functionalities, capabilities, applications, and the tech stack must be crafted. Everything, from the data catalog to business process workflows must be mapped. Business leaders can ask the following questions to analyze the current state and assess impact points:
    • Which projects should we prioritize and what capabilities will they require?
    • Which processes rely upon what software applications?
    • If we remove one or two apps, how will that impact processes?

    Once the blueprint of the current state is mapped out, a business can work in a structured way to create the future state of its architecture with ease and at its own pace. 

    • Balance agile transformation with daily operations – Continuing to work with some of the current architecture for a time can allow the top level of the organization to ease into adopting the agile mindset and culture. In the iterative process of developing agile architecture, the architects, executives, and tech leaders will need to strike a balance between transforming initiatives and effectively maintaining daily operations. They will have to discern how much direction is needed versus support and how much they will set the pace.

    If the focus is first on value delivered to customers, then the architecture will be prioritized according to the most strategic projects, capabilities and applications. An agile architecture will adopt a ‘just enough’ approach to project design and delivery. 

    Agile Architecture Optimizes Collaboration

    Enterprise agility facilitates more direct communication and involvement of enterprise architects within the cross-functional teams. Through daily discussions and various touchpoints, architects can provide guidance, advice, and feedback on IT decisions and assist teams in jointly owning the solutions. 

    Working together, enterprise and solution architects can develop a complete view of the IT landscape and how it will need to be modified to fit into future needs; minimizing waste and optimizing resources. Solution architects can view the architectural map across all apps, solutions, and systems and collaborate on improvements. They can also help enterprise architects scale standardization across the entire business. That cornerstone of collaboration at the core of agile transformation enables architects and non-tech leaders alike to gain more clarity and alignment.

    The Value of Agile 

    Business value is greatly improved when enterprise architecture combines agile practices. According to a report from McKinsey & Co., businesses transitioning to more agile models can improve financial performance by 20-30%, with 30-50% improving operational performance, TTM, and customer satisfaction scores. Embedding an agile model within processes, people, and technology will cultivate ongoing support for CI/CD, groundbreaking innovation, synergistic collaboration — and ultimately maximize customer outcomes.