devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • How to Secure C-Suite Buy-In for Your DevOps Efforts

    How to Secure C-Suite Buy-In for Your DevOps Efforts

    You could be forgiven for assuming DevOps is the modus operandi only for big modern enterprises like Tesla and Uber, but this is not the case. DevOps builds on earlier approaches, such as Agile development and Lean methodology, by emphasizing the importance of collaboration between development and operations teams and the use of automation to streamline the software development process.

    So, while the term “DevOps” is relatively new, the principles and practices that it encompasses have been in use for many years.

    In a traditional waterfall model, the software development process is split into distinct stages, with each stage being completed before moving on to the next one. The development and operations teams work in separate silos, with little interaction between them until the end of the development process when the finished product is handed over to the operations team for deployment and maintenance.

    Benefits of DevOps

    In contrast, a DevOps model emphasizes collaboration and communication between the development and operations teams throughout the entire software development life cycle. The goal is to create a culture of shared responsibility for the success of the project, with both teams working together to ensure that the software is developed, tested and deployed quickly and efficiently.

    DevOps also emphasizes automation, with the use of tools and processes to streamline the aforementioned project tasks. This automation helps to reduce the likelihood of errors and allows the teams to focus on higher-level tasks.

    Overall, the key differences between DevOps and a traditional waterfall model are the emphasis on collaboration and communication, the use of automation and the focus on shared responsibility for the success of the project.

    In short: Speed, time to market, agility and cost savings.

    What’s Driving DevOps Transformation?

    Any company, regardless of size, can release faster with DevOps. Look at the agile FinTech success stories; look at the generative AI success stories. In terms of cost savings too, it’s not ‘How much am I paying for 10 people, 20 people, 100 people…?’ From a DevOps standpoint, cost means penalties avoided from failing to release on time, project delays and project poor quality.

    Overall, development practices are evolving. Some clients call DevOps ‘zero rework’ development, others refer to it as ‘shift-left development.’ Whichever term you prefer, the context is the same: They want faster release and to reduce duplication of work. The way to achieve ‘shift-left’ or ‘zero rework’ would be through DevOps transformation. All of the individual pillars of DevOps concern shifting left, causing them to release faster and achieve their cost optimization.

    However you refer to it, DevOps is mainstream because of its scalability benefits, as well as the agility to react to unforeseen situations during development, QA or operations. This equates to competitive advantages and future-proofing.

    Challenges of Implementing DevOps

    Every business is different, but many of the challenges are the same. One of the main challenges is understanding your company’s specific blend of pain points on its road to DevOps transformation. Common challenges include:

    Vision – DevOps is not a top-down initiative. The role of executives in DevOps is to define a vision; if the vision is not clear, you won’t have buy-in. You can have the best tools and technology but if you don’t have buy-in, then it won’t work.

    Investment – DevOps is a change in every aspect of people, processes, technology and culture. You can’t just bring in this change with zero investment. It’s not unusual for costs to go up for the first couple of years after moving to DevOps because you’re transitioning from manual testers and normal developers to fungible team members that come at a premium. The cost advantage shows up after at least a year as rework goes down and the quality of release incrementally increases, leading to a better customer experience. So, it’s self-funded from the efficiencies it will bring in … but only if you persevere.

    Celebrating Success – Too many people try to boil the ocean with DevOps transformation. This leads to frustration, misaligned teams and, eventually, poor results. If you start small with DevOps transformation, it soon shows its superiority as a business model. This inspires your leadership and your executive teams to see the benefit of DevOps and buy into it by osmosis rather than taking the word of that executive with a big idea and a big purchase order.

    Culture and People – This is more than just a technology play. In fact, technology is the easy part. People, processes and culture all need to transform to implement DevOps. You need to move from a project to a product-based methodology. From a people perspective, you can’t bring in manual testers or manual validators to do DevOps. Any successful DevOps implementation has to get this right.

    How to Implement DevOps: The Hub and the Spoke

    Again, start small. You could start with only continuous integration and then move on to add continuous deployment. If you start over aspirationally, you are more likely to fail and go back to the old ways. As you expand DevOps gradually, you can follow the hub and spoke model.

    In the hub and spoke model, the hub is a lean team responsible for setting up the tools, automation and best practices. The hub is responsible for defining what is to be executed. The spoke, on the other hand, is responsible for execution. The spoke is the extended team and their individual projects which they are responsible for executing.

    The hub is also responsible for enablement by defining what processes, best practices, tools and pipelines should be used. Teams in the spoke will use the pipeline to promote their goal from development to QA, from QA to staging and from staging to production.

    Next-Gen DevOps

    The evolution of DevOps continues even for early adopters, especially with the introduction of AI and generative AI. How do you use AI to validate your requirements? How do you use AI to decide what infrastructure is needed and when? How can you use AI to optimize your test sets? Next-gen DevOps is about smarter DevOps and intelligent DevOps.

  • Despite DevOps, Software Supply Chain Security Challenges Persist

    Despite DevOps, Software Supply Chain Security Challenges Persist

    A survey of 397 IT, cybersecurity and application development professionals conducted by Enterprise Strategy Group (ESG) found that while most respondents work for organizations that have adopted DevOps practices, multiple software supply chain security issues and DevSecOps concerns have yet to be addressed.

    The survey, conducted on behalf of Data Theorem, a provider of a platform for securing application programming interfaces (APIs), the survey also found 82% of respondents have adopted DevOps, with 51% reporting they make extensive use of DevOps workflows (51%). Nearly three-quarters (73%) are deploying code weekly and 31% are deploying code daily.

    However, securing DevOps workflows remains a work in progress, with 41% reporting security teams lack visibility and control of development processes. Other issues include new builds deployed to production with misconfigurations, vulnerabilities and other security issues (40%), developers skipping security processes (39%), software released without going through security checks and/or testing (38%), lack of security process consistency across different development teams (38%), security team can’t keep pace with release cadences (34%) and developers’ reluctance to work with security (29%).

    API security is also becoming especially challenging, with 92% of respondents dealing with at least one API security incident in the last 12 months. Well over half (57%) have dealt with multiple API security issues in the last 12 months.

    On the plus side, the survey also found that more than half (54%) of teams responsible for securing APIs are involved with development as soon as or before the APIs are published. A full 97% said developers either have a good (22%) or high level (71%) of API security knowledge. A total of 89% reported their organization provides formal API security training to their development teams. More than three-quarters (78%) can remediate an API vulnerability within a day, and 39% reported they can do so within hours. A total of 60% reported a dedicated budget has been specifically allocated to API security, with another 30% identifying it as a discrete item within their cybersecurity budget. Two-thirds (66%) said spending on API security would significantly increase in the next 12 to 18 months.

    The use of APIs is only going to increase in the months and years ahead. Well over one-third (36%) of respondents are using APIs across all their applications, with 50% expecting this to be the case in two years. Another 44% reported most of their applications are using APIs., with 32% expecting this to be the case in two years. More than three-quarters (76%) of organizations reported that they have an average of 26 APIs per application and 53% have more than a quarter of their APIs facing the internet.

    More than a third (35%) are updating APIs daily while another 40% do so weekly. A full 61% are using APIs to connect microservices.

    Data Theorem COO Doug Dooley said those APIs are now being targeted by cybercriminals that routinely test the weaknesses of every API after each deployment and update. The most common types of API attacks reported by survey respondents included exposure of data (32%), account takeover (31%), denial of service (DoS) attack (31%), attack on misconfigured API (30%), fake account creation (28%), API injection attack (28%), ransomware (27%), data breach (26%) and content scraping (26%).

    Despite these issues, however, nearly three-quarters (74%) of survey respondents said they believe their organizations have a robust API security program in place, with 22% reporting they have some process in place. Organizations are using a mix of approaches including API security tools (59%), web application firewalls (WAFs) (57%) and API gateways (50%), distributed DoS mitigation (48%) and bot management solutions (42%).

    Challenges included managing multiple tools (29%), visibility into and control of APIs (28%), inventorying APIs (27%), inconsistent adoption of APIs (27%), API security testing (27%), data governance/exposure (26%), discovering and remediating misconfigured APIs (26%), keeping pace with threats (26%) and accurately tracking API usage (25%).

    Significant API security concerns included API authentication issues (41%), account takeovers (39%), identifying and tracking third-party APIs (38%), using tools not purpose-built for security (38%), shadow/undiscovered APIs (38%), abusive behavior bypassing app security tools (38%), denial of service attacks (38%), data governance and/or data exposure issues as a result of insecure APIs (37%), abusive behavior bypassing bot management (37%) and outdated/unneeded APIs, also known as zombie APIs. (33%).

    The type of vulnerabilities that are of the greatest concern are sensitive data exposure (34%), access control vulnerabilities (31%), API business logic flaws (31%) and DDoS attacks (30%).

    More than 79% of respondents, however, are relying on manual testing and review processes to ensure APIs don’t expose sensitive data (79%), with 63% having automated alerting capability for when sensitive data is exposed.

    Overall, it’s apparent that more resources as being applied to securing software supply chains in general and APIs specifically. However, given the amount of software being built and deployed, it may be a while before organizations see the return on those investments.

  • GitHub Adds Accessibility Improvements for Developers With Disabilities

    GitHub Adds Accessibility Improvements for Developers With Disabilities

    GitHub today revealed it added several capabilities to make it easier for developers with disabilities to access its code repository platform. The new capabilities were announced to coincide with Global Accessibility Awareness Day.

    Capabilities highlighted include a navigation tool and a search tool that can be used with a screen reader that GitHub provides.

    In addition, GitHub’s contribution graph has been updated to remove barriers for keyboard-only and screen reader users.

    Finally, GitHub has made site-wide color contrast improvements to the light and dark themes presented by Primer, the GitHub Design System.

    Ed Summers, head of accessibility for GitHub, said there are more than 1.3 billion individuals with a wide variety of disabilities that could, with access to the right tools, write code. That community today, however, is underrepresented as a percentage of the larger IT community, he noted.

    However, as tools such as GitHub Copilot—a generative artificial intelligence (AI) tool that helps developers write better, more secure code—is paired with GitHub Copilot Voice—a speech interface for GitHub Copilot that is available in technical preview—the GitHub platform becomes more accessible to developers with disabilities, he said.

    Generative AI coupled with a speech interface that eliminates the need for a keyboard is lowering the barrier for entry for every potential developer. But for individuals with disabilities, it represents a tremendous advance, noted Summers. Anyone who understands how code should be constructed can build an application, he added.

    It’s not clear what impact generative AI will have on the global shortage of developer talent that has persisted for decades but it’s clear the pace at which applications can be developed is about to dramatically increase. In addition, many organizations have embraced working from home in the wake of the COVID-19 pandemic. That shift makes it easier for many individuals with disabilities to become part of the workforce because they no longer need to commute to an office.

    Generative AI platforms are fundamentally changing the way humans interact with machines. Instead of requiring a developer to create a level of abstraction to communicate with a machine, it’s now possible for machines to understand the language humans use to communicate with each other. Via a natural language interface, developers will soon be asking generative AI platforms to not only surface suggestions but also debug applications and write the code used to create an application.

    At this point, like it or not, the generative AI genie is out of the proverbial bottle. Just about every job function imaginable will be impacted to varying degrees. In the case of DevOps teams, the ultimate impact should involve less drudgery as many of the manual tasks that conspire to make managing DevOps workflows tedious are eliminated.

    It may be a while before the impact of generative AI is fully appreciated, and there will certainly be more advances to come that will be applicable to individuals with disabilities. In the meantime, however, many members of a DevOps team may not even realize a member of the team has a disability as the writing of code continues to be increasingly automated.

  • Boomi Brings Generative AI to Integration Platform

    Boomi Brings Generative AI to Integration Platform

    Boomi announced today it is applying generative artificial intelligence (AI) to enable organizations to automate integrations via a natural language interface.

    Boomi CEO Steve Lucas said Boomi AI makes it much simpler for professional and citizen developers alike to create integrations at scale using generative AI platforms to invoke the low-code engine that Boomi already provides to create integrations between thousands of applications.

    Those generative AI capabilities will complement a wide range of AI capabilities based on machine learning algorithms that are already embedded in the Boomi platform, he added.

    Generative AI’s Impact on Integration and Automation – Steve Lucas, Boomi from Techstrong.TV on Vimeo.

    Boomi has been widely used to drive business process integration via an integration platform-as-a-service (iPaaS) environment it delivers as a managed service. The low-code platform the company created significantly reduces the need to rely on professional developers to write procedural code using Java, for example, to create those integrations.

    Generative AI capabilities make it possible for almost anyone of any skill level to create those integrations across multiple types of application programming interfaces (APIs), noted Lucas.

    That capability also makes it possible for Boomi to notify end users when, for example, a data attribute in an application has changed and ask them if they would like to apply that change across their entire application environment.

    In addition, Boomi AI can also interpret the intent of a prompt to better orchestrate processes in addition to providing an audit history in natural language that explains how decisions were made and what outcomes occurred as a result. As part of that effort, Boomi algorithms are also trained to avoid biases and unfairness with established ethical guidelines.

    Boomi’s AI engine is trained using anonymized data collected from approximately 20,000 customers that have created more than 200 million integrations using the Boomi platform. Those integrations have been exposed to third-party large language models (LLMs) that the company is using to train Boomi AI, said Lucas. Boomi is not storing any customer data to train its AI models.
    DevOps teams, of course, spend hours creating integrations that should be much simpler to create, test and validate with the rise of generative AI. In fact, the number of integrations being created to drive a wide range of digital business transformation initiatives should substantially increase.

    In the meantime, DevOps teams would be well-advised to start evaluating which tasks they manually perform today that could instead be automated by a generative AI platform. Many of the routine integrations that DevOps teams create today using low-level APIs will be increasingly automated by end users. The ultimate goal should be to identify more complex tasks that previously DevOps teams would not have attempted simply because there wasn’t enough time.

    It’s still too early to determine what impact generative AI will have on the way IT is managed, but DevOps teams that are typically committed to automating as many processes as possible should be at the forefront of adoption. The challenge is going to be simply keeping track of the rate of change that might soon be enabled by these platforms.

  • Google Previews Generative AI Service to Improve Code

    Google Previews Generative AI Service to Improve Code

    Google is previewing an instance of a generative artificial intelligence (AI) service that surfaces code recommendations to make it possible to build high-quality applications faster.

    Specifically, Duet AI for Google Cloud surfaces code recommendations in real-time, generates full functions and code blocks and suggests fixes for vulnerabilities and errors in the code that it discovers. It also promises to enable DevOps teams to create custom large language models (LLM) based on their own code. In fact, GitLab and Tabnine are, respectively, already integrating LLMs running on Google Cloud with their continuous integration/continuous delivery (CI/CD) platform and testing tools.

    Chen Goldberg, vice president of engineering for Google, told attendees at the Google I/O 2023 conference that Duet AI for Google Cloud will fundamentally change how developers will be able to build applications, regardless of their skill level.

    Duet AI for Google Cloud was trained using data collected from Google Cloud using sample code and documentation to fine-tune behaviors and patterns.

    Duet AI for Google Cloud is available under an AI Trusted Tester Program and provides developers with many of the same capabilities that can be found in the GitHub Copilot tool that is based on the LLMs created by OpenAI.

    In addition to making Duet AI for Google Cloud accessible via both its tools and managed services, Google said it also plans to integrate the service with multiple integrated development environments (IDEs).

    It’s still early days as far as the use of generative AI within DevOps workflows is concerned, but it’s clear that the amount of code being created by developers is about to increase exponentially. Hopefully, AI technologies will also one day help software engineers find ways to manage the massive volume of code moving across their DevOps pipelines.

    Generative AI platforms will fundamentally change the way humans interact with machines. Instead of requiring a developer to create a level of abstraction to communicate with a machine, it’s now possible for machines to understand the language humans use to communicate with each other. Using a natural language interface, developers will soon be asking generative AI platforms to not only make suggestions but also debug applications and, eventually, write the code used to create an application.

    At this point, like it or not, the generative AI genie is out of the proverbial bottle. Just about every job function imaginable will be impacted to varying degrees. In the case of DevOps teams, the ultimate impact should involve less drudgery and toil as many of the manual tasks that conspire to make managing DevOps workflows tedious are eliminated.

    In the meantime, DevOps teams should spend some time identifying which tasks will become automated by generative AI and other technologies that employ machine and deep learning algorithms. It doesn’t make much sense to onboard new members to a DevOps team only to discover that the tasks assigned to that individual are about to be automated. Hopefully, when all these AI technologies are finally implemented, the entire DevOps experience will be better than ever.

  • The Hidden Configuration Tax Affecting Uptime SLO

    The Hidden Configuration Tax Affecting Uptime SLO

    This article is a preview of a talk by Greg Arnette for SLOconf 2023 on May 15 – 18. To watch this talk and many more like it, register for free at sloconf.com.

    Tax season was just a few weeks ago, and that got me thinking about how frustrating it is to get hit with a surprise tax bill and how that can relate to uptime: Configuration sprawl. DevOps practitioners constantly strive to maintain uptime service level objectives (SLO) while working with complex and ever-evolving infrastructure and applications. Configuration sprawl is a significant challenge many teams face, negatively impacting system performance and reliability.

    My company’s recent survey of 900 DevOps and platform engineering leaders revealed a startling (but not too surprising) conclusion: Most teams struggle managing secrets and configs at scale for infrastructure and applications.

    Where Does Configuration Sprawl Come From?

    As organizations scale their operations, the number of configuration files and settings grows, making it increasingly difficult to manage and maintain consistency across the board. They may have multiple environments, such as development, staging and production environments, often requiring different configurations, leading to duplicated settings and increased complexity.

    Configuration sprawl may also lack standardization as teams adopt different naming conventions, file formats and storage locations without enforced configuration standards, resulting in fragmentation and confusion. Relying on manual processes for configuration management increases the likelihood of errors and inconsistencies.

    Grinding to a Halt

    So you’ve got a lot of configuration–why should you care? Sure, it means more files to manage, but does it really impact your environment? There are a few places where configuration sprawl will negatively impact you. You might see an increase in error rates because as complexity grows, the likelihood of errors and misconfigurations increases, leading to system instability and downtime.

    Increased configuration complexity also means slower troubleshooting. Identifying the root cause of issues becomes more challenging and time-consuming when configurations are disorganized and fragmented. Ultimately this means less agility as you are forced to slow the deployment of new features and updates, impeding the organization’s ability to respond to market demands.

    How to Overcome Configuration Sprawl

    Don’t worry–you can tackle configuration sprawl, keep your SLOs on track and reduce the efforts required to manage and maintain configurations. To mitigate the harmful effects of configuration sprawl, consider adopting these best practices:

    Don’t repeat yourself (DRY)
    DRY config means variable names are declared once. Then flexible variable values are injected across build, deploy and runtime phases as defaults, inheritances and overrides. DRY avoids errors by allowing single changes to propagate where they are needed.

    DRY solves the common problem of forgetting to update critical secrets or parameters and failing to synchronize related changes (e.g., client and server port numbers or rotating a database password.)

    Decouple and abstract
    Decoupling and abstracting config means externalizing config from source code, separating how code and configuration interact, and isolating the specific values (log level) from the configuration interface (how the log level value is fetched.)

    This approach allows the same code to run in different environments using parameterized config. Abstracting config supports a best practice of implementing independent “code and config change” life cycles. Teams that frequently need to change config independently from code should decouple and abstract.

    Centralized config
    Consolidate configuration complexity into a single location because it’s easier to manage complexity within a single location versus multiple locations. Syncing config to the edge (where it is consumed) mitigates downtime. For example, it’s easier to fetch the configuration for service X versus coding all the logic needed to generate the configuration for service X.

    Standardize your configuration management by enforcing consistent naming conventions, file formats and storage locations across teams to reduce confusion and streamline configuration management. Centralizing the configuration and generating that configuration with transformations allows all the places that consume the configuration to be simple. Simple equals easy to understand and troubleshoot.

    Version control & monitoring
    Use a version control system like Git to track config changes to configuration files, making identifying and resolving issues easier. And continue to monitor changes and identify potential problems before they impact system performance.

    Train your team
    Ensure all team members know your best practices and patterns for configuration management. The proliferation of microservices and teams operating in parallel, establishing their unique naming schemes and patterns, exacerbates this problem for DevOps teams.

    Cutting the Uptime Tax

    By understanding the causes of configuration sprawl and implementing best practices to manage and reduce its impact, teams can improve their uptime SLOs and maintain a more reliable, high-performing infrastructure. Remember, investing in the proper tools and processes for configuration management is not only a best practice but also a critical factor in ensuring the long-term success of your DevOps initiatives. Those are tax cuts I can get behind.

  • Spotify Adds More Plugins for Backstage Developer Portal

    Spotify Adds More Plugins for Backstage Developer Portal

    Spotify this week added additional plugins for its open source Backstage platform that is used to build developer portals. The new plugins make it simpler to address role-based access and access Insights, a tool from Spotify that tracks Backstage usage trends.

    In addition, Spotify is also enhancing a Soundcheck plugin for Backstage that is used to visualize and track development of software components. Forthcoming capabilities include a no-code interface that makes it possible to programmatically create checks of code without writing any code.

    DevOps teams will also be able to view, export and understand how teams and components are doing compared to established best practices and monitor trends, graphs and historical views, and receive notifications when levels change.

    Finally, Spotify is committing to adding additional integrations with third-party tools and platforms such as Snyk, Sonarqube and the open source Argo continuous delivery (CD) platform.

    There are now five plugins available via a Spotify Plugins for Backstage subscription service, and Spotify said more are planned.

    Meg Watson, a group product manager at Spotify, said Backstage has been gaining traction in cloud-native application environments that are especially challenging to develop using multiple tools. Because of Backstage, developers are not only more productive but turnover has also been reduced because developers are provided with a better experience, she noted.

    There’s a lot more focus than ever on developer productivity, which Backstage addresses by creating a catalog of blueprints that developers can readily consume rather than requiring them to build these capabilities themselves multiple times over. The goal is to create scaffolds that developers can consistently reuse across multiple application development projects.

    Backstage was donated to the Cloud Native Computing Foundation (CNCF) and is now being advanced by contributions from multiple vendors. Much of that focus is on lowering the barrier of adoption for the platform, said Watson. Specifically, Spotify is working on a QuickStart for Backstage edition of the platform that is simpler to install, she said.

    In general, Spotify is trying to strike a balance between centralizing the management of DevOps workflows and the need to enable developers to define workflows that are natural to them versus ones that have been imposed on them, added Watson.

    It’s not clear to what degree Backstage will help fuel the adoption of a shift toward platform engineering to centralize the management of DevOps tools and platforms. The concept of a portal through which developers can self-service their own needs may not be new, but as an open source project, Backstage has made it simpler for many organizations to achieve that goal.

    Just about every DevOps team is now being asked to find ways to help improve developer productivity at a time when many organizations are trying to do more with fewer resources. Regardless of the motivation, however, it’s apparent that DevOps workflows are continuing to evolve and mature.

    In the meantime, DevOps teams would be well-advised to eliminate as many bottlenecks as possible before they negatively impact developer productivity.

  • Why We Need to Rethink Data Pipelines

    Why We Need to Rethink Data Pipelines

    Data is the fuel that powers modern business. But as demand for data surges, so does the pressure on data leaders and practitioners to deliver it. Businesses need resilient data pipelines that deliver critical insight for real-time decision-making to users on demand. However, against the backdrop of today’s chaotic modern data ecosystems, this is much easier said than done.

    DevOps teams are struggling with a complex combination of legacy and diverse technology systems across multiple environments. The resulting siloed systems and an ever-shifting data supply chain cause significant data integration friction. This friction keeps businesses from creating seamless and resilient pipelines to drive digital transformation and other business outcomes. Businesses need a better way to take the pressure off data leaders and DataOps practitioners and enable line-of-business teams to do more of the “last mile” data collection and analysis themselves. To achieve this, they need a unified, end-to-end platform designed to build resilient pipelines.

    Data Demand Sparks Friction

    The acceleration of digital transformation has created a demand for data that far exceeds supply. This has become especially true as competitive advantage gets harder to carve out and worrying macroeconomic headwinds gather. In today’s climate, all areas of the business are demanding data, but it’s hard to meet every request all the time.

    Research that polled data leaders and practitioners shows nearly half (48%) of admin and operations and customer service departments request data at least weekly, followed closely by accounting and finance (44%), IT and digital (43%) and sales and marketing teams (40%). And the strain is showing. Over half (59%) of data leaders said changing priorities have created significant data supply chain challenges.

    The problem is one not just of capacity but complexity. Building pipelines from source to destination requires rules to integrate, transform and process data. But when data is siloed across cloud, legacy and mainframe systems and stored in inconsistent formats, creating bespoke data pipelines to fulfill departmental requests is a huge challenge.

    Legacy systems also bring another challenge. Over half of data leaders say it’s so difficult to unlock data from legacy systems like mainframes that they simply don’t bother. That will ultimately undermine the value of the cloud analytics tools that many business leaders see as a single source of truth for decision-making–especially as legacy systems often hold decades of vital business insights.

    This patchwork approach creates extra toil for DevOps, DataOps and technical teams and means that over two-thirds (68%) of data leaders said they are being prevented from delivering data at the speed the business needs.

    The Scourge of Broken Pipelines

    For many organizations, building pipelines is also a labor-intensive job requiring a high degree of manual effort to produce hand-coded, one-off solutions. The resulting pipelines are brittle and vulnerable to being disrupted whenever there is a shift in the environment–such as adding new data sources. In fact, two-fifths (39%) of data leaders admit their pipelines crack at the first sign of trouble, and 87% have experienced a break at least once a year. More than one in 10 say it happens at least once a day.

    The most significant cause of breakages is due to bugs and errors being introduced during a change (44%) while infrastructure changes such as moving to a new cloud (33%) and credentials altering or expiring (31%) are also creating disruption. These broken pipelines immediately impact the corporate bottom line. They also force technical teams into a vicious cycle of firefighting. Data engineers spend, on average, almost a third (31%) of their time troubleshooting and recoding broken pipelines–time that could be better spent on value-adding tasks.

    But the negative impacts aren’t only felt operationally. The breakage of any pipeline can also lead to bad decision-making. For example, a supply chain director working with old data may over- or under-order goods or a financial trader making stock picks on out-of-date intel may lose money for a client.

    Unleash the Power of Data

    For organizations looking to overcome these challenges, the first goal is to reduce the workload on DevOps, DataOps and technical data teams by empowering business units and end users to do more. Research shows that 70% of technical data leaders are responsible for the last mile of data collection and analysis while 86% of them would prefer lines of business teams, such as marketing or finance, to be empowered to do this independently. However, this can only become a reality if organizations invest in the right data integration platform. It must be able to build, run, monitor and manage smart data pipelines at scale from a single console–across all cloud and on-premises environments. And it must be able to deliver resilient pipelines built to withstand continuous change.

    Only by eliminating data integration friction and enabling self-service analytics for lines of business can organizations accelerate digital transformation. Doing so will unleash the power of data across the enterprise and reduce the burden of overworked data leaders and practitioners.

  • CDF Report Reveals Lack of DevOps Maturity

    CDF Report Reveals Lack of DevOps Maturity

    A global survey of nearly 26,000 DevOps professionals published today by the Continuous Delivery Foundation (CDF) at the cdCon + GitOpsCon event found 84% of respondents are involved in some type of DevOps activity. However, the report also found just under half 49% are using a continuous integration (CI) or continuous delivery (CD) platform and only 22% are using CI and CD approaches to automate all building, testing and deployment of code in a production environment.

    The report, conducted in collaboration with SlashData, found the number of developers using CI or CD platforms only increased two percentage points year-over-year, with the use of self-hosted CI/CD tools dropping from 32% in the first quarter of 2022 to 23% in the same period this year.

    The survey also noted that only 7% of respondents this year described themselves as a DevOps engineer or specialist.

    Overall, medium-sized businesses have the highest involvement in DevOps practices (94%), followed by small businesses and large enterprises at just over 90%. More than three-quarters of developers (79%) are now using DevOps practices.

    The most common DevOps activity is monitoring software or infrastructure performance (47%), followed by application security testing and using CI to build and test code, tied at 37% each, and approving code deployments into production environments at 36%, the survey found.

    The average number of DevOps-related technologies used by developers (4.5) has, meanwhile, remained stable over the last year. The most widely used tools address issue tracking (39%), source code management (38%), test automation management (34%), collaboration and knowledge-sharing tools (32%) and application performance management/observability and agile project management tools tied at 31%.

    The survey also noted that the percentage of top performers—defined as those with lead times of less than one day—now stands at 15%. Survey results for top performers over the years continue to fluctuate between 13% and 17% of the survey total. On the plus side, 40% of developers have lead time changes of less than one week, the second-highest ranking for this survey.

    More than a third of all DevOps practitioners (34%), however, take longer than a week to restore service. Only 12% of DevOps practitioners take less than an hour to restore service.

    In general, the survey finds that organizations that use 10 or more DevOps tools are twice as likely to be high performers.

    Fatih Degirmenci, executive director for the CDF, told conference attendees the consortium, which now oversees nine projects, including the open source Jenkins CI/CD platform, continues to make progress as additional projects are added.

    However, while it’s clear that many organizations have adopted multiple DevOps tools and platforms, far too many of them still have a long DevOps journey ahead. Each organization will need to decide whether DevOps workflows need to be adopted to fit their needs, even as the pressure to build and deliver software faster continues to mount. In fact, at a time when more organizations than ever are dependent on software to drive mission-critical digital business transformation initiatives, time may be running out to adopt DevOps workflows at a deeper level if organizations hope to stay competitive.

  • Cycode Adds ASOC Module to Streamline DevSecOps Workflows

    Cycode Adds ASOC Module to Streamline DevSecOps Workflows

    Cycode has added a module to its platform for mapping metadata and events using graph technology that makes it simpler for application development and cybersecurity to consolidate alerts generated by their DevSevOps tools.

    Lotem Guy, vice president of product for Cycode, said the Cycode Application Security Orchestration and Correlation (ASOC) module will automatically discover all the tools that make up a DevSecOps workflow without requiring any integration effort.

    That capability makes it simpler to aggregate results in a way that enables development teams and cybersecurity professionals to identify the most critical issues that need to be addressed before applications are deployed in production environments, he added.

    In effect, Cycode is now providing a method for bridging the historic divide between cybersecurity professionals and application development teams. More cybersecurity professionals are embedded within DevOps teams, but in the absence of a platform for aggregating alerts, they often encounter communications issues. ASOC provides an agentless approach to aggregating the data that DevSecOps teams need to prioritize remediation efforts, said Guy.

    While a lot of progress has been made in terms of adopting DevSecOps best practices, many organizations still struggle with securing their software supply chains. Most of the members of a DevOps team have limited cybersecurity expertise, so they need the help of a cybersecurity professional to determine what specific actions are required to remediate a vulnerability. Cybersecurity professionals, conversely, don’t have a lot of application development expertise and generally would prefer not to be overwhelmed by multiple DevOps tools generating alerts about the same potential application security issue.

    As the number of regulations specifically focused on application security steadily increase, it’s now just a question of time before every organization that builds software will need to embrace DevSecOps. The challenge is that while everyone involved generally agrees software should be more secure before it’s deployed, the security tools made available to DevOps teams vary widely. In fact, it’s not uncommon for DevOps teams to find themselves spending significant amounts of time sorting through multiple conflicting alerts.

    Nor is every alert that gets generated relevant—in some cases, a specific module that generated an alert might not find its way into the final application. In addition, many of the alerts generated also tend to lack context about severity. Application developers can’t devote all their time to fixing bugs, so choices will need to be made based on the actual risk to the application.

    Cycode is betting that, as DevSecOps continues to evolve, organizations will look for tools and platforms that make it simpler to make sense of the current level of application security chaos that tends to pervade IT organizations. In the meantime, cybersecurity professionals and application developers will need to learn to trust one another. Developers have too often viewed cybersecurity as an obstacle to deployment while cybersecurity teams have historically considered developers to be one of the primary causes of the breaches they are expected to clean up. The first step toward achieving that goal is, of course, to realize that neither is really the enemy of the other.