devopsy.kinsta.cloud

Category: DevOps Practice

  • Spotify Adds More Plugins for Backstage Developer Portal

    Spotify Adds More Plugins for Backstage Developer Portal

    Spotify this week added additional plugins for its open source Backstage platform that is used to build developer portals. The new plugins make it simpler to address role-based access and access Insights, a tool from Spotify that tracks Backstage usage trends.

    In addition, Spotify is also enhancing a Soundcheck plugin for Backstage that is used to visualize and track development of software components. Forthcoming capabilities include a no-code interface that makes it possible to programmatically create checks of code without writing any code.

    DevOps teams will also be able to view, export and understand how teams and components are doing compared to established best practices and monitor trends, graphs and historical views, and receive notifications when levels change.

    Finally, Spotify is committing to adding additional integrations with third-party tools and platforms such as Snyk, Sonarqube and the open source Argo continuous delivery (CD) platform.

    There are now five plugins available via a Spotify Plugins for Backstage subscription service, and Spotify said more are planned.

    Meg Watson, a group product manager at Spotify, said Backstage has been gaining traction in cloud-native application environments that are especially challenging to develop using multiple tools. Because of Backstage, developers are not only more productive but turnover has also been reduced because developers are provided with a better experience, she noted.

    There’s a lot more focus than ever on developer productivity, which Backstage addresses by creating a catalog of blueprints that developers can readily consume rather than requiring them to build these capabilities themselves multiple times over. The goal is to create scaffolds that developers can consistently reuse across multiple application development projects.

    Backstage was donated to the Cloud Native Computing Foundation (CNCF) and is now being advanced by contributions from multiple vendors. Much of that focus is on lowering the barrier of adoption for the platform, said Watson. Specifically, Spotify is working on a QuickStart for Backstage edition of the platform that is simpler to install, she said.

    In general, Spotify is trying to strike a balance between centralizing the management of DevOps workflows and the need to enable developers to define workflows that are natural to them versus ones that have been imposed on them, added Watson.

    It’s not clear to what degree Backstage will help fuel the adoption of a shift toward platform engineering to centralize the management of DevOps tools and platforms. The concept of a portal through which developers can self-service their own needs may not be new, but as an open source project, Backstage has made it simpler for many organizations to achieve that goal.

    Just about every DevOps team is now being asked to find ways to help improve developer productivity at a time when many organizations are trying to do more with fewer resources. Regardless of the motivation, however, it’s apparent that DevOps workflows are continuing to evolve and mature.

    In the meantime, DevOps teams would be well-advised to eliminate as many bottlenecks as possible before they negatively impact developer productivity.

  • The Right Tool for the Job: AI Edition

    The Right Tool for the Job: AI Edition

    (Actually, the right job for the tool.)

    We are currently in a really weird space with regard to AI capability. For my AI friends, I will make the disclaimer that what we call AI today is not actually AI, it is more akin to LiSP “AI” than actual intelligence (and, in reality, is firmly somewhere in between). In short, we have great language processing tools that we call AI but that have no cognitive ability—but, man, are they useful.

    And the suddenness with which the rest of the world realized they were useful was nothing short of astounding. One day we are using AI algorithms to process test results and filter security data; next thing you know, our business people are bringing up AI and how advanced it is in every meeting.

    They’re not wrong, but we need to differentiate the technologies. The promise and danger of chatbots is not the same as the promise and dangers of AI in security (where it can be used by attackers as easily as defenders, btw). We’re going to have to work out the concerns and benefits in real-time, but we need to start by pushing back and making clear that it is not a panacea, any more than the last 50 technologies that came along. It has great promise to change the world, but so did VMs and Java and containers and …

    This one seems bigger, but that’s largely because it came on so fast and with seemingly broad applicability. But in the end, it is good at processing large datasets and noticing both patterns and anomalies in the data. This is a great, hugely powerful tool, but for things with sparse data? Totally useless. This is critical to remember moving forward. AI that monitors a certain aspect of security in the organization can detect patterns and anomalies, but unless the dataset is massive, there will be a lot of false positives along the way, and a more traditional tool is preferred. If that same AI (same software, algorithm, etc.) is being trained on data from hundreds of companies? Sign the check, it’s probably going to be better and more forward-looking than your current solution.

    And that’s the key. It took the entirety of the World Wide Web—including, I might add, The Gutenberg Project—to train the language processors, so unless that security AI has a massive dataset, it’s not likely ready for prime time.

    In IT, look to AI today (and this will change relatively rapidly, so check back often) for large dataset processing and for advanced automation. Those are the two real use cases. More will come, but right now, the ability to find patterns in large datasets or to say, in real-time, “A+B+C = block that port!” are the biggest viable use cases.

    Over the years, I have grown ever more skeptical of claims about removing the need for developers. Those claims have been made so many times that my friends and I have a stable of jokes about them. Java, XML, 4GLs, RoR, UI generation tools—the list goes on and on. And each one created even more code. I expect AI will be the same, but there is no denying it can offer initial frameworks, much like mobile UI generation tools do. We’ll have to see how much change that bit introduces.

    And keep rocking it. While I hope my readers don’t need reminding not to upload their source code to any of these AI tools, I will point out that your (Dev and Ops) source is so proprietary and critical that employers freak out when it is uploaded to an AI chatbot. No one cares what is done with chatbot-generated source, but the DevOps team’s source is considered critical. Sleep well knowing that.

  • GitGuardian Survey Surfaces Secrets Management Challenges

    GitGuardian Survey Surfaces Secrets Management Challenges

    A survey of 507 IT decision-makers in the U.S. and the United Kingdom published today found 75% of respondents said a secret leaked from at least one application, with 60% noting that the leak caused issues for either the company, employees or both.

    Conducted by Sapio Research on behalf of GitGuardian, a provider of a platform for securing application secrets, the survey found less than half of respondents (48%) are confident in their ability to protect application secrets to a great extent.

    The top source of risks when it comes to secrets management are source code and repositories (58%), followed by hard-coded secrets (47%). Just under a third (30%) of respondents to the GitGuardian survey admitted they share passwords and secrets when building applications as clear text within a messaging application.

    Just over a quarter (27%) still relied on manual code reviews to prevent secrets leaks, the GitGuardian survey found.

    On the plus side, 94% of the respondents said they were considering improving their secrets management practices in the coming 12-18 months, even though 93% said either “all” or “some” of their repositories were continuously scanned for hard-coded secrets.

    Thomas Segura, a cybersecurity expert for GitGuardian, said as the number of applications developed continues to increase, secrets sprawl has become a bigger challenge for application security. The way secrets are managed during application development has become a bigger issue in the wake of a series of software supply chain breaches that led to malware being injected into applications.

    The concern is cybercriminals that compromise software development environments will be able to retrieve secrets that they can later use to compromise an application after it has been deployed in a production environment. Cybercriminals are also now routinely scanning for secrets that may have been stored in text after an application has been deployed.

    The challenge is that secrets have not been historically managed effectively, so many organizations are now looking to implement more rigorous processes. Another challenge is that, in many cases, it’s not apparent whether those processes will be implemented by a DevOps team familiar with how an application is constructed or an application security team with more cybersecurity expertise, noted Segura.

    Regardless of the approach to remediating these issues, there is always going to be an issue because the humans who build applications are always going to make mistakes. The goal is to minimize the number of secrets that might be leaked by implementing DevSecOps practices that automate the review process, said Segura.

    In theory, at least, advances in artificial intelligence (AI) should make it easier to automate more of the secrets review process. In the meantime, however, organizations need to review their existing processes in advance of pending legislation that looks to hold organizations more accountable for application security if it is determined that the organization did not take reasonable precautions.

    It may be a while before such legislation becomes law, but one way or another, it’s clear that DevOps teams are soon going to be asked tougher questions about how secrets were managed every time an application security breach occurs.

  • CDF Report Reveals Lack of DevOps Maturity

    CDF Report Reveals Lack of DevOps Maturity

    A global survey of nearly 26,000 DevOps professionals published today by the Continuous Delivery Foundation (CDF) at the cdCon + GitOpsCon event found 84% of respondents are involved in some type of DevOps activity. However, the report also found just under half 49% are using a continuous integration (CI) or continuous delivery (CD) platform and only 22% are using CI and CD approaches to automate all building, testing and deployment of code in a production environment.

    The report, conducted in collaboration with SlashData, found the number of developers using CI or CD platforms only increased two percentage points year-over-year, with the use of self-hosted CI/CD tools dropping from 32% in the first quarter of 2022 to 23% in the same period this year.

    The survey also noted that only 7% of respondents this year described themselves as a DevOps engineer or specialist.

    Overall, medium-sized businesses have the highest involvement in DevOps practices (94%), followed by small businesses and large enterprises at just over 90%. More than three-quarters of developers (79%) are now using DevOps practices.

    The most common DevOps activity is monitoring software or infrastructure performance (47%), followed by application security testing and using CI to build and test code, tied at 37% each, and approving code deployments into production environments at 36%, the survey found.

    The average number of DevOps-related technologies used by developers (4.5) has, meanwhile, remained stable over the last year. The most widely used tools address issue tracking (39%), source code management (38%), test automation management (34%), collaboration and knowledge-sharing tools (32%) and application performance management/observability and agile project management tools tied at 31%.

    The survey also noted that the percentage of top performers—defined as those with lead times of less than one day—now stands at 15%. Survey results for top performers over the years continue to fluctuate between 13% and 17% of the survey total. On the plus side, 40% of developers have lead time changes of less than one week, the second-highest ranking for this survey.

    More than a third of all DevOps practitioners (34%), however, take longer than a week to restore service. Only 12% of DevOps practitioners take less than an hour to restore service.

    In general, the survey finds that organizations that use 10 or more DevOps tools are twice as likely to be high performers.

    Fatih Degirmenci, executive director for the CDF, told conference attendees the consortium, which now oversees nine projects, including the open source Jenkins CI/CD platform, continues to make progress as additional projects are added.

    However, while it’s clear that many organizations have adopted multiple DevOps tools and platforms, far too many of them still have a long DevOps journey ahead. Each organization will need to decide whether DevOps workflows need to be adopted to fit their needs, even as the pressure to build and deliver software faster continues to mount. In fact, at a time when more organizations than ever are dependent on software to drive mission-critical digital business transformation initiatives, time may be running out to adopt DevOps workflows at a deeper level if organizations hope to stay competitive.

  • GitHub Revamps Search Engine for Code Repository

    GitHub Revamps Search Engine for Code Repository

    GitHub today unveiled a new search engine it has developed to make it simpler and faster to browse and navigate the world’s largest code repository.

    A completely redesigned search interface now provides suggestions, completions and the ability to slice and dice results, while a redesigned code view tool has integrated search, browsing and code navigation.

    Mario Rodriguez, vice president of product for GitHub, said the popular code repository provider decided to create a new search engine to improve developer productivity. There are more than 200 million GitHub repositories that can now be more easily explored, he noted. The updated search engine also makes it much easier to search for specific vulnerabilities that might exist in a codebase, added Rodriguez.

    This latest addition to the GitHub portfolio is part of an ongoing effort to inject more intelligence into software development using tools such as GitHub Copilot that employ generative artificial intelligence (AI) to make developers more productive, he said.

    During uncertain economic times, there’s always going to be more focus on productivity. Organizations of all sizes are reluctant to hire additional full-time developers, so any effort to make existing developers more productive is welcome. The challenge is the DevOps teams that support those developers are now under increased pressure to reduce or eliminate as many bottlenecks as possible. In most instances, that will require increased investments in automation and AI to enable greater amounts of code moving through multiple pipelines to be managed at an unprecedented scale.

    In the meantime, instead of focusing on metrics such as recovery times, DevOps teams might want to focus more on how much they are improving developer productivity. After all, developers are an expensive asset, so in the age of digital business transformation, most business executives are squarely focused on increasing the rate at which applications are being built and deployed. In many cases, the amount of profit and revenue being generated by any business is now a direct reflection of the pace at which quality software is developed and deployed.

    One way or another, a new age of software development is dawning. Most of the innovations being brought forward are focused on issues that impact developers, but it’s only a matter of time before AI is also increasingly applied to DevOps workflows. As more organizations begin to realize how much of a bottleneck manual steps in DevOps processes have become, the willingness to invest in more advanced platforms should increase.

    The challenge, as always, is there are a lot more developers than software engineers, so the immediate focus is on developer productivity. That’s not necessarily a bad thing, but in the absence of any advances on the back end of the software development process, it is likely to aggravate an imbalance that most DevOps teams strive daily to correct.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    Northwestern Mutual
    Milwaukee, Wisconsin
    Assistant Director for Product Management for Cloud and DevOps Platforms
    $139,300 to $186,200

    CareerBuilder.com

    Brown Brothers Harriman
    New York, New York
    DevOps Operations Manager
    $130,000 to $190,000

    SimplyHired.com

    Spiff
    Salt Lake City, Utah
    DevOps Engineer
    $120,000 to $170,000

    Indeed.com

    Rose International
    Jefferson City, Missouri
    Azure DevOps Architect
    $114,000 to $145,000

    Dice.com

    ComTec Information Systems
    Remote
    DevOps Engineer
    $100,000 to $150,000

  • ReversingLabs: Increased Focus on Software Supply Chain Security

    ReversingLabs: Increased Focus on Software Supply Chain Security

    A global survey of 300 global executives, technology and security professionals found software containing vulnerabilities (82%) followed by secrets leaked through source code (55%), malicious code (52%) and suspicious code (46%) posed a serious risk to the business.

    Conducted by Dimensional Research on behalf of ReversingLabs, a provider of a platform for securing software supply chains, the survey found 87% of respondents detected significant risks in their software supply chain in the last year. Nearly three-quarters (74%) said legacy application security tools, including static application security testing (SAST) (54%), dynamic application security testing (DAST) (42%) and software composition analysis (SCA) (40%), are ineffective against threats to modern software supply chains.

    A full 88% said software supply chain security is an enterprise-wide risk, but only 60% said their software supply chain defenses were up to the task. Nearly two-thirds (65%), however, acknowledged their organization’s software supply chain security program wasn’t as mature as it should be. As a result, 80% are currently focused on improving security for the software supply chain, with 96% noting a more comprehensive approach to software supply chain security that detects more than vulnerabilities is needed.

    Overall, the survey found nearly all respondents (98%) recognized that software supply chain issues pose a significant business risk. Nearly 90% reported detecting security or other issues in their software supply chain in the last 12 months. Those issues stemmed from internally developed software (47%) as much as open source software (49%), followed by commercial software (30%), the survey finds.

    Tyson Whitten, vice president of global marketing for ReversingLabs, said the survey results make it clear that software supply chain issues extend well beyond open source software dependencies. Use of open source software has expanded greatly over the last decade, but security flaws in software are pervasive, he noted.

    In addition, many of these issues stemmed from software created by contractors and third-party companies, added Whitten. More than half of respondents worked for organizations that use contractors (67%) and third-party development companies (59%) and many organizations don’t have the tools required to review applications before they are deployed in a production environment.

    Software supply chain security has become a major issue in the wake of a series of high-profile breaches. The challenge is shortage of cybersecurity expertise available to secure those supply chains. Usually, someone on the DevOps team needs to emerge as a security champion for the good of the larger application development team. Unfortunately, most developers have had limited exposure to cybersecurity training, so it can take a significant amount of time for those champions to emerge.

    In the meantime, it’s less clear whether organizations will upgrade their DevOps environments to make them more secure, but it’s clear they will soon be held more accountable for application security. Governments around the world are currently drafting legislation that would impose penalties should it be determined an organization recklessly built and deployed software. Once that legislation is passed, it may only be a matter of time before the pressure on organizations to adopt DevSecOps workflows increases. The issue is determining how quickly to get in front of that issue before a potential legal quagmire ensues.

  • 7 Benefits of Self-Contained Apps for BPM

    7 Benefits of Self-Contained Apps for BPM

    According to Gartner, “As business needs change, organizations must be able to deliver innovation quickly and adapt applications dynamically—reassembling capabilities from inside and outside the enterprise. To do this, organizations must understand and implement the ‘composable enterprise.’”

    This is an appealing concept for companies looking for a good balance between highly centralized and highly distributed technologies and information systems.

    For enterprise IT, the use of composable infrastructure – built up of independent components that can be swapped out when needed – is already widely applied. Another place to look for the advantages of composability: Business process management  (BPM) automation. Why? Because IT teams, DevOps teams and leadership are facing a number of pressures:

    • Business teams need responsiveness and speed from IT as everyone continues to look for ways to maximize go-to-market.
    • Current business process automation platforms deploy applications that are all dependent on shared resources. The DevOps team must take interoperability and integration among a group of applications using shared resources into consideration for new developments, new deployments and maintenance. This can be complex, time consuming and can require extensive testing before updates can be deployed.
    • Similarly, whenever the process application platform is upgraded to a new version to take advantage of new features, or when maintenance on a single application is needed, all applications are affected and the need to offline and upgrade everything can impose a high cost in downtime.
    • Scaling grouped business process applications can add unnecessary cost as it is applied across the board even if not all of those applications need to be scaled.
    • The approach of building business applications from interoperable elements promises to address some of these key issues.

    Modularity and component-based systems have already been successfully applied for many years in IT and software development. Infrastructures are composed; we no longer buy machines but CPU, memory, disk space, bandwidth and so on. In software development we see composability applied through containerization, an approach that is already widely used and becoming the standard for DevOps teams, as shown for example by the growing use of platforms such as Docker and Kubernetes.

    Self-contained applications is setting the same trend for composable BPM process-based applications. BPM technologies already fit perfectly into the approach of composable architectures. Applications themselves are also already internally made up of composable elements: Processes, sub-processes, connectors, services, reusable pieces of code, business rules and UI fragments, just to name a few.

    Self-contained apps combine the benefits of containerization approaches and technologies with the power of its BPM application platform technologies for application development. The BPM base offers wide extensibility to integrate with enterprise IT, with good auditability, traceability and compliance; deployment of individual applications using containers offers faster delivery, portability and easier management.

    DevOps teams can deploy complex applications more easily, deliver continuously, and thus support innovation and faster time to market. Each self-contained app can be deployed individually, independently and quickly to deliver increased value through improvements as quickly as the business requires.

    DevOps teams, IT leadership and CIOs will be happy to see self-contained apps because:

    • The containerized approach included with self-contained apps is already familiar to CIOs and their technical teams.
    • The less complex, independent development and deployment of self-contained applications means faster application delivery to production overall. As it’s not necessary to take interoperability and integration among a group of applications into account during development, development is faster, testing is easier and the resulting application is easier to maintain.
    • Scaling an independent application is simpler and it can be applied to each application individually.
    • Application maintenance costs are reduced, as smaller and more regular updates are now possible.
    • Because it’s easier to implement multiple projects, development teams’ efficiency and productivity, and therefore ROI, increase.

    Business will be happy to see self-contained apps because:

    • Automation and business application projects are more easily adapted to business changes; the IT team can respond more quickly and with more confidence due to the smaller scope and less interdependence among other applications.
    • Faster innovation and competitive advantage offers a faster response to new business needs.
  • How LinkedIn is Investing in Developer Productivity and Satisfaction

    How LinkedIn is Investing in Developer Productivity and Satisfaction

    More and more companies are beginning to examine the role developer experience has in improving engineering productivity and morale. And one company often referenced as a model for developer productivity engineering is LinkedIn—its engineering leadership has invested heavily into reducing time-consuming developer tasks and constructing an internal platform to gauge developer satisfaction on the tools they use.

    Engineers at LinkedIn recently shared how the company has invested in measuring developer productivity and happiness as well as their strategies for automatically cherry-picking commits. I also met with LinkedIn engineering leadership for a closer look at how the organization is working to improve the developer experience across its internal platform.

    Below, we’ll take an inside look at LinkedIn Engineering’s efforts to enhance developer happiness. We’ll explore their approach to improving their internal software development trends and consider the benefits LinkedIn has gained throughout the process.

    A Look at LinkedIn Internal Developer Solutions

    Productivity is challenging to measure, said Jared Green, VP of engineering, head of developer productivity and happiness at LinkedIn. So, the company opted to take a different approach by including happiness as a variable in the equation. Thus, they measure not only how fast a workflow is but continually monitor how enjoyable the process is to ensure developers have a delightful experience.

    “We don’t just strive for productivity, but for productivity and happiness,” said Green. And to his point, on-the-job satisfaction can have real-world results—on average, happy workers are 13% more productive, according to a University of Oxford study.

    LinkedIn now oversees a custom-built internal platform for evaluating the sentiment of developer tools. The platform, known as iHub, is a holistic listening channel to gauge developer satisfaction across various tools. It collects real-time qualitative feedback from users and correlates it with objective quantitative telemetry. It then surfaces this data in a UI dashboard for team leads and individual contributors to view.

    But there’s an art to engendering the trust required to collect accurate feedback, says Max Kanat-Alexander, principal staff software engineer at LinkedIn. For example, individual contributors need to be sure the data won’t be used for performance reviews. Once that trust is established, folks are more willing to give genuine feedback, he said.

    Benefits of Investing in Developer Enablement

    The benefit of centralizing developer experience metrics with a shared platform means that tech leads can make data-backed decisions on how to improve their DevOps processes. “It’s really about thinking about giving people insights versus giving them data,” said Green. The idea is that frontline engineers can rapidly investigate any metric to expose problems and see trends. “The optics have given leaders ways to improve various parts of developer experience,” he said.

    Through their investment into developer experience, LinkedIn Engineering has discovered and streamlined hangups in the Git flow, exposed inefficiencies in their code review processes, and fixed specific developer build times. LinkedIn Engineering reported it has reduced time spent on certain time-consuming developer tasks by 99.6%.

    To construct iHub, LinkedIn Engineering wrote its own front end. The hub pre-computes data dimensions and stores them in an offline data store, increasing speed, said Kanat-Alexander. The system uses a sophisticated survey technology to collect the data and hides the complexity of manual collection in a sleek dashboard.

    LinkedIn managers have already leveraged iHub data to identify macro trends and resolve specific points of friction. For example, Kanat-Alexander describes how one director noticed that developer build time was worsening in their organization. They were able to isolate it to a single developer who wasn’t using remote development, which was spiking the metric. Leaders then assisted in unblocking them.

    Takeaways For Your Organization

    Most companies could benefit from a UI like iHub, explained Kanat-Alexander. But, you really must have a large engineering organization to see the benefits of the work. Plus, up to 80% of the legwork is simply data integrations—this involves hooking into systems, cleaning up data and deciding how to measure certain metrics. “Whatever you can measure, you can improve,” said Green.

    LinkedIn Engineering has flirted with the idea of open sourcing iHub, but it’s a bit too custom to LinkedIn to be valuable to other enterprises. “There’s a lot of variance in build technologies nowadays, and large-scale engineering systems are bespoke,” said Green. Implementing such a dashboard also requires a deep understanding of developer personas and how people work, which can be entirely unique for each organization.

    Although other organizations can’t fork LinkedIn’s efforts, there are some takeaways that could help others invest in their internal DX. First, Green recommended starting with the fundamentals. “Having a core of understanding about what you’re trying to move the needle on is important, as is coming up with ways to talk about it consistently.” This forethought should help establish what you want to measure.

    Turning DX Metrics Into Action

    Studies indicate that more companies are seeking to implement platform engineering or similar developer productivity engineering initiatives. The end result of these efforts should be to empower people and ensure they are satisfied with their work and not just a cog in the system.

    On that note, it’s critical that there are mechanisms for action to happen on the data that is collected, said Kanat-Alexander. Someone needs to monitor the insights and make changes to improve the metrics over time. “Then you can win hearts and minds.”

  • FIDO/WebAuthn Passkeys is Inevitable: Get on the Train ¦ IBM CEO Hates WFH

    FIDO/WebAuthn Passkeys is Inevitable: Get on the Train ¦ IBM CEO Hates WFH

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The Passkeys authentication standard gets a huge boost, and IBM’s Arvind Krishna wants workers back in the office. (more…)