devopsy.kinsta.cloud

Tag: code repository

  • GitHub Revamps Search Engine for Code Repository

    GitHub Revamps Search Engine for Code Repository

    GitHub today unveiled a new search engine it has developed to make it simpler and faster to browse and navigate the world’s largest code repository.

    A completely redesigned search interface now provides suggestions, completions and the ability to slice and dice results, while a redesigned code view tool has integrated search, browsing and code navigation.

    Mario Rodriguez, vice president of product for GitHub, said the popular code repository provider decided to create a new search engine to improve developer productivity. There are more than 200 million GitHub repositories that can now be more easily explored, he noted. The updated search engine also makes it much easier to search for specific vulnerabilities that might exist in a codebase, added Rodriguez.

    This latest addition to the GitHub portfolio is part of an ongoing effort to inject more intelligence into software development using tools such as GitHub Copilot that employ generative artificial intelligence (AI) to make developers more productive, he said.

    During uncertain economic times, there’s always going to be more focus on productivity. Organizations of all sizes are reluctant to hire additional full-time developers, so any effort to make existing developers more productive is welcome. The challenge is the DevOps teams that support those developers are now under increased pressure to reduce or eliminate as many bottlenecks as possible. In most instances, that will require increased investments in automation and AI to enable greater amounts of code moving through multiple pipelines to be managed at an unprecedented scale.

    In the meantime, instead of focusing on metrics such as recovery times, DevOps teams might want to focus more on how much they are improving developer productivity. After all, developers are an expensive asset, so in the age of digital business transformation, most business executives are squarely focused on increasing the rate at which applications are being built and deployed. In many cases, the amount of profit and revenue being generated by any business is now a direct reflection of the pace at which quality software is developed and deployed.

    One way or another, a new age of software development is dawning. Most of the innovations being brought forward are focused on issues that impact developers, but it’s only a matter of time before AI is also increasingly applied to DevOps workflows. As more organizations begin to realize how much of a bottleneck manual steps in DevOps processes have become, the willingness to invest in more advanced platforms should increase.

    The challenge, as always, is there are a lot more developers than software engineers, so the immediate focus is on developer productivity. That’s not necessarily a bad thing, but in the absence of any advances on the back end of the software development process, it is likely to aggravate an imbalance that most DevOps teams strive daily to correct.

  • npm is Scam-Spam Cesspool ¦ Google in Microsoft Antitrust Thrust

    npm is Scam-Spam Cesspool ¦ Google in Microsoft Antitrust Thrust

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The npm registry suffers spam infestation, and Microsoft makes Google sad. (more…)

  • ActiveState Makes All Tiers of Curated Artifact Repository Service Free

    ActiveState Makes All Tiers of Curated Artifact Repository Service Free

    ActiveState today announced it is making all tiers of its ActiveState Artifact Repository service available for free for a limited time. The move aims to enable organizations to better secure open source software components incorporated within applications.

    Loreli Cadapan, vice president of product for ActiveState, said the ActiveState Artifact Repository exposes a set of curated instances of open source software for creating binaries that are validated to be secure. That capability reduces the likelihood vulnerabilities will be inadvertently introduced into application environments by developers that have downloaded open source modules from repositories that may contain compromised code, she added.

    In addition, the ActiveState Artifact Repository also makes it possible to automatically generate software bills of materials (SBOMs) that identify which software components were used to build an application after all the code used to construct an application is stored in the repository, Cadapan noted.

    ActiveState’s repository is delivered as a cloud service to organizations that would otherwise have to build their own. Cadapan noted that the goal is to make managing and securing software supply chains simpler when more attacks are being launched against them.

    Most organizations have limited visibility into how they are consuming open source software. It’s become apparent many organizations have no idea what software components have been incorporated across their software supply chain. As a result, when a vulnerability is discovered, there’s no easy way for them to know whether they are impacted.

    However, awareness of this issue has risen, thanks to an executive order issued by the Biden administration which requires federal agencies to have an SBOM for every application they employ by next summer. The executive order was issued in the wake of a zero-day vulnerability in the Log4j tool discovered late last year that is widely used to create logs within Java applications. Many organizations are still looking for all the vulnerable instances of Log4j in their application environments. SBOMs must provide a list of the “ingredients” used to create an application to make it easier to find any component that might one day be similarly affected by a zero-day vulnerability.

    Organizations that want to implement a similar SBOM policy will spend massive amounts of time cataloging software on an ongoing basis. Every time an application is updated, there will be a need to once again verify what components are being employed within the context of larger DevSecOps workflows.

    SBOMs will undoubtedly play a critical role in improving overall application security. However, having an SBOM is not the same thing as operationalizing it. Organizations are ultimately hoping to be able to use an SBOM to decide whether to green-light the deployment of an application, so DevOps teams should expect to be required to not only provide an SBOM but also continuously update it. The challenge, of course, will be determining how many internal resources to devote to that effort versus relying on external platforms to manage that process.

  • GitHub Adds Tools to Simplify Management of Software Development

    GitHub Adds Tools to Simplify Management of Software Development

    GitHub has made generally available Projects powered by GitHub Issues, a set of tools for managing application development projects based on a spreadsheet interface.

    Mario Rodriguez, vice president of product management at GitHub, said these tools enable DevOps teams to more easily filter, sort and group issues and pull requests. That capability makes it possible to, for example, configure and track cycle velocity and current work status and create visualizations using cumulative flow diagrams.

    Over the course of the rest of this year, Rodriguez said GitHub also plans to add additional capabilities including custom triggers, conditionals and action logic along with a mobile edition of Projects powered by GitHub Issues.

    The overall goal is to provide DevOps teams with a set of project management tools that enable organizations to plan where they code, said Rodriguez. In many cases, that approach eliminates the need to deploy a separate set of project management applications, he added.

    As more organizations become dependent on software to drive digital business processes, it’s become apparent that managing all the ongoing application development projects has become a significant challenge. GitHub has been developing Projects powered by GitHub Issues to provide organizations with a set of tools that can be accessed not just by DevOps teams but also by application owners and business leaders to track the status of various projects, said Rodriguez.

    Most organizations have not reached the point where revenue growth is completely dependent on the rate at which software is developed and deployed, he noted. However, as DevOps teams more regularly engage other departments in the business, Projects powered by GitHub Issues can provide anyone with access to a spreadsheet-like application that is simple to use, said Rodriguez.

    In general, GitHub is committed to improving developer experiences at a time when organizations are competing for the best talent available, added Rodriguez. If salary and benefits are equal, the decision to work for one company versus another comes down to the quality of the application development experience, he noted.

    As the provider of the most widely-used code repository, GitHub has been extending the reach and capabilities of its platform to address a range of DevOps workflow issues. The issue that DevOps teams need to address is how to anchor those workflows to a code repository versus the continuous integration/continuous delivery (CI/CD) platforms they employ to build and deploy applications.

    Regardless of approach, there’s no doubt that DevOps teams can now construct more scalable workflows. That’s critical, because the number of concurrent application development initiatives that organizations are managing simultaneously has increased substantially in recent years. The underlying platforms used to manage those projects need to be extensible to prevent workflows from becoming overly rigid, said Rodriguez.

    It’s not clear to what degree business leaders appreciate the value of DevOps workflows, but as business processes continue to evolve, the ability to effectively compete could rest on the agility of an application development team.

  • The Risks of Shadow Code

    The Risks of Shadow Code

    As the economy struggles to recover after the last two years of the COVID-19 pandemic, we have all learned a thing or two about supply chains—and what happens when they break down. But many people do not realize that modern websites also have digital supply chains, third-party ones that can be compromised in myriad ways. And just like the disruption of the physical supply chain affects things like pricing, availability and inventory, digital supply chains found on today’s websites play a big role in critical areas like site performance, customer experience and overall analytics. As a result, businesses today need to be aware of how the digital supply chain can introduce significant risk, especially when third parties do not adhere to their own set of stringent standards around security and compliance.

    According to a recent study by Source Defense, organizations that collect and use sensitive data—such as health care, finance, travel and online retailers—need to strengthen their focus on third-party risk management. As a result of mandates like GDPR and PCS DSS, these companies are now bound by compliance requirements to ensure their client-side threats are avoided and/or mitigated. This effort means recognizing more than just how to protect private data—it means understanding the properties of malicious code known as “shadow code” on modern websites and how it generates risk from dynamic and unpredictable scripts. Companies dealing in data today need to be aware of the different ways shadow code can affect their security, as well as the productivity and integrity of their DevOps teams.

    Third-Party Scripts and Shadow Code

    Shadow code refers to code that is baked into an application without proper vetting by the website’s IT department. This malicious code thrives in the cloud landscape, primarily because it allows for ease of deployment through CI/CD automation.

    When a web page calls in a third-party script, it loads directly into the browser from a remote server, which means general security controls such as firewalls and network monitoring tools are bypassed. And if one of these scripts has been compromised by a bad actor, shadow code can be embedded within. Sometimes these scripts include open source software that hasn’t been tested well—or at all. Other times, the code may be pulled from another layer of the organization where malicious code is hiding.

    Shadow code presents an array of threat opportunities, including the ability to mount an attack through digital supply chain partners. When these compromised scripts are used to steal data, redirect web traffic,or any other malicious activity, a clear path is essentially created into thousands of different websites, in almost every industry that collects data or deals in digital transactions. When a third-party script with shadow code enters a system, it gains the ability to:

    • Record keystrokes
    • Redirect visitors to malicious websites
    • Monitor clicks and track website activity
    • Change content on websites, including images and text
    • Steal credentials, such as personal logins and social security numbers

    As we know, these digital activities can lead to big consequences, like data breaches, fraud, theft, compliance violations, loss of reputation and hefty fines. Although code reviews can provide some software quality assurance, the process is not effective against shadow code, mostly because these scripts are constantly altered by third-parties. And when the threat is underestimated and unattended, these malicious scripts have the potential to permeate every area of a company’s web property.

    Because the benefits of using these types of third-party scripts are considerable, it can be hard to find a modern enterprise or agency that does not rely on at least a few to run web applications. In fact, many enterprises and government agencies today are running third-party scripts on every page of their sites. Even though these organizations are technically not responsible for this harmful code, regulators can (and do) hold web property owners accountable for any digital maleficence on their sites.

    DevOps Teams and Shadow Code

    Websites can provide rich experiences to visitors largely because developers use client-side JavaScript to improve the look and feel of these digital properties—all without the need for ongoing maintenance. Today, DevOps professionals can access vast libraries of these free and low-cost scripts from many different sources, such as allied coder groups, open source organizations, social media companies, web analytics firms, content delivery networks and more.

    But the reality is, code libraries accessed by DevOps teams are not always secure or trustworthy. Considering these packages often contain code from fourth and fifth parties, DevOps folks face the unavoidable possibility of tainted code—hard to identify and dangerous to use. In fact, the Source Defense report revealed that one in four scripts represented fourth- or fifth-party code, on average. Because developers tend to focus on keeping their code updated and as capable as possible, they often pull from the dynamic library from their repository. If there is a CI/CD process happening, DevOps teams will pull their library from that point. This is where shadow code introduces new vulnerabilities for developers.

    Shadow code also jeopardizes the integrity of DevOps work. Most developers take pride in what they do and are not interested in creating unsecure or low-quality code—not purposefully, anyway. And given that speed, productivity and constant innovation are all business goals for DevOps teams, it’s important to recognize how problematic shadow code can be for those in the business of building secure and high-quality software.

  • Tabnine, GitLab Build Custom Apps Faster With AI

    Tabnine, GitLab Build Custom Apps Faster With AI

    Tabnine today revealed it has integrated its code completion tool with the GitLab continuous integration/continuous delivery (CI/CD) platform. Tabnine’s platform employs artificial intelligence (AI) to accelerate application development.

    Brandon Jung, vice president of ecosystems and business development at Tabnine, said the goal is to make it easier for developers to automatically write code based on custom models created using AI. Those models would reside in the CI/CD platform provided by GitLab. Developers can now create custom models of their private source code, he noted.

    The Tabnine AI assistant can then be trained on a customer’s private code, style and security best practices that have been defined by an internal DevOps team, added Jung. That assistant, accessible from within a developer’s integrated development environment (IDE), is conversant in 35 programming languages.

    The code completion tool doesn’t eliminate the need for developers to write code; instead, it completes code once developers start writing it based on the model stored in the CI/CD platform.

    Nima Badiey, vice president of global alliances for GitLab, said the overall goal is to increase developer productivity at a time when organizations are launching more application development initiatives than ever. Many of those applications are being constructed using multiple programming languages that each individual developer may not have equally mastered.

    In addition, an AI model will, over time, improve the quality of the code being created, noted Badiey. That should make it easier to review code that is more accurate because it was created with the aid of a machine, he added.

    In general, GitLab has been moving to incorporate AI within DevOps workflows. The company last year acquired UnReview, a provider of a tool that uses machine learning algorithms to identify which code reviewers to assign to a project based on both the quality of their previous efforts and their current workloads.

    It’s not clear to what degree AI will be pervasively embedded within DevOps workflows. However, it’s now just a matter of time before algorithms are more widely employed. Organizations that employ DevOps today devote significant resources to maintaining platforms that could be simpler to manage if more tasks and processes were automated. In theory, more resources could be then devoted to building applications versus maintaining DevOps platforms.

    One way or another, more organizations are moving to increase developer productivity. Developers are both hard to find and retain, so organizations have a vested interest in finding ways to increase the rate at which an existing pool of developers can build applications capable of driving a wider range of digital business transformation initiatives. Of course, an increase in the rate at which applications are developed also has significant implications for the rate at which they might need to be deployed.

    Not everyone in IT is, of course, entirely comfortable with AI. Nevertheless, most DevOps organizations are highly committed to automating as many processes as possible. The only thing that will be different going forward is determining to what degree those processes will be automated by humans versus a machine.

  • Source Code Management Best Practices

    Source Code Management Best Practices

    Source code is, of course, the foundation of any software development practice. Managing that source code is the first task tackled by modern software development pipelines, with all subsequent stages of the pipeline dependent on the source code for their success and functionality. Thus, it is critical that source code is properly managed without introducing bottlenecks or defects into the delivery pipeline. To do so, there are some best practices to follow when interacting with source code.

    Regularly Commit Changes

    While this may seem obvious, commits often are done sparingly, as most developers tend to commit only after a significant change. The commit frequency depends on the workflow and it is acceptable to commit only after major modifications if that commit triggers another task.

    However, it is better to commit changes—even minor ones—regularly. It not only keeps an auditable history of changes but also enables easy comparisons between commits for troubleshooting and ensures that the code is up to date with the version control system (VCS). However, it does not mean that users should make commits for every small change; instead, adapt a development style that includes frequent commits as part of the workflow. Even if there are many commits, most VSCs provides users with the option to combine multiple commits to a single commit using a rebase for better management.

    Always Ensure That the Repository is Up to Date

    Most of the time, development is a team effort with multiple developers working on a single codebase. Thus, there will be frequent updates to the source code. So it is always advisable to use git pull or fetch to obtain an updated copy of the code before making any changes. This practice not only mitigates merge conflicts but also helps users to avoid errors while pushing the code, such as ‘failed to push some refs to‘ errors which occur when the local copy of the repository is behind the remote.

    Create Detailed Commit Messages

    Commit messages should be meaningful and include the ability for anyone reading the message to grasp the reason behind the commit quickly. It does not mean that users should create lengthy commit messages; instead, they should describe the commit in a concise yet meaningful manner. The commit message should include why the code was committed and not how it can be identified by comparing the files.

    A good commit message can include details such as bug fix ID or requirement ID, which reference the specifications of that code change in project management platforms like Jira and Azure Boards.

    Review Changes Before Committing

    Always make sure to review the code before making commits—this is imperative if you are committing to a shared repository. Typically, code reviews are a part of the delivery pipeline before merging code with the master branch or a specific repository via a review system or a pull request.

    Code reviews put a second set of eyes on code modifications and help detect any issues in the code. It also acts as a quality control mechanism that helps organically improve the overall code quality while helping to increase the awareness of each code change.

    Implement a Proper Workflow

    A proper workflow with standardized practices must be implemented across the development team even while using a source code management (SCM) tool. Some considerations for a standardized workflow:

    • Decide on a branching strategy and branching best practices like naming conventions for specific development types. For example, using a prefix to indicate the type of development feature or bugfix
    • Ensure proper use of scopes to decide who has access to each repository
    • Make sure the review workflow includes which users have the final say in the review process and proper authorization to merge the changes.

    A properly defined workflow describes the complete source code management cycle from creating new branches to making changes and, finally, merging the code to the master branch after review. This process helps organizations simplify the overall code management by pushing users to follow a specific workflow. This can also help easily onboard new developers.

    Implement Proper Security Practices

    This practice goes hand-in-hand with proper workflows, as security will be one of the primary considerations when defining the workflow. It should start from developer authentication and authorization, which controls who has access and what areas of the source code they have access to. Additionally, security practices should provide visibility into all the user activities with auditable trails to pinpoint any unusual activity in the source code.

    Security practices coupled with a good backup and disaster recovery strategy are vital for the continuous operation of the development pipeline.

    These best practices can help developers manage their source code securely and efficiently. A properly implemented source code management system will lead to fewer issues in the preceding stages of the pipeline and ensure smoother operation in the long run.

  • Pulumi Adds Registry to Share Secure IaC Code

    Pulumi Adds Registry to Share Secure IaC Code

    Pulumi this week added a public registry to its portfolio that makes it easier to discover packages that have been created using its tools to manage infrastructure-as-code (IaC).

    Joe Duffy, Pulumi CEO, said the Pulumi Registry will make it simpler for DevOps teams to share and reuse IaC tools that have been vetted versus always requiring each development team to write new code each time they want to programmatically provision infrastructure. Pulumi Packages provide modern cloud reference architectures in the form of software development kits (SDKs), code samples and how-to guides.

    One of the primary reasons that security has become a major cloud issue is that developers often misconfigure infrastructure. The registry provides a way to reduce those configuration errors by making it easier for developers to reuse code written in TypeScript/JavaScript, Python, Go and .NET that they have stored in repositories such as npm.

    In addition to the major cloud service providers, there are also Pulumi Provider Packages for Auth0, CloudFlare, Confluent Cloud, Datadog, DigitalOcean, Docker, GitHub, Kong, MinIO, MongoDB Atlas, PagerDuty, Snowflake, Spot by NetApp and others.

    There are also Pulumi Component Packages for deploying container applications on instances of Kubernetes and other related platforms in addition to deploying applications on serverless computing frameworks.

    Duffy said in addition to providing a “golden image” of their configurations the registry reduces the overall level of friction that often exists between developers, IT operations teams and cybersecurity professionals.

    The latest Pulumi offering is part of an ongoing effort to democratize cloud computing in a way that is repeatable and reliable, noted Duffy. Pulumi, for example, just added support for a set of AWS Cloud Control application programming interfaces (APIs) that reduce the number of APIs that developers would otherwise have to master. The more APIs there are, the more likely it is a developer will make a mistake.

    IT teams should expect that other cloud service providers will similarly streamline the APIs they present developers as part of an effort to make their services both more accessible and secure. In the meantime, frameworks provided by Pulumi provide a means to invoke those lower-level APIs at a higher level of abstraction for both developers and internal IT operations teams alike.

    It’s not clear today how many developers routinely employ IaC tools, but as the number of workloads deployed in the cloud steadily increases, so too does the percentage of developers that employ these tools. While that shift represents a major boon in productivity for developers, it brings with it additional security risks that many organizations fail to appreciate until it’s too late.

    The only way to really solve that issue is for organizations to adopt DevSecOps best practices that require a level of collaboration between DevOps and security teams that is still relatively rare. However, as the tooling provided continues to improve, the technical challenges associated with achieving that goal continue to decline. The next big issue, as always, is bringing together all the diverse cultures within an IT organization to actually implement those tools and define a set of best practices that work for all concerned.

  • GitLab Responds To GitHub Making Teams Free – We All Win

    GitLab Responds To GitHub Making Teams Free – We All Win

    Microsoft’s GitHub recently announced that it was going to make a change to its product lineup by making the team version of GitHub free and lowering the price on a full-featured version. The move seemed aimed squarely at matching GitLab’s offering. Well, now GitLab has responded to GitHub.

    While both of these excellent offerings compete to bring the best product to market, the real winners here are all of us. True market dynamics are at play resulting in both GitLab and GitHub lowering prices and adding more features (and not just GitHub and GitLab but also their competitors, including Atlassian among others).

    I had a chance to sit down with Sid Sijbrandij to discuss GitHub’s move, what GitLab will do and what this means for the market. (BTW, I have to apologize to Sid—it seems I have been mispronouncing his name for years and he is too much of a gentleman to correct me. The proper pronunciation of his name is “See Brandy.”) The video of our conversation appears below, and a blog on the GitLab site explains it all.

    In our interview, Sid was very clear that GitLab owes a huge amount of gratitude to GitHub, as it was GitHub that spurred on the early development of GitLab. He truly respects what GitHub has done, as well as the great job Microsoft has done since acquiring GitHub.

    The GitLab blog post lays out very clearly what the GitHub offering is and how it compares to the GitLab offering. It’s no coincidence that the new GitHub paid teams offering is the same price as the GitLab bronze offering.

    Price aside, according to Sid, the real advantage of GitLab is that its vision of an end-to-end total DevOps solution and the features it adds to achieve this vision give GitLab customers more than they can get from GitHub. While I agree that is true today, I don’t think GitHub is going to be content with its current lineup for long. I think the Microsoft entity already envisions a total DevOps solution as being part of GitHub’s future.

    On top of that, I think GitHub and Microsoft are already thinking of how they can leverage Azure to give GitHub an advantage. To Microsoft’s credit, though, at least until now there has not been any special advantage of using GitHub with Azure. Azure remains an open platform that all vendors seem to get fair access to and support from.

    All of this aside though, we are all the real winners here. What a great time to be a developer/DevOps professional. While GitLab and GitHub compete with each other to bring more features to market for less money, as I mentioned other companies are competing to bring end-to-end DevOps solutions to market as well. We benefit from all of this competition by getting better tools and solutions at better prices. That’s true market dynamics at work.

    It should get only better in the months and years ahead as well. In spite of COVID19 and the rest, it is a great time to be alive.

    — Alan Shimel