Tag: GIT

  • Jellyfish Adds Tool to Visualize Software Development Workflows

    Jellyfish Adds Tool to Visualize Software Development Workflows

    Jellyfish today unveiled a tool that identifies bottlenecks in software engineering processes using data the company collects via Git repositories and project management software from Atlassian.

    Krishna Kannan, head of product for Jellyfish, said Life Cycle Explorer provides managers with a tool that makes it simpler to identify the root cause of a process breakdown.

    Life Cycle Explorer goes beyond providing simple top-level metrics to calculate the time consumed at each stage of a workflow for any given process and then visually map it to surface trends, noted Kannan.

    That makes it possible to identify gaps, see where work areas overlap—indicating parallel processes—and drill into how outlier events may be impacting the overall life cycle of a development project.

    The launch of Life Cycle Explorer comes at a time when there is more focus on the efficiency of DevOps teams than at any time in recent memory. During uncertain economic times, organizations of all sizes are looking to optimize DevOps workflows to ensure maximum efficiency because many organizations are hesitant to hire additional full-time engineers.

    The important thing to remember when analyzing DevOps workflows is there is no such thing as the optimal “Goldilocks” workflow that can be universally applied, noted Kannan. Instead, most DevOps leaders will discover one team excels at one segment of a DevOps workflow while others are more efficient elsewhere. Life Cycle Explorer makes it easier to identify those best practices in a way that makes them simpler to replicate when appropriate, said Kannan.

    Life Cycle Explorer follows the addition of a benchmarking tool and DevFinOps offering that addresses cost capitalization reporting for research and development. The core Jellyfish engineering management platform is designed to ingest data from the tools an organization already uses to build applications and then applies proprietary algorithms to create a comprehensive view of workflows across an organization. The challenge many organizations encounter today is they often lack visibility into workflows that have each been individually constructed by a diverse range of DevOps teams working on very different types of software projects.

    At the very least, DevOps teams need to establish a baseline that surfaces the current rate at which software development occurs. Once that baseline is established it then becomes possible to measure the impact changes to software engineering practices are having. Today many organizations make educated guesses about what changes are needed without really understanding the impact they might be having across the entire software development life cycle.

    Of course, it will be up to each organization to determine the degree to which they want to measure workflows. The Jellyfish platform makes it simpler to identify workflow issues, but there are also value stream management (VSM) platforms that track the impact software engineering workflows are having on specific business goals. Those VSM platforms are typically much more challenging to implement and maintain. The Jellyfish approach focuses more narrowly on improving the efficiency of DevOps teams.

    Regardless of approach, things that are not measured are not being managed. At a time when organizations are more dependent on software than ever, the need to manage software development life cycle as efficiently as possible has become a lot more important.

  • GitBOM Tool Automatically Identifies Software Artifact Components

    GitBOM Tool Automatically Identifies Software Artifact Components

    An open source GitBOM tool, discussed at the Open Source Summit Europe conference this week, can automatically track every source code file incorporated into each built artifact.

    Nell Shamrell-Harrington, a principal software engineer for Microsoft, told conference attendees via a video link that the GitBOM tool, based on a compact Artifact Dependency Graph (ADG) technology, can verify which artifacts are being used across any programming languages, environments and packaging formats without any effort made by developers. To do this, GitBOM repurposes the directed acyclic graph made available in Git version control software.

    All that is required is for GitBOM identifier—a unique, content-addressable reference—to be inserted into the artifact at build time. That identifier then creates a Gitoid that can be consumed by ADG, Shamrell-Harrington said. Each Gitoid then views an artifact as a binary large object (BLOB) through which of all the components that went into it, including source files, dependencies and object files can be identified.

    Support is also included for the Software Package Data Exchange 2.3 format for sharing software bill of material (SBOM) information discovered by GitBOM.

    GitBOM can also detect potential vulnerabilities at runtime, regardless of the depth, in every software artifact that might be impacted to make it simpler for IT teams to remediate issues. That approach makes it simpler to find, for example, every instance of the Log4j vulnerability that might exist in an application environment. GitBOM only includes the minimum information required to create a fingerprint to enable efficient runtime scanning for a known-vulnerable artifact.

    The goal is not to replace SBOM tools but rather to make it easier for them to collect data about artifacts in the most frictionless way possible for developers, Shamrell-Harrington said. GitBOM enables any metadata to be linked to a specific set of corresponding software artifacts to make it easier to discover exactly how they were constructed. Artifact IDs can then be extended to produce GitBOM documents that can be consumed by other build tools.

    GitBOM is being shared at a time when the focus on securing software supply chains has never been higher. In the wake of the Log4j vulnerability disclosure, the Biden administration is requiring every federal agency to construct SBOMs to enable developers to find instances of vulnerable software components. Most large enterprise organizations are generally following suit.

    However, collecting SBOMs is only the first part of a bigger challenge. Organizations will then need to acquire tools to analyze SBOMs and provide feedback to application developers. In effect, eventually, organizations will be able to accept or reject software based on the components being used to create it. Armed with those insights, developers then have the option to upgrade software components to bring applications in line with any policy an organization defines.

    It may be a while before SBOMs proliferate across IT organizations, but the immediate issue at hand is simply finding the least-intrusive way possible to create them.

  • The Secret to Secrets Management: Reduce Source Code Vulnerabilities

    The Secret to Secrets Management: Reduce Source Code Vulnerabilities

    We are in the cyberattack age. From Microsoft to Crypto.com, no company is completely safe from malicious attackers, and it seems as if a new major cyberattack is announced weekly, if not more frequently. As more companies not only go digital but expand their digital offerings and initiatives, there are more opportunities for cyberattackers. Security teams within IT organizations are hard at work to ensure that their company isn’t next in the headlines. However, as the tech world moves to everything-as-a-service (IaaS, PaaS, SaaS, etc.), the move toward everything-as-code is happening in tandem. As a result, many developers are unknowingly keeping sensitive data (secrets, passwords, certs, keys, etc.) stored in Git source code repositories. This information is often pushed to production which puts sensitive information at greater risk of exposure.

    The good news is there is a way to ensure that secrets are locked tight within source code. Let’s explore the complexity and realities of securing source code today and how to take the guesswork out of finding at-risk code that could land your company on the front page of the news tomorrow.

    Secrets Management is Challenging and Costly

    Sophisticated cyberattackers will stop at nothing to breach valuable data assets. Secrets leaked from source code make it easy for threat actors to access applications using the same credentials intended for legitimate purposes.

    At the pace of continuous software development, developers must move quickly to build applications. However, security teams that are expected to prevent breaches are not often well-versed in all facets of the software development life cycle (SDLC). This becomes an issue as the number of lines of code (and the secrets kept within) rise alongside the number of apps developed. According to a recent report from 1Password, “two in three (65%) IT and DevOps employees estimated their company has more than 500 secrets.” And, according to the same report, “the average cost of a secret’s leak is $1.2 million. Poor secrets management can result in organizations losing $8.5 billion annually.” Secrets are vital to building any connected application or process. If hard-coded or left unprotected in source code, the proverbial keys to the kingdom are at risk of exposure—as are your loyal customers.

    It’s clear that source code security should be a priority for security teams and DevSecOps professionals. Let’s examine how they can achieve this without sacrificing developer productivity.

    How to Reduce Threats Through Secrets Management Best Practices

    Manually searching source code repositories for vulnerable secrets can be almost impossible. However, there are some best practices to follow to ensure vulnerabilities are caught early and often:

    ● Identify and remove secrets in code: You can’t protect what you can’t see. It’s important to have visibility into source code to be able to see where vulnerabilities may exist. There are a few ways of potentially going about shining a light on vulnerabilities. One is through manual scanning of source code, but this takes away valuable time developers could be spending on more innovative tasks. There are also some open source solutions that can help with scanning source code repositories, but as with everything open source, this means developers will have to maintain it (keep it up to date, integrate it, etc.). Instead, security teams should consider looking for a unified toolchain platform to help with source code vulnerability identification and remediation.
    ● Alert and remediate in code repositories: Provide a fast feedback loop to a developer or DevOps engineer before releasing code into production. Remediation should happen at the time of commit. The idea here is to block secrets from being committed to your source code repositories in the first place.
    ● Adopt a secrets management tool: Don’t store secrets or binaries within source code—nothing but code should be in your source code repositories. Remove secrets to a secrets manager (i.e. HashiCorp Vault), keys and certificates manager (i.e. Azure Key Vault), etc. Modularize and isolate these managers, but make sure they still communicate with each other so that they don’t expose secrets.

    As cyberattacks continue to increase, it is imperative that security teams take a hard look at their source code to ensure that it is locked up tight. Keeping secrets out of source code entirely is recommended in theory but difficult to ensure in practice. Following the above best practices will help reduce the risk of exposure of sensitive data and reduce the risk of losing loyal customers and revenue dollars.

  • Git for Mainframe: Back to the Future 

    Git for Mainframe: Back to the Future 

    Software delivery modernization should be done with care, especially with mission-critical mainframe apps. But for some, modernization will feel like a trip back to the future.

    Mainframe applications are typically systems of record with high-value transactions, significant compliance risk and an essential need for stability and security. They’ve been running reliably, often for decades, with code changes made by teams of dedicated developers using time-tested interfaces like ISPF (green screens) and Eclipse integrated with mainframe-based source management tools—Endevor is the most widely used example.

    So Why Modernize?

    Three primary drivers of change are present:

    • Business Agility: Digital disruption is generating an increased volume and velocity of business demands often resulting in cross-platform application architectures. It’s increasingly common to have cloud/mobile customer-facing apps calling back-end mainframe services for seamless experiences.
    • Staffing: A talent crunch resulting from career mainframers retiring while traditional mainframe practices and tools lack appeal for the next generation. Backfilling is a challenge as next-gens would rather be using popular, cloud-native tools that offer broader career potential.
    • Performance: DevOps adoption has raised the bar in terms of productivity, delivery velocity, quality, lead times, etc. The opportunity to achieve comparable performance levels with mainframe development is entirely possible but requires modernization.

    The confluence of these drivers leads to an obvious conclusion—the time for modernizing mainframe application development is now. Version control is the foundation of software development, acting as the focal point with IDEs/code editors, Agile tools, DevOps automation tools (like code scanners and CI/CD orchestrators) and service management tools. This article focuses specifically on the adoption of Git as a critical enabler of modernization.

    The Role of Git

    The next generation of talent, typically millenials and Gen Zs, know and love Git. With 94% of professional developers viewing Git as a fundamental development tool per Stack Overflow, it has become the de facto standard for enterprise IT, eclipsing all alternatives. Built initially for open source development of Linux, it offers a decentralized model where devs can work independently in branches yet collaborate closely through pull requests, issue management, etc.

    It has become so dominant that popular editors like VS Code have built-in Git capabilities. With the introduction of the free Code4z extension pack by Broadcom, VS Code and compatible tools (e.g., cloud-hosted GitHub Codespaces) are now viable options for mainframe developers, making Git adoption a natural, ‘parallel’ modernization opportunity. The Developer Cockpit Simulator illustrates the Git-ready experience using VS Code for mainframe application development.

    Back to the Future

    Beyond version control, leading Git platforms like GitHub and GitLab have expanded to become comprehensive DevOps platforms (e.g., GitHub Issues, Actions, Codespaces, Projects, etc.).  Yet the precedent for this evolution was set decades ago when Endevor was envisioned as an environment for development and operations with SCM at its core. Yes, DevOps before ‘DevOps’ ever existed. For many mainframers, seeing the Git version control tools evolve into comprehensive platforms comes with a distinct feeling of déja vu.

    With automation that has stood the test of time, Endevor delivers code changes reliably and, with workflows optimized for the platform, does so very efficiently and securely. And while Endevor is continuously updated with new DevOps capabilities—including the recently introduced Dynamic Environments to facilitate developer self-service—there is no denying that Git is valuable from an enterprise perspective. In addition to attracting next-gens and facilitating onboarding, a Git experience empowers any developer, if authorized, to easily update mainframe apps. All they need is a few mainframe basics and experience with the relevant languages (e.g., COBOL, PL/I).

    Back when the mainframe was the sole enterprise platform, a single, common experience was the norm, but IT evolved along many dimensions resulting in a sometimes disparate and siloed environment. Today, however, most developers are comfortable using multiple languages in a hybrid environment, so normalizing around a modern developer experience makes sense and brings things full circle.

    Git-Native Approach

    One option to take advantage of Git is to move source code to a Git enterprise server, but the inherent costs and risks associated with this approach should be carefully considered. Career mainframers are some of the most productive developers around and moving to a new paradigm will naturally be disruptive. In addition to changing workflows, this approach reestablishes build and deployment automation that has been working reliably for decades, which is both time-consuming and risky. Mainframe applications are often monolithic and complex with career mainframers’ expertise often filling gaps in documentation, so preserving this institutional knowledge should be a priority.

    Career mainframers can, of course, learn Git, but training takes time. And as many are nearing the end of their careers, will they be motivated to embrace the change? Repo migration may have the unintended consequence of accelerating the retirement of a critical knowledge base. As Deloitte forewarns in their Modernization and the Aging Workforce paper, “Don’t underestimate the value of legacy coders.”

    There are also costs associated with the migration—testing, documentation, training, project management—so the opportunity cost of porting code rather than investing in other high-priority business initiatives further qualifies this option.

    With these factors taken into consideration, migration should start with applications that are both stable (e.g., infrequent change requests) and pose a low risk to the business (e.g., lower dependence on career mainframers).

    A Git-native approach would require a specific tool such as Team Build, for example; a lightweight processor-based build engine that enables teams to collaborate natively using enterprise Git repositories while building on the z/OS platform. It is independent of Endevor and can be used for any build-and-ship needs for z/OS. It converts build logic in JCL and makes it maintainable off-platform with JavaScript.

    Hybrid Git Experience

    This ‘risk-managed’ option offers the benefits of Git—easier recruiting and onboarding, increased collaboration, etc.—while eliminating many of the costs and risks outlined above. With it, career mainframers continue working as they always have with existing tools and practices, without disruption, while next-gens use the tool they prefer, Git.

    Endevor Bridge for Git enables the Git experience with Endevor continuing to serve as the main repository. It accomplishes this by synchronizing a ‘source control mirror’ on the Git enterprise server. With this option, those who prefer to use Git can do so while colleagues using native Endevor can work on the same codebase at the same time. They can use Eclipse, VS Code or VS Code-compatible cloud-based IDEs, which offer the added benefit of working in Git without a local copy of the code.

    And with enablers like the Zowe CLI, modernizing-in-place offers the best of both worlds: Mainframe and off-platform tooling. For example, run Jenkins CI/CD pipelines for code scanning, testing and other functions off-host following a code change while allowing developers to continue to use the ISPF interface. To summarize, this option offers the advantages of the Git-native model without repo migration.

    Recommendations

    Due to the nature of mainframe applications, risk management is usually paramount so a thoughtful, pragmatic approach to the adoption of Git is recommended. Provide those career mainframers that are open to adoption with the time they need to embrace the technology by introducing Bridge for Git, for example. As organizations become more comfortable with the hybrid model and are satisfied they have enough Git-based developers with institutional knowledge of their codebase, they can identify apps to port to their Git enterprise servers.

    At the other end of the spectrum, for companies with a Git-only imperative, one option is migrating an initial set of applications quickly using, for example, Team Build. However, a cold cut-over at the portfolio level could be excessively disruptive and it’s not recommended. Accelerated retirements and/or increased retention costs are a real risk, so a phased approach over time is the best approach. The key to success with Git adoption is recognizing it is a journey for the organization that requires a well-thought-out plan.

    For a third-party perspective on the adoption of Git for mainframe apps, IDC has published an Infobrief, Git for Mainframe: Considerations for Enterprises, highlighting the opportunities and the challenges.

    There are many paths to modernization, so take advantage of others who have blazed the trail.  But whatever the approach to Git adoption, younger team members should be prepared for a “We were doing that way back when” story or two.

  • It is Time to Secure Git

    It is Time to Secure Git

    At this point, we’ve got a ton of experience with the bits of Git that we use. And locking down Git is well-documented. Okay, it is documented; we can say that, at least. Interestingly, much of the ‘how to secure Git’ information out there is actually ‘how to keep critical information out of Git,’ and that is the problem. ‘Critical information’ always includes warnings to “not put configuration information into Git. Ever.” And that has been the default way to lock down Git for nearly its entire existence.

    Then along comes GitOps. What is its purpose, in a nutshell? To put that critical information into Git. So we have been waltzing along; only a fraction of us actually locking down our implementation because we were making sure the data in Git, while valuable, was not full of bits of information that would be useful in compromising systems. But now it will be. In fact, in GitOps, nearly everything is one of those bits of information useful to attackers.

    All of which means that it is time to get serious about Git security. Not content, but actual Git security. I won’t go crazy with links to lockdown documents here—it is a vastly different process to lock down a local Git repository versus a remote one and it is drastically different when you’re locking down a service like GitHub or GitLab (actually, some of the work is done for you with a SaaS vendor) … so, you’ll have to look it up yourself, based on your Git implementation. But there are documents out there that walk you through it; you’ll just have to make sure you find the ones that say, “How to lock down Git repositories” and not “How to keep privileged data out of Git repositories.”

    But whatever you do, stop putting it off. Now, if a neer-do-well gets into your repository, it won’t just be source code for various apps that are leaked—and that is certainly bad enough, even if your organization is not a software organization—it will be infrastructure information that can be used and abused. So make certain that information is protected in every possible way. Point security at Git and tell them, “We know you are insanely busy, but this is the most important thing to protect if we are going to implement GitOps.” Because it quite literally is. If database configuration, network configs and possibly even login info are to be stored in Git, locking it down (both local and remote copies) is imperative. You wouldn’t leave that info in plain text on a centralized data store and count on existing protections. So don’t do the equivalent with Git.

    If GitOps isn’t in your immediate future, that’s still your source code full of vulnerability advertisements out there, so at least ask your security team to give your Git install/policies/config a review. Even as a developer, I am less worked up about source code repositories than configuration repositories. There are several reasons for that, but the big one is you all. The bulk of organizations just don’t seem that worked up about source code repository security, but that changes when operations is asked about the list of holes in the firewall being moved into Git—and being configurable from Git (as just one example).

    So, keep kicking it! You all are making, delivering and securing a metric ton of software, and the complexity of our environments did not decrease after the remote work wave hit. Yet, you are still rocking it. May you have enough staff to do the job and applicants that come looking to work with you over competitors. And keep showing the world that software is manageable. But secure your Git.

  • GitHub Updates CLI to Make Creating Extensions Easier

    GitHub Updates CLI to Make Creating Extensions Easier

    GitHub this week made available an update to its command-line interface (CLI) that enables DevOps teams to add extensions to workflows.

    Billy Griffin, director of engineering for GitHub, said the company’s initial focus was providing DevOps teams with a set of workflows that could be invoked easily via a CLI. Now GitHub is encouraging developers to extend those workflows using version 2.0 of its CLI that hopefully will be shared with the larger community, he added.

    For example, extensions have already been created that order branches by how recently they were created and then display all associated pull requests.

    At the other end of the spectrum, developers are creating simple screensaver extensions for their own amusement.

    The level of interest in extensions varies widely across DevOps teams. Many of them simply want to employ an opinionated workflow that has been created on their behalf. Other DevOps teams want to be able to make modest modifications. Others prefer to customize workflows as much as possible to fit their specific purposes. Version 2.0 of the CLI provided by GitHub enables each DevOps team to decide how much to modify their workflows. However, as a DevOps team gains more experience, increasingly they tend to want to customize workflows.

    The number of workflows being created around the GitHub repository continues to increase steadily as more organizations embrace best GitOps practices. GitHub is the most widely employed repository for managing software artifacts, and the level of sophistication of workflows built using either scripts or GitHub actions is increasing steadily. Much of that effort is focused on streamlining processes that otherwise would get in the way of enabling developers to spend more time writing code.

    Of course, the pressure on developers to write more code faster has never been greater. As organizations embrace digital business transformation, they are discovering how dependent they are on software to create customer experiences that drive revenue. The challenge comes at a time when many developers are working from home to help combat the spread of the COVID-19 pandemic and many existing workflows need to be re-engineered. At the same time, development teams are becoming more distributed as organizations become more comfortable with hiring developers anywhere in the world.

    It’s not clear to what degree the changing application development landscape might drive organizations to re-evaluate what platforms they are employing to drive DevOps processes. Many on-premises development platforms have been replaced by cloud-based alternatives in the past year. However, because necessity is the mother of invention, the workflows relied on prior to the pandemic are not only now forever changed, but they will also likely be altered again as development teams work more intermittently in-office and from home once the pandemic subsides.

    Regardless of how those workflows evolve, the single-valued DevOps attribute is going to be flexibility. The challenge now is determining which platform does the most to enable that flexibility at a time when uncertainty continues to prevail.

  • Download Rocket Open Source Languages for z/OS Today!

    Download Rocket Open Source Languages for z/OS Today!

    Empower your developers to use their preferred open source languages. We offer many open source languages and tools—including Git, R and Python—that allow any developer to harness the power of the mainframe without spending months learning new skills.

    Git
    Git is a version control system (VCS) for tracking changes in computer files and coordinating work on those files among multiple people. It is primarily used for software development, but it can be used to keep track of changes in any files. The z/OS port of git supports automatic tagging and conversion of text files between ISO8859-1 (ASCII) and IBM-1047 (EBCDIC) character encodings.

    Bash
    Bash is an sh-compatible shell that gives users a command-line interpreter through a traditional interface via text window and a script execution capability. It is the default shell on Linux and Mac OS X.

    Bison
    Bison is a programming utility for creating parsers. It’s commonly used in both build automation and build automation generation, and converts annotated context-free grammar into a C, C++ or Java parser. Bison is used by PHP, Bash, Ruby and many other languages and tools.

    bzip2
    bzip2 is a data compressor for single files (it is not a file archiver). It typically compresses files to within 10% to 15% of the best-available techniques of statistical compressors at rates approximately twice as fast at compression and six times faster at decompression. This version of bzip2 was not written to exploit z Systems architecture; Gzip may work best for these needs.

    cURL
    cURL is an open source tool that can transfer data to and from servers using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT, TELNET, LDAP or FILE). This tool can work without user interaction. cURL functionality includes proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer, resume and more.

    Diffutils
    Diffutils is a package of several programs intended to find differences between files. The diff command, for example, shows exact line differences between two files.

    Gzip
    Gzip is a popular data compression/decompression program (our ported tools have been compressed using this utility). This version of Gzip has also been modified to exploit z/OS EDC compression hardware, if present on your system.

    libssh2
    libssh2 is a software library that allows developers to use the SSH-2 network protocol in an easy-to-use, self-contained package. This package only contains LIBSSH2 libraries intended for use by other programs or utilities.

    m4
    M4 is a general-purpose macro processor. Most users require M4 simply as a dependency of Autoconf or Bison, although the language is used in other contexts such as text template re-use, text editing and text-processing applications.

    Make
    Make controls the generation of program executables and other non-source files from the program’s source files. Make allows users to build, install and update packages without knowing the process details.

    Mktemp
    The mktemp utility takes a given filename template and overwrites a portion of it to create a unique filename. Mktemp also manages the creation of temporary files and directories that allow the user to avoid cleaning up files that are not needed permanently.

    OpenSSL
    OpenSSL is a software library used in applications that need to secure communications against eavesdropping or need to ascertain the identity of the party at the other end.

    Pkg-Config
    pkg-Config is a programming tool that assists with compiling applications and libraries. It helps you insert the correct compiler options on the command line without the need for hard-coding values on where to find glib (or other libraries).

    Sed
    sed (stream editor) is a utility for parsing and transforming text. It is typically used for extracting part of a file using pattern matching, or substituting multiple occurrences of a string within a file.

    sudo
    sudo is a command-line utility that allows admins to delegate authority, giving certain users the ability to run commands as a super-user or as another user, while providing an audit trail of the commands and their arguments.

    Unzip
    UnZip is an extraction utility for archives compressed in .zip format. Unzip is also compatible with PKZIP and PKUNZIP utilities for MS-DOS as well as Info-ZIP’s and supports bzip2 compression methods, as well.

    Vim
    Vim is a highly configurable text editor built to make creating and changing any kind of text very efficient.

    Zip
    Zip is a compression and file packaging/archive utility. Zip puts one or more compressed files into a single ZIP archive, along with information about the files.

    zlib
    zlib is a general-purpose, lossless data compression library providing in-memory compression and decompression functions, including integrity checks of the uncompressed data.

    Download Now
    Read more about Rocket Support for Open Source Tools
    Join the FREE Rocket Open Source Tools Forum