devopsy.kinsta.cloud

Author: Mike Vizard

  • Linux Foundation Project Secures Software Supply Chains

    Linux Foundation Project Secures Software Supply Chains

    The Linux Foundation today embraced a sigstore project founded by Red Hat, Google and Purdue University to make it simpler for developers to employ cryptographic software, enabled by transparency log technologies, to secure software supply chains.

    Luke Hinds, security engineering lead in the office of the CTO at Red Hat, said he and Dan Lorenc, a staff software engineer for Google, led a small team creating a lightweight mechanism for signing software artifacts, such as release files, container images and binaries, in way that creates an immutable, tamper-proof log.

    Now, the goal is to take that project to the next level by contributing it to the Linux Foundation, at a time when the focus on software supply chain security has increased in the wake of a spate of high-profile breaches. As part of that effort, a free service, based on sigstore, will also be made available by the Linux Foundation to make the cryptographic software easily accessible to a wide range of developers.

    Hinds said one of the reasons that artifacts are not signed by developers is that the current processes employed are too cumbersome. The cryptographic software is designed to be integrated as an extension of existing workflows, Hinds noted.

    Because the log created is immutable, everyone participating in the software development process, including auditors, can now transparently track chain of custody for any software artifact, Hinds said. At its core, sigstore creates a set of short-lived ephemeral keys with a trust root based on an auditable public log.

    Existing approaches rely on digests stored on insecure systems that are susceptible to tampering. Cybercriminals can, for example, swap out digests if they can gain access to compromised credentials. Those types of targeted attacks can be thwarted using sigstore, because the log management system that underpins it is immutable, said Hinds.

    Of course, no system for securing software supply chains is going to be useful if developers won’t employ it. Key management, key compromise/revocation and the distribution of public keys and artifact digests are challenging to manage. Developers, then, need to determine which keys to trust in addition to learning how to sign them when different platforms are employed. Digests and public keys are also often distributed across vulnerable websites susceptible to hacks, or stored as a README file on a public git repository.

    As organizations embrace DevSecOps best practices, it’s only a matter of time before the entire software supply chain comes under greater scrutiny. Ultimately, the issue will come down to finding a way to secure those supply chains, either from the top down by deploying and maintaining a platform, or simply making it easier for developers to better secure artifacts from the bottom up.

    Regardless of approach, the signing of software artifacts should, one way or another, become more routine in the weeks and months ahead. The biggest challenge will be overcoming the simple inertia that currently exists among developers who routinely ignore the whole process.

  • GitGuardian Reports Careless Handling of Application Secrets

    GitGuardian Reports Careless Handling of Application Secrets

    A new report, the 2021 State of Secrets Sprawl on GitHub, published today by GitGuardian, a provider of a tool for monitoring usage of application secrets, suggests developers are not especially good at keeping those secrets safe.

    Based on an analysis of every single commit made to GitHub, the report finds there has been a 20% year-over-year increase in the number of secrets – such as application programming interface (API) keys, private keys, certificates, usernames and passwords –  discovered on a public repository.

    Developers, as they build an application, make secrets easily accessible as plain text for the sake of convenience. However, when those applications are deployed in a production environment, developers often forget to delete the secrets they stored in GitHub.

    GitGuardian CEO Jeremey Thomas noted that cybercriminals are now actively scanning repositories looking for secrets that would enable them to compromise applications.

    As the number of application development projects being launched increased, so too did the number of application secrets being stored in repositories like GitHub. The GitGuardian report notes 60 million repositories were created in the last year, representing a 35% increase over the previous year. GitGuardian claims it detected more than 5,000 application secrets per day. Most of those secrets have been detected in repositories that belong to developers (85%), while the other 15% belonged to organizations. The top four countries where secrets were disclosed were India, Brazil, the United States and Nigeria.

    GitGuardian also notes it sent out 937,539 pro bono alerts to 558,086 developers last year that informed developers that application secrets were exposed. The mean time for GitGuardian’s algorithm to detect a leak is four seconds. The average mean time to response of a developer when informed of the issues is about 25 minutes, the report noted.

    As a best practice, GitGuardian recommends not storing unencrypted secrets in git repositories, and that unencrypted secrets should never be shared via messaging systems like Slack. DevOps teams should also restrict API access and permissions.

    As more attention is paid to how software supply chains are managed in the wake of some recent, high-profile breaches, there is no doubt the number of audits conducted will increase. Many of those auditors will specifically be trying to ascertain how application secrets are being managed. Developers may have to get used to a new, higher level of scrutiny, especially as DevSecOps practices become more widely adopted.

    In the meantime, DevOps teams may want to review their dependency on public repositories. There are plenty of private repository options that provide most of the same source code sharing capabilities as a public repository. That may mean someone on the DevOps team is required to manage the private repository, but, given the fact that many regulatory bodies will soon be looking to make an example of organizations that don’t secure application secrets, chances are good those costs will be trivial compared to a fine that would be levied.

  • Kong Integrates Service Mesh with OPA

    Kong Integrates Service Mesh with OPA

    Kong, Inc. today announced an update to Kong Mesh, built on top of open source Kuma service mesh, that adds support for Open Policy Agent (OPA) software that enables DevOps teams to manage compliance as code.

    Kuma and OPA are both being developed under the auspices of the Cloud Native Computing Foundation (CNCF). Kuma, unlike other service mesh offerings, provides connectivity across both microservices-based applications running on Kubernetes platform and monolithic applications deployed on virtual machines.

    In addition to adding support for OPA, the 1.2 release of Kong Mesh adds authentication mechanisms between the global and remote control planes operating across multiple zones. That functionality extends the Kuma Discovery Service (KDS) to support multiple authentication mechanisms.

    Finally, Kong has added support for the federal information processing standard (FIPS).

    Like the Istio service mesh, advanced by Google and IBM for Kubernetes environments, Kuma is based on open source Envoy proxy software. In contrast, however, Kuma is both lighter-weight and designed to span both cloud-native and legacy application environments.

    Marco Palladino, CTO at Kong, said OPA support will be a significant advance for both classes of workloads as organizations start to employ Kuma within the context of zero-trust computing environments. As DevSecOps processes are extended to include managing compliance as code, OPA will provide a programmable mechanism for ensuring that policies are not only properly implemented, but can also be easily audited, Palladino said.

    In general, an effort led by Kong, VMware and others to elevate the service mesh conversations to more broadly address application connectivity is underway. A service mesh, at its core, provides the means for integrating hundreds of application programming interfaces (APIs) at scale. It also provides a level of abstraction above the network layer that makes it easier to connect applications across a heterogeneous networking environment.

    Kong is making a case for a lighter-weight approach that spans multiple platforms and is highly extensible. As such, third-party vendors and IT teams can add extensions to Kuma as they see fit.

    Ultimately, Palladino said, the time has come to take cloud computing to the next level. If all organizations wind up doing is lifting and shifting monolithic applications to public clouds, an opportunity to advance the state of IT will have been lost.

    Of course, it’s still early days as far as adoption of service meshes are concerned. Most organizations today are relying on API gateways, proxy software or ingress controllers to manage APIs. It’s not clear to what degree a service mesh may remove the need for some of that existing application connectivity middleware.

    Regardless of which approach is ultimately favored, it’s apparent that, as microservices-based applications become more widely deployed, a different approach to application connectivity will be required. Monolithic applications, however, are not going away. In some cases, a monolithic application might be carved up into a smaller set of microservices, but it’s unlikely that every existing monolithic application will lend itself to that architecture any time soon.

  • Despite the Pandemic, DevOps Salaries Rose in 2020

    Despite the Pandemic, DevOps Salaries Rose in 2020

    Puppet today published the results of an annual global DevOps survey that found, despite the downturn in the global economy brought on by the COVID-19 pandemic, most DevOps professionals saw their salaries rise in 2020.

    However, for the first time since Puppet started collecting this data, DevOps professionals working in the life sciences, health care and pharmaceutical sectors, on average, made more than those working in the financial services sector. Not surprisingly, the report also found that DevOps professionals working in the U.S. made more than their global counterparts.

    Nearly two-thirds of respondents in the life sciences, health care and pharmaceutical sectors earn more than $100,000. That compares to 53% in the financial services sector and 45% of those working for technology companies.

    The survey was based on responses from 2,243 DevOps professionals, and noted that organizations with highly evolved DevOps practices pay better than those that are less evolved. Among respondents working at highly evolved companies, 80% earn $75,000 per year or more. That compares to 57% that earn as much at less evolved companies. Much of that discrepancy is attributable to DevOps managers that make much more at highly evolved companies compared to their counterparts at less-evolved companies, the report noted. Two‑thirds of practitioners make less than $100,000 per year, the survey found. In general, respondents working at companies with revenue over $1 billion had the highest percentage of those earning $150,000 to $250,000 (22%).

    The largest percentage increases in salaries among DevOps professionals occurred in Japan and the United Kingdom (UK). Salaries, on average, did not rise in Singapore, Germany, New Zealand and Australia. In the U.S., the demographic that saw the most growth was among DevOps professionals earning between $150,000 to $250,000, most of whom are managers. More than half of all DevOps managers in the U.S. fall within this salary range, the report noted.

    The Puppet report also noted that DevOps professionals working in dedicated engineering and application development tend to make more than those that are part of a larger IT team. A third of respondents in engineering or development report salaries over $125,000 per year, compared to 22% percent in IT and 15% percent in information security.

    The Puppet report also showed a divergence between what men and women earn as salary ranges increase. For both men and women earning up to $75,000 a year, salaries are similar. Between $100,000 and $125,000, women actually earn more. However, at $125,000, men (28%) earn more than women (17%). The survey, however, does not mention the experience levels of any respondents.

    It’s also not clear whether the higher salaries among DevOps professionals found in the life sciences, health care and pharmaceutical sectors represent a fundamental shift, or is an aberration based on the amount of COVID-19 research being conducted.

    Alanna Brown, a senior director of marketing for Puppet, said the one thing that COVID-19 has changed forever is that competition for DevOps expertise is now global. Organizations are no longer limiting themselves to hiring individuals within 50 miles of their office. In fact, DevOps professionals that have shown they can successfully work remotely will, in the months and years ahead, be in greater demand than those that have only worked in an office setting, Brown predicted. Organizations of all sizes are trying to reduce their commercial real estate expenses, Brown added.

    In the meantime, the most important thing for IT professionals is to stay relevant. Site reliability engineers (SREs) with certified programming skills will make more than a traditional IT administrator. It’s up to each IT professional to make sure they have the skills that are highest in demand at a time when many IT functions are being ruthless automated by increasing numbers of DevOps professionals.

  • LogDNA Adds Usage Quota Tools to Help Control Costs

    LogDNA Adds Usage Quota Tools to Help Control Costs

    LogDNA announced today it will enable IT teams to apply usage quotas to help rein in log management costs.

    Tucker Callaway, CEO, LogDNA, said as applications become more complex – thanks, in part, to the rise of microservices – the amount of log data that a DevOps team can collect has significantly increased. IT teams can now assign DevOps teams both hard and soft quotas to limit the amount of log data collected during a specific time period, Callaway said.

    IT teams can set a daily or monthly hard limit on the volume of logs stored or, alternatively, they can also set soft daily or monthly quotas that can apply throttling logic to ensure mission-critical log data will continue flowing through LogDNA as usage volume approaches the limit threshold. It will then be up to each IT team to determine, based on the tasks at hand, whether to enable a DevOps team to exceed those thresholds, Callaway said.

    IT teams can also make use of soft limits to reallocate storage resources from one DevOps teams to another based on how much log data might be generated in a specific time period, Callaway noted.

    LogDNA will notify team members any time a quota is hit, or when an exclusion rule is triggered because a quota has been reached. Those alerts can also be integrated with email systems or the Slack messaging platform. Logs that are not retained are not counted toward the monthly bill, but are displayed and will trigger other relevant alerts.

    In the wake of the economic downturn brought on by the COVID-19 pandemic, IT organizations are, generally, more sensitive to costs. At the same time, many of them are encouraging developers to assume more responsibility for application life cycle management as they embrace DevOps best practices. The usage quotas are intended to enable IT teams to strike a better balance between what can be two competing agendas, Callaway said. IT leaders are not necessarily trying to discourage DevOps teams from collecting log data as much as they are trying to make sure costs are predictable, Callaway added.

    Log data, of course, is one of the core elements of any observability strategy. While observability has always been a core DevOps principle, the number of organizations that have been able to meaningfully achieve it has been limited. However, as organizations start to deploy microservices with multiple dependencies, the need to provide context when troubleshooting those applications has become more acute.

    It’s not clear to what degree more granular control over log data might sway an IT organization toward one platform versus another. However, when it comes time to justify a choice to senior IT and business leaders, it’s always a good idea to address, at some point, how one decision versus another might have a more positive impact on the bottom line.

  • GrammaTech Allies with GitLab to Advance DevSecOps

    GrammaTech Allies with GitLab to Advance DevSecOps

    GrammaTech announced today it has partnered with GitLab to integrate its GrammaTech CodeSonar static application security testing (SAST) tools with the GitLab Ultimate DevSecOps platform.

    Vince Arneja, chief product officer at GrammaTech, said integration with continuous integration/continuous delivery (CI/CD) platforms such as GitLab is critical because it enables security scans to run automatically any time code is merged. That capability reduces the amount of code that is scanned at any one time, Arneja said.

    GrammaTech is partnering with multiple CI/CD platforms and integrated development environment (IDE) providers to make it simpler to create multiple points for scanning code during the application development process, Arneja noted.

    GitLab, meanwhile, is providing its own tools for analyzing artifacts as they move through the software development life cycle. The GrammaTech tools, available via the highest tier level made available by GitLab, analyze code at a deeper level, said Arneja.

    Ultimately, the goal is to enable developers to discover security flaws as early as possible in the application development life cycle. The later those flaws are discovered, the more expensive they become to fix, Arneja said.

    Developers, naturally, tend to postpone security scans as they race to meet an application deadline. That practice creates issues; the more code there is to analyze, the longer such scans take. It’s generally more efficient to scan smaller amounts of code more frequently. Otherwise, developers can become overwhelmed by the number of security bug fixes that need to be addressed at the back end of the application development process. Automating security scans allows an organization to move toward embracing DevSecOps best practices in a way that doesn’t rely on a developer remembering to initiate a scan.

    Most developers are not deliberately ignoring security issues – it’s just that the existing, manual processes for discovering those flaws is inefficient.

    Automated scans also provide the added benefit of simplifying discovery of common security flaws long before a code review, creating more time to address significantly more complex issues.

    Application development teams are also under pressure to reduce the total number of bugs that need to be fixed after an application is deployed. As the number of applications being deployed steadily increases, developers can find themselves spending more time fixing bugs than writing new application code.

    Most organizations are not very far along in their journey toward embracing DevSecOps. However, as it becomes easier to integrate a variety of scanning tools within DevOps workflows, the number of organizations moving up the DevSecOps maturity curve should increase. In the wake of some recent, high-profile attacks on software supply chains, the sense of urgency surrounding the adoption of DevSecOps best processes has undoubtedly increased.

    In the longer term, there may soon come a day when security is viewed as just one of many quality assurance gates that code needs to pass through before it’s allowed to be promoted. In the meantime, an ounce of cybersecurity prevention at the front end of any application development is most certainly going to be worth more than several pounds of cybersecurity cure applied too late.

  • Scarf Gateway Tracks Open Source Software Usage

    Scarf Gateway Tracks Open Source Software Usage

    Scarf today exited stealth to launch a gateway that makes it simpler for individuals or organizations to track usage of open source software.

    Avi Press, founder of Scarf, said the Scarf Gateway is an online service through which providers of open source software can redirect downloads from a wide range of repositories and registries. That extra step then provides those entities with data and insights about the type of individuals and organizations using their software.

    Fresh off raising initial seed funding, Press said the Scarf Gateway will make it easier for developers of open source software to employ multiple repositories to distribute their code, because they won’t become overly dependent on metrics shared by the provider of a single repository.

    Most of those registries today only provide download counts for packages. Scarf Gateway will provide insights into, among other attributes, downloads by version, platform, location, company and cloud environment(s).

    The majority of organizations making open source software available don’t have a lot of visibility into how their offerings are being employed until organizations using the software sign up for commercial support. Armed with more insights into how their software is being used, many of those providers would be able to more proactively collect feedback, in addition to promoting their commercial services.

    Many open source project maintainers have a difficult time prioritizing what capabilities to add or fix, simply because they don’t know enough about how their software is being employed, Press said.

    The ability to more granularly track usage comes at a time when tensions over open source monetization and compensation are rising. In many cases, providers of open source software are changing the way they license their software to prevent cloud service providers from reaping most of the benefits of their work. Startups, typically funded by venture capitalists, make their software available as open source code in an effort to build a customer base on the strength of their support services. Cloud service providers have discovered they can take the code these vendors create to launch their own service. Those cloud service providers may contribute code to the project, but the startup vendor doesn’t typically enjoy any of the economic benefits.

    Many individual contributors don’t work for a company that sponsors their contributions, and also want to be paid for their efforts. A survey of 4,440 developers who actively participated in open source projects in 2020 finds more than half (54%) of respondents said they feel individuals should be paid for their contributions to open source projects. The survey, conducted by Digital Ocean, a cloud service provider, found only 14% of respondents are currently paid for their contributions.

    It’s too early to say what impact the ability to track usage of open source software may have. Many individuals using open source software may prefer their current level of anonymity. Others might participate more in a project if engaged. Like it or not, however, large swaths of the open source community want to know the degree to which their efforts are making an actual difference.

  • Lightrun Tool Now Debugs On-Premises Apps

    Lightrun Tool Now Debugs On-Premises Apps

    Lightrun today announced it has made its namesake debugging tools available for applications deployed in on-premises IT environments that, for one reason or another, can’t be deployed on a cloud platform.

    lan Peleg, CEO of Lightrun, said Lightrun is unique in that software teams can capture live production data from software installed on their customers’ sites via a set of command line interfaces (CLIs). That approach eliminates the need for the end customer to set up a separate environment to troubleshot an application, said Peleg.

    Instead, software teams can troubleshoot their software without requiring on-site visits by support engineers or even a phone call, Peleg said.

    Lightrun makes it possible to add logs, virtual “breakpoints” and metrics while an application is running. Those capabilities are critical because many applications are not fully instrumented. Lightrun makes it simpler to triage issues by adding a missing log line, instrumenting a new metric or taking a snapshot of an application as it runs in a production environment. Those capabilities make it possible to find bottlenecks and errors without requiring the application to stop running.

    The performance impacts of Lightrun range from tens to hundreds of microseconds per invocation to minimize overhead, Peleg said.

    In general, Peleg said, debugging applications is becoming more challenging. Monolithic applications continue to grow in size while emerging microservices-based applications have dependencies that make them challenging to debug. Many organizations are employing application performance management (APM) platforms and other observability tools to try and address these issues. However, those platforms require IT teams to continuously collect data, and even then there is no guarantee the insights needed to troubleshoot an application will be surfaced. Lightrun, conversely, makes it possible for developers to troubleshoot an application and capture the metrics needed in real-time. That data can then be shared with an APM platform to make it easier to address similar issues if required, Peleg said.

    As developers become more accountable for application performance, many of them are not going to want to employ the same tools and processes that IT operations teams have historically relied on. Lightrun provides a lighter-weight approach to troubleshooting applications, within the context of a DevOps workflow, that doesn’t require as much overhead, said Peleg.

    Although observability is a core DevOps principle, many organizations have been unable to achieve it in a meaningful way. There is no shortage of tools for monitoring IT environments, but correlating all the data those tools generate is problematic. It’s not uncommon for organizations to spend weeks trying to identify the source of an issue that only takes a few minutes to fix. Now that developers are assuming more responsibility for managing the entire lifecycle of an application it is likely different approaches to achieving observability will be considered.

    In the meantime, as the number of applications deployed in production environments continues to increase, different approaches to debugging software will be required. Otherwise, at the current rate of deployment, it won’t be long before developers find themselves spending most of their time debugging existing applications, versus actually writing new code.

  • Value Stream Management Consortium Makes Debut

    Value Stream Management Consortium Makes Debut

    A consortium dedicated to advancing the adoption of value stream management best practices is being launched today with backing from Digital.ai, HCL Software, Plutora, ServiceNow and Tasktop.

    The Value Stream Management Consortium (VSMC) will create an open community around which organizations that are seeking to expand DevOps workflows and processes across functions and organizations can more easily collaborate.

    VSMC chair Helen Beal said that while value stream management has been around as a concept for years, the number of organizations that have fully embraced the concept remains relatively small. The VSMC is dedicated to enabling organizations to orchestrate software development processes more flexibly, she said.

    VSMC Chair Helen Beal

    That goal has proven elusive, because it requires a level of collaboration between release managers, DevOps managers and product managers as well as IT and business leaders. Value stream management provides a framework for enabling organizations to embrace value stream management concepts as they become more adept at employing DevOps workflows to manage multiple application development initiatives simultaneously.

    Initially, Beal said, the VSMC will function as a central hub of information and education pertaining to value stream management. Ongoing research will track how organizations measure value. Eventually, the VSMC will provide certified training, Beal added.

    The concept of value stream management traces its lineage back to lean manufacturing methods, which called for each step of a manufacturing process to be continuously measured. As software development has evolved from being a craft to a process that is automated as much as possible using DevOps best practices, an appreciation for the value of monitoring things such as the impact of missed software development deadlines on the business is growing steadily as organizations realize how dependent on software they have become.

    As organizations come to terms with this new reality, many also realize there is a much greater need to align software development with their core business processes. Value stream management platforms automatically take technical metrics, gathered via a DevOps process, and match them to a set of key performance indicators (KPIs) defined by business leaders. The challenge many organizations still face is making sure they have the tools in place to collect the appropriate technical metrics and then define the KPIs that truly matter to the business. Once those metrics are gathered and KPIs established, organizations are then able to apply advanced analytics to the massive amounts of DevOps data they collect.

    Business leaders are especially anxious to understand how any software development project delay might impact future revenue projections. Rather than being a set of tools employed mainly by IT leaders to track software development projects, value stream management platforms make it possible for other stakeholders to track KPIs spanning multiple software development projects. It then becomes easier to make informed decisions about how to reassign resources across various projects as different bottlenecks are encountered.

    It may be a while before organizations achieve that level of DevOps sophistication, but as software continues to eat the world it’s only a matter of time before value stream management becomes the table stakes organizations will need to survive and, hopefully, thrive.

  • Datadog to Meld Observability and DevSecOps

    Datadog to Meld Observability and DevSecOps

    Datadog is planning to expand the scope of the security and observability services it provides following the acquisitions of Sqreen, an application security platform provider, and Timber Technologies, a provider of a tool, called Vector, for collecting and normalizing log data.

    Ilan Rabinovitch, vice president of product and community for Datadog, said Vector makes it possible to collect, enrich and transform logs and other observability data from both on-premises and in-cloud environments in a way that makes it easier to route that data to wherever it is needed via a data pipeline.

    Sqreen is a provider of a software-as-a-service (SaaS) platform that detects, blocks and responds to application-level attacks using runtime application self-protection (RASP) capabilities enabled by a web application firewall (WAF) embedded within an application.

    Rabinovitch said as IT continues to evolve, it’s clear organizations are looking to embrace DevSecOps best practices in a way that will require security tools that can be embedded within an application, coupled with observability tools that provide more context about the overall IT environment. That context is especially critical, given the amount of data that needs to be analyzed to determine the potential scope of a security threat to an IT environment, Rabinovitch said.

    The Datadog platform is already widely employed to monitor cloud computing environments. There is an industry wide effort to take monitoring to the next level by enabling observability, which applies analytics to provide more context. Surfacing that context, however, requires the ability to normalize data across a pipeline using Vector, Rabinovitch said.

    Of course, the Vector tools are equally applicable to DevOps workflows that have yet to incorporate security technologies. The goal, now, is to make it easier for IT teams, made up of developers, IT operations teams and security specialists, to collaborate by breaking down the various data silos created by IT and security management tools.

    It’s too early to tell what impact the rise of DevSecOps will have on roles within IT organizations. Most organizations will continue to employ a range of specialists that will be able to work more closely with IT generalists, Rabinovitch said. Managing IT is becoming the equivalent of a barn raising, in that it requires members of a community with different skills to come together for a certain amount of time to accomplish a specific task, Rabinovitch added.

    It’s also not clear whether IT organizations view observability as a distinct capability separate from monitoring, or whether they will simply view observability as the next logical extension of existing tools. At the core of that issue is debate over whether observability platforms need to be acquired separately, as part of an effort to replace existing monitoring tools, or whether those tools will evolve over time to provide observability capabilities.

    Regardless of the path forward, it’s apparent the increased complexity of modern IT environments – made up of microservices, containers and serverless computing frameworks – is about to force that issue.