devopsy.kinsta.cloud

Author: Mike Vizard

  • Pulumi Embeds Terraform Conversion Tool in Platform

    Pulumi Embeds Terraform Conversion Tool in Platform

    At its virtual PulumiUP event, Pulumi this week announced it embedded an ability to convert instances of Terraform into its infrastructure-as-code (IaC) platform via a simple command line interface (CLI) command. In addition, Pulumi added a Review Stacks capability that automatically creates a dedicated cloud environment to review code every time a pull request is opened.

    Pulumi is also making available a beta release of Pulumi Azure Native Provider 2.0, which decreases the size of the software development kit (SDK) by 60%. This is part of an effort to improve deployment times and reduce resource consumption by, for example, removing support for deprecated application programming interfaces (APIs).

    Finally, Pulumi has added support for property names and values to Resource Search for Pulumi Cloud, a search and analytics tool for cloud computing environments. DevOps teams can now additionally search for instances of, for example, versions of databases or specific availability zone as a result.

    Aaron Kao, vice president of marketing for Pulumi, said previously the company made available a separate tool for converting snippets of Terraform code. For the first time, Pulumi is also making available a converter plugin in the Pulumi engine which allows conversion tools from other IaC tools to be used.

    Review Stacks, meanwhile, makes it simpler for DevOps teams to pick up an unfamiliar codebase, make changes to both application and infrastructure code and share a live environment to review that code.

    Pulumi has been making a case for an alternative to Terraform that centralizes the management and provisioning of infrastructure using a set of embedded guardrails. One of the major issues enterprise IT organizations are encountering these days is the number of misconfigurations created when developers use tools like Terraform to provision infrastructure on their own. Those misconfigurations often lead to, for example, ports being left open and cybercriminals exfiltrating data.

    That approach is gaining traction among enterprise customers that are looking to centralize the management of DevOps processes, said Kao.

    In the wake of a series of high-profile breaches, more organizations are now reviewing the security of their software supply chains, including how infrastructure is provisioned. Many of those organizations will need to revisit those processes as pending legislation that requires more stringent requirements for building software eventually become law, noted Kao.

    At the same time, many organizations are also looking to improve the productivity of DevOps teams during an economic downturn that limits the number of full-time employees they might be able to hire. The pace at which software is being developed has not slowed, so there is a greater urgency to do more with less.

    It’s not clear how quickly organizations are centralizing the management of DevOps workflows, but with the rise of platform engineering, there is clearly more interest in defining consistent backend processes that multiple DevOps teams are currently managing individually. The challenge is finding a way to achieve that goal without impeding the pace of innovation enabled by embracing DevOps in the first place.

  • BMC Adds Generative AI Capabilities to Helix Management Platform

    BMC Adds Generative AI Capabilities to Helix Management Platform

    BMC today revealed it is embedding generative artificial intelligence (AI) capabilities across its BMC Helix service and operations management platforms.

    BMC CTO Ram Chakravarti said the company is incorporating the open source GPT-J large language models (LLMS) to build generative AI models. Those models are optimized for specific domains using data collected by BMC and imported from other data sources, like the Jira project management software from Atlassian. That persistent layer of storage is designed to aggregate multiple types of data within what amounts to a Helix data lake.

    The generative AI capabilities added to BMC’s portfolio include natural language summaries of issues and events captured by BMC Helix Operations Management, along with an ability to create a historical analysis of a user’s specific environment to identify recurring issues more easily.

    BMC Service Management, meanwhile, can now correlate incidents and provide resolution summaries that can be stored. It’s also now possible to run clustering analysis on chat archives to find new or improved utterances that help virtual agents answer queries better and execute workflows.

    BMC initially opted to build its own LLMs because general-purpose LLMs—such as the one created by OpenAI—are subject to “hallucinations.” Hallucinations are created when incorrect data is included in a model. BMC is training its LLMs on a narrow range of trusted data to ensure the recommendations and subsequent automations it creates are valid.

    Over time, BMC will add additional LLMs; the company hasn’t yet decided if there are use cases where it makes sense to rely on an existing general-purpose LLM to add additional capabilities, noted Chakravarti.

    Organizations can also opt to use a containerized version of the BMC LLM if they prefer, but BMC will encourage most organizations to invoke generative AI via its Helix SaaS platform, he added.

    BMC Helix is an IT services management (ITSM) platform that integrates with a range of DevOps platforms. Helix is part of a larger existing BMC Connected Digital Ops initiative that seeks to unify ServiceOps and DevOps with AIOps, DataOps and AutonomousOps. Generative AI will automate a range of tasks from within Helix that can then be extended to DevOps platforms such as Jenkins.

    It’s still early days as far as applying generative AI to processes, so organizations should focus on use cases where it will add some immediate value, said Chakravarti. Once they develop a level of trust in the automation created and executed, they can then pragmatically expand the number of generative AI use cases where they make sense, he added.

    One way or another, generative AI is going to be applied to the management of IT. Many routine tasks that conspire to limit the productivity of IT teams today will be eliminated, but there will always be a need for a human to remain in the loop to ensure the right outcomes are achieved. However, it will also soon become possible for existing teams to manage IT at levels of scale that were previously unattainable.

    Image source: Photo_Credite_Indira-Tjokorda via Unsplash

  • HashiCorp Releases Slew of Updates to Improve DevSecOps

    HashiCorp Releases Slew of Updates to Improve DevSecOps

    At a HashiDays event this week, HashiCorp launched updates to its Terraform implementation and a software-as-a-service offering, dubbed HCP Vault Secrets, that promise to simplify infrastructure security and ensure that secrets are not leaked.

    In addition, HashiCorp has added a self-hosted edition of its remote access security tool, Boundary, that can now also record SSH sessions. The company is also announcing general availability of Vault Secrets Operator, which provides a high level of abstraction for managing secrets in a Kubernetes environment and which synchronizes with the secrets management interfaces defined by the Kubernetes Technical Oversight Committee.

    Finally, HashiCorp is making available in beta a 1.16 update to the Consul service mesh for containers and virtual machines that adds a Sameness Groups capability to improve the resiliency of microservices by redirecting requests to an identical service. HashiCorp has also added a transparent proxy mode to enable failover between services along with a permissive mutual TLS (mTLS) that enables a service to be configured to accept both mTLS traffic from mesh-enabled services and non-Consul mTLS traffic from external downstream applications.

    Chris Van Wesep, senior director of product marketing for HashiCorp, said Version 1.5 of HashiCorp Terraform adds config-driven import workflow and check blocks capabilities that make it possible to declaratively use code to add existing resources into a Terraform state. Previously, the only way to update a Terraform state was to use an import command that could only be applied to one resource at a time. Terraform 1.5 also adds automatic code-generation capabilities for imported resources.

    In addition, version 1.5 of Terraform also includes a tool to validate that provisioned infrastructure is behaving as intended. That latter capability makes it possible to reduce the number of misconfigurations created by developers that might one day be exploited by cybercriminals to breach an IT environment.

    HCP Vault Secrets, meanwhile, provides a beta release of a SaaS edition of the existing Vault offering specifically designed to enable organizations to better manage application secrets that might inadvertently be exposed to cybercriminals in a production environment, said Van Wesep.

    HashiCorp is moving toward providing a portfolio of tools that enable DevOps teams to securely provision and manage hybrid cloud computing environments using a common workflow, he noted. A Forrester Consulting survey found nearly three-quarters (73%) of respondents are reporting this strategy is helping them reach their business goals, with another 19% adding they expected this strategy to help them achieve their business goals in the next year.

    It’s not clear whether organizations are fully embracing multi-cloud computing, but each additional cloud added to an enterprise IT environment increases the total cost of IT. The only way to mitigate those costs is to define a set of best practices for building and deploying applications that can be applied to any cloud computing environment.

    Unfortunately, many organizations will discover that, as they attempt to define those best practices, they have inadvertently become dependent on a set of proprietary application programming interfaces (APIs) that thwart those efforts.

  • Grafana Labs Slashes Time to Create Observability Dashboards

    Grafana Labs Slashes Time to Create Observability Dashboards

    At its GrafanaCON 2023 event today, Grafana Labs updated its core platform for visualizing data which makes it simpler to set up dashboards in minutes. The updates are part of a celebration of the 10th anniversary of a tool that is now widely used by DevOps teams.

    Ryan McKinley, a distinguished engineer for Grafana Labs, said Grafana 10 should help make DevOps more accessible to a wider range of practitioners with limited experience working with observability tools. The latest release also makes it simpler to share knowledge with external stakeholders via public dashboards, he added.

    In addition, Grafana 10 adds expanded role-based access control (RBAC) capabilities and streamlined implementation of the secure access markup language (SAML) to ensure authentication.

    Grafana Cloud has also been updated to provide a secure way to connect to data residing in an on-premises IT environment or in a cloud with the observability platform the company provides via a software-as-a-service (SaaS) platform.

    Finally, Grafana Labs is also making the engine it created to visualize data more accessible to third-party developers that are embedding visualizations based on Grafana within their applications, noted McKinley.

    Grafana Labs

    There are now more than 150 plugins for various data sources that have been made available to more than 20 million users of the Grafana visualization framework. Not all those users are working in DevOps environments, but Grafana is widely used to enable observability across increasingly complex IT environments. The Grafana Cloud seeks to capitalize on that popularity by adding a service that uses an instance of open source Grafana Mimir software to store data collected in a format originally defined for the open source Prometheus monitoring tool.

    It’s still early days as far as adoption of observability is concerned, but it’s apparent that as application environments become more complex, the plethora of monitoring tools that DevOps teams rely on will need to be streamlined. Observability platforms promise to unify logs, metrics and traces to simplify launching queries to identify the root cause of an issue.

    The rate at which DevOps teams will embrace observability will naturally vary, but the biggest obstacle might not be the platforms themselves. Instead, the issue is understanding what queries can help DevOps teams better understand the root cause of an IT issue before there is a major disruption. In the long term, it’s expected that machine learning algorithms will leverage the data collected by observability platforms to automatically identify issues that might lead to a disruption long before it actually occurs.

    Of course, the result of any query that gets launched needs to be visualized in a way that is easily understood. Grafana Labs is now making a case for a more integrated approach that, at its core, employs its visualization tools to make it simpler to discern what’s occurring in IT environments. After all, no matter how talented any DevOps team may be, it’s still not possible to manage what was never seen in the first place.

  • Survey Finds Cloud Management Strategies Shifting

    Survey Finds Cloud Management Strategies Shifting

    Even though organizations continue to consume cloud resources inefficiently, they continue to increase the amount of money allocated. These were among the findings of a global survey of 1,000 technology practitioners and decision-makers conducted by Forrester Consulting on behalf of HashiCorp.

    The survey found a full 94% of respondents reported that their organization struggled with cloud waste. Top factors contributing to avoidable cloud spending included overprovisioning resources (50%), idle or underused resources (50%) and a lack of needed skills (43%). Nevertheless, more than half of respondents (56%) said they boosted their cloud spending in the last year compared to 22% which reduced it.

    At the same time, however, 92% of respondents said they are moving toward adopting, standardizing or scaling platform teams as part of an effort to centralize the management of cloud services, but only 39% are scaling the use of platform teams throughout the organization. The survey also found, however, that organizations are addressing skills shortages by adopting a common operating model/shared automated workflows (58%), reskilling/upskilling (56%) and working with partners (43%).

    The survey also found many organizations are also committed to multi-cloud computing, with nearly three-quarters (73%) reporting this strategy is helping them reach their business goals. Another 19% said they expected this strategy to help them achieve their business goals in the next year.

    The most-cited reasons for adopting a multi-cloud strategy were reliability (51%), cost reduction (48%), security and governance (45%), digital transformation (45%) and scalability (45%). Only 10% had no choice but to go multi-cloud due to acquisitions or other issues.

    Security (88%), availability (84%) and scalability (84%) were identified most often as being important/very important multi-cloud success factors, followed closely by regulatory/compliance requirements (82%), staffing and skill levels (82%), budget (80%), visibility into cloud infrastructure (78%), automated tooling (77%) and platform teams (75%).

    The top cloud initiatives were data protection (77%), followed by secrets management and access control (both at 75%), the survey finds. The top security concerns were password/credentials/secrets leakage (50%) followed by data theft (49%), phishing (46%) and ransomware (42%).

    Overall, respondents said the biggest benefits of having a multi-cloud computing strategy were a stronger security posture (74%) and helping attract, motivate and retain skilled staffers (68%).

    Chris Van Wesep, senior director of product marketing for HashiCorp, said it’s clear organizations are purposely using multiple clouds to reduce costs by pitting one cloud service provider against another. The challenge organizations face when using that approach is they need to define a consistent set of development processes that can be used across multiple clouds to keep total costs under control.

    Organizations that embrace multi-cloud computing are also more likely to implement security best practices because of their need to consistently manage multiple cloud computing environments, he added.

    Of course, there is always going to be a great disparity between high- and low-performing organizations. The survey found that about 25% of survey respondents are high performing, with an equal number falling at the other end of the spectrum. The rest fall somewhere in between those two ends of the spectrum. The issue is that the gulf between the high and low ends of that spectrum appears to be widening as more organizations fail to muster the level of skills and expertise needed to succeed in the cloud era.

  • Cycode Leverages eBPF to Secure CI/CD Pipelines

    Cycode Leverages eBPF to Secure CI/CD Pipelines

    Cycode today added a Cimon extension to its application security platform that uses extended Berkeley Packet Filtering (eBPF) to thwart cyberattacks against continuous integration/continuous delivery (CI/CD) pipelines.

    Alex Ilgayev, head of security research for Cycode, said Cimon uses eBPF to inspect network connections, running processes and file modifications within a pipeline to learn standard behaviors. Armed with those insights, Cimon will then surface any anomaly in real-time, he added.

    DevOps teams can then craft and embed security and compliance rules within their pipelines based on the severity of the anomalies discovered, said Ilgayev.

    eBPF makes it possible to run sandbox programs at the kernel level without adversely impacting performance. In effect, eBPF bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that, historically, were completely independent of one another. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput to improve overall scale at a time when the volume of cybersecurity attacks continues to increase.

    Cycode is taking advantage of eBPF to enable DevOps teams to not only defend against known attacks involving malicious package installation, typosquatting, repojacking, dependency confusion, dependency hijacking and other types of dependency attacks but also zero-day threats, noted Ilgayev.

    Cycode provides a platform for mapping metadata and events using graph technology that makes it simpler for application development and cybersecurity to consolidate alerts generated by their DevSevOps tools. Most recently, Cycode added a Cycode Application Security Orchestration and Correlation (ASOC) module that will automatically discover all the tools that make up a DevSecOps workflow without requiring any integration effort.

    The approach enables the Cycode platform to discover, for example, dependency issues that would otherwise be missed by software composition analysis (SCA) or static application security testing tools (SAST).

    While a lot of progress has been made in terms of adopting DevSecOps best practices, many organizations are still struggling with securing their software supply chains. Most of the members of a DevOps team have limited cybersecurity expertise, so they need the help of a cybersecurity professional to determine what specific actions are required to remediate a vulnerability. Cybersecurity professionals, conversely, don’t have a lot of application development expertise.

    As the number of regulations specifically focused on application security steadily increase, every organization that builds software will soon need to embrace DevSecOps to better secure software supply chains. Unfortunately, developers have too often viewed cybersecurity as an obstacle to deployment while cybersecurity teams have historically considered developers to be a primary cause of the breaches they are expected to clean up. The first step toward bridging that divide is, of course, to realize that application developers and cybersecurity teams need to be invested in each other’s success.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    Underdog.io
    New York, New York
    DevOps Engineer
    $114,000 to $206,000

    SimplyHired.com

    Verizon
    Lone Tree, Colorado
    DevOps Engineer
    $107,000 to $226,000

    Indeed.com

    Erie Insurance
    Erie, Pennsylvania
    DevOps Solutions Architect
    $104,838 to $167, 468

    Dice.com

    Nityo Infotech Corp
    Mountain View, California
    DevOps Engineer – AWS Cloud
    $100,000 to $140,000

    CareerBuilder.com

    Boston College
    Chestnut Hill, Massachusetts
    Senior DevOps Systems Administrator
    $96,000 to $120,000

  • Snyk to Add ASPM Platform via Enso Security Acquisition

    Snyk to Add ASPM Platform via Enso Security Acquisition

    Snyk this week announced it plans to acquire Enso Security, a provider of an application security posture management (ASPM) solution that tracks events and analyzes metadata collected from DevOps and security tools.

    At the same time, Snyk revealed it has added a DeepCode AI Fix tool that creates validated fixes for code written by either humans or machines and an Insights tool that helps organizations better prioritize remediation efforts based on the risk to the business. DeepCode AI Fix extends the capabilities of a generative artificial intelligence (AI) capability based on large language models (LLMs) that Snyk gained with the acquisition of DeepCode in 2020.

    Manoj Nair, chief product officer for Snyk, said all of these initiatives represent an effort to extend the reach of Snyk’s portfolio to address a wider range of issues within the context of a larger DevSecOps workflow as well as give developers tools that enable them to write more secure code.

    The acquisition of Enso Security is expected to close this quarter. The company has been making a case for an ASPM platform that eliminates the need to manually collect cybersecurity data from the various tools and platforms that a DevOps team may be using.

    Despite innumerable cybersecurity concerns, the rate at which applications are being deployed and updated continues to accelerate. The Enso platform makes it possible for cybersecurity teams to keep pace with those projects by discovering issues before an application is deployed in a production environment.

    The challenge, of course, is that most development teams are not going to be able to address every issue before an application needs to be deployed. In addition to knowing who is developing what application code, cybersecurity teams need to also prioritize potential issues versus rather than just sharing a list of potential issues that have been discovered. Enso Security’s ASPM platform provides visibility into events so that it’s simpler for cybersecurity teams to trace issues back to individual developers or the AI platform that generated them.

    That latter issue is becoming more significant as more development teams rely on general-purpose generative AI platforms to write code, noted Nair. Those platforms are writing that code based on samples collected across the internet, many of which are deeply flawed from a cybersecurity perspective, he added.

    The Enso platform is intended to enable cybersecurity teams to have the most impact on an application development project without being forced to interrogate every developer to determine what’s occurring in any application environment. In fact, one of the reasons developers resent those conversations is they know all the relevant data has already been entered into a system—cybersecurity professionals just need to access it.

    There’s still a lot of work to be done when it comes to bridging the historic divide between cybersecurity professionals and application developers. However, the more data that can be seamlessly shared between these teams, the simpler it becomes for organizations to embrace DevSecOps best practices.

  • Report Surfaces DevOps Challenges for Mobile Applications

    Report Surfaces DevOps Challenges for Mobile Applications

    An assessment of 1,600 DevOps teams involved in building and deploying mobile applications found that 62% were adversely impacted by manual processes that slowed the rate at which these applications were deployed and updated.

    Based on the Mobile DevOps Assessment (MODAS) created by Bitrise, a provider of a continuous integration/continuous delivery (CI/CD) platform for building and deploying mobile applications, the metric measured the creation, testing, deployment, monitoring and collaboration phases of building and deploying applications.

    The survey also found (44%) of respondents reported their release approval process is mostly or entirely manual, with only 9% having fully automated application releases. Only 22% of teams said they’ve been able to complete internal release processes in less than an hour.

    The survey also found that nearly two-thirds of organizations (66%) have deployed applications that can’t be opened in two seconds or less, which is widely considered the accepted level of performance required.

    In addition, three-quarters of organizations (75%) required more than two days to address bug fixes. Only 21% of teams said that they had implemented some form of app performance monitoring to keep track of bugs, the report found.

    Daniel Balla, chief strategy officer for Bitrise, said MODAS applies many of the concepts from the DevOps Research and Assessment (DORA) metrics that Google now oversees. Those metrics, however, are not specifically designed to assess mobile application development initiatives, he noted.

    It’s not clear what percentage of applications DevOps teams are building today are deployed on mobile devices. But given the emphasis on digital business transformation initiatives, many of these applications are increasingly mission-critical to organizations. The general expectation among mobile application end users is that apps will be regularly updated with new functionality. However, if release processes are largely manual, regularly improving the user experience can become problematic. In addition to the increased likelihood that mistakes will be made, manual processes are often the reason why a vulnerability was introduced into an application environment.

    DevOps teams will also need to also consider the level of scale that will be required to support mobile app development, noted Balla. Many of the tools that developers rely on today to build applications were not necessarily designed to enable developers to build and deploy mobile applications at scale, he added.

    Finally, most enterprise applications need to run on multiple devices, so the tools used to create these applications can’t focus on applications that can only run on one platform, said Balla.

    Regardless of the approach to building and deploying mobile applications, the amount of focus on them within most organizations is high; any issues that arise are likely to be noticed by the highest echelons of an organization. That level of attention can easily put pressure on DevOps teams that frequently work on multiple projects. The challenge and the opportunity now is to create a set of DevOps workflows optimized for mobile applications that almost by definition require much more frequent updates than any other type of application.

  • Atlassian Advances DevSecOps via Jira Integrations

    Atlassian Advances DevSecOps via Jira Integrations

    Atlassian announced today it has allied with Snyk, Mend, Lacework, Stackhawk and JFrog to make it simpler to aggregate vulnerability data within the Jira project management software that many organizations rely on to manage application development and deployment efforts.

    Andrew Pankevicius, a senior product manager for Atlassian, said the collaborations will make it easier to prioritize a list of vulnerabilities that need to be addressed and for DevOps teams to automatically assign that work to a continuous integration/continuous delivery (CI/CD) pipeline. Atlassian provides those third-party integrations via an Open DevOps initiative.

    These capabilities, accessible via the Security Tab in Jira, will make it simpler for DevOps teams to manage output from the multiple application security tools that have become part of a larger DevSecOps workflow, noted Pankevicius.

    These additions to the Atlassian portfolio are arriving at a time when there is a lot more focus on securing software supply chains in the wake of a series of high-profile breaches. Those breaches have led to a wave of legislative proposals in the U.S. and Europe that will soon require organizations to embrace DevOps best practices to address increased potential liability that might stem from an application breach.

    For example, a proposed Cyber Resilience Act being negotiated by the member states of the European Union seeks to require organizations that sell internet-connected hardware platforms to ensure that their devices—and the software that runs on them—comply with cybersecurity best practices.

    In the U.S., meanwhile, a National Cybersecurity Strategy proposal put forward by the Biden administration seeks to hold organizations that collect data or build software more accountable for breaches.

    While both proposals are a long way from becoming the law of the land anytime soon, they are indicative of changing attitudes. Governments around the world are concluding the only way to ensure better cybersecurity is to require it. Previous suggestions made by government agencies are going to soon give way to actual mandates. As such, DevOps teams would be well-advised to get ahead of this sea change by implementing the tools and processes that inevitably will be required. As a result, many organizations are now embracing DevSecOps best practices that shift more responsibility for application security further left toward developers.

    The challenge is finding the right set of cybersecurity tools that surface vulnerabilities as code is being developed to make sure context isn’t lost, said Pankevicius. Historically, cybersecurity teams have discovered vulnerabilities long after an application has been deployed in a production environment. By then, however, most developers have moved on to other projects, so remembering how a specific piece of code was developed and by whom can be a significant challenge, he noted.

    There’s no doubt that when it comes to application security, a lot of progress has been made, but there is still much work to be done. The issue now is finding the best way to organize those efforts at a time when more applications are being simultaneously developed within organizations than ever.