devopsy.kinsta.cloud

Author: Mike Vizard

  • Buildkite Tightens CI/CD Integration With GitHub

    Buildkite Tightens CI/CD Integration With GitHub

    Buildkite has tightened the integration between its namesake continuous integration/continuous delivery platform and the GitHub source code repository.

    Fred Showell, a senior product manager for Buildkite, said the integration will accelerate application development efforts using the Buildkite application in GitHub. Any connected GitHub account can be selected as a source when creating Buildkite pipelines, he noted.

    Repository webhooks can also be created automatically, making it faster to kick off builds on a Buildkite pipeline from pull requests and then feed build information back into GitHub using the Status application programming interface (API).

    The bulk of organizations that employ Buildkite are software vendors that are continuously updating software-as-a-service (SaaS) applications. Most of those organizations routinely employ software artifacts that are stored in the GitHub repository.

    While GitHub continues to expand its DevOps capabilities in lockstep with its parent company Microsoft, Showell said there’s plenty of opportunity to extend capabilities of frameworks such as GitHub Actions, a set of APIs that DevOps tools can invoke to automate workflows.

    In general, Showell noted the number of SaaS-based platforms continues to expand as organizations embrace cloud applications to drive digital business transformation initiatives. Some of those efforts are now also expanding to include use cases involving edge computing initiatives, which is expanding the reach and scope of the Buildkite platform, he said.

    Of course, many enterprise IT organizations are now applying many of the same best DevOps practices employed by SaaS platform providers to their own application development efforts. Many of those organizations are realizing they are essentially software companies that also manufacture something or provide a service within the context of a specific vertical industry. As such, the number of organizations embracing best DevOps practices should substantially increase in the months ahead.

    Most of those organizations are already trying to manage a large portfolio of applications. Digital business transformation initiatives may allow them to in time retire some of those applications, but chances are good legacy applications will need to be maintained for some time until the digital process that has been put in place proves effective. It’s likely then that sound software development lifecycle management practices will be more relevant than ever.

    Competition among providers of CI/CD platforms to provide those capabilities is already fierce. Fueled by massive amounts of capital, providers of these platforms are expanding the scope of their offering to address a market that is expected to be valued in the tens of billions. Buildkite itself raised $28 million in additional funding last summer.

    It’s too early to say how the next phase of the DevOps platform wars might play out. However, it’s apparent that when it comes to DevOps platforms the level of investment required just to keep pace is substantial.

  • Gravitational Becomes Teleport to Streamline Access Management

    Gravitational Becomes Teleport to Streamline Access Management

    Gravitational today announced it has changed its name to Teleport and launched the Teleport Unified Access Plane, an open source security gateway that comes in the form of a single binary that acts as a proxy for remotely accessing Linux servers and Kubernetes clusters.

    Teleport Unified Access Plane (UAP) for managing those gateways can be either deployed by an IT team or accessed via a cloud service managed by Teleport.

    Offerings included in Teleport UAP are Teleport Server Access, Teleport Kubernetes Access and Teleport Application Access, which combined enable IT organizations to lock down access to an IT environment.

    Company CEO Ev Kontsevoy said Teleport UAP gives engineers via a single login access to just the IT resources they have permission to access. That approach reduces the myriad credentials that IT organizations currently need to manage across multiple networks.

    Teleport automatically discovers all the resources in an IT environment along with the protocols needed to access them. It provides a unified audit log in a JSON format that enables IT teams to track what resources are being accessed when and by whom.

    Access control is becoming a bigger issue because the overall IT environment is becoming more complex as the infrastructure stack required to run applications grows. Teleport provides a way to streamline access management across servers, clusters and databases.

    That issue is also becoming a bigger challenge with IT personnel now working from home to help combat the spread of the COVID-19 pandemic, added Kontsevoy.

    That fine-grained approach to access will ultimately eliminate the need to rely on virtual private networks (VPNs) at a time when the network perimeter continues to dissolve. In effect, Teleport enables IT teams to move toward a more identity-centric approach to unifying the management of access, he said.

    In general, he noted, Teleport enables organizations to manage security as code rather than having to rely on separate overlays. As such, they can shift responsibility further left toward DevOps teams.

    It may take some time for IT teams to aggregate all the access control protocols they currently employ under a single proxy. However, Kontevoy said, the massive amount of time and effort spent managing access control today will drive organizations that have embraced best DevOps practices to look for ways to automate what today is a very manual process.

    The challenge, of course, is finding the time to achieve that goal. Teleport is betting a single binary in combination with a control plane accessed primarily as a managed service will make it too simple for DevOps teams to ignore. Regardless of approach, however, it’s clear there needs to be a better way to manage access within the sea of servers, databases and applications that now make up the typical IT environment.

  • OpsRamp Makes AIOps Platform Simpler to Configure

    OpsRamp Makes AIOps Platform Simpler to Configure

    OpsRamp today took the wraps off an update to its namesake artificial intelligence platform for IT operations (AIOps) that makes it easier for IT organizations to declaratively configure the software-as-a-service (SaaS) platform.

    Ciaran Byrne, vice president of product management for OpsRamp, said the goal is to enable IT teams to embrace AIOps without having to hire a small army of consultants.

    The OpsRamp Fall 2020 Release adds a revamped user interface that provides access to a set of wizards that discover the assets that make up a specific IT environment and subsequently automate the configuration process.

    OpsRamp has also added curated dashboards that enable IT teams to launch queries using the Prometheus Query Language (PromQL), developed for the open source monitoring tool being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The latest edition of the AIOps platform also adds monitoring capabilities for containerized workloads and Kubernetes clusters. In addition, IT teams can monitor other cloud services including AWS ECS, Azure Functions, Azure Hyperscale (PostgreSQL) and Azure SQL Managed Instance.

    OpsRamp has also added IT process automation capabilities that can be triggered by alerts, updates to resources or on a recurring schedule. An OpsQ bot now also recommends automation workflows. Alternatively, IT teams can require manager approval before a process is launched.

    In general, Byrne said the goal is to improve the mean time to value for an AIOps platform managed by a team of data scientists who have extensive experience applying machine learning algorithms to IT processes.

    Efforts to automate IT have increased sharply now that most IT staff are working from home to help combat the COVID-19 pandemic. However, few organizations are able to set up and configure an AIOps platform in an on-premises IT environment on their own, especially now that outside consultants are no longer being invited to spend weeks working on projects, Byrne said, noting a SaaS-based approach makes it possible for IT organizations to take advantage of an AIOps platform while continuing to work from home.

    In general, the downturn in the economy has increased interest in AIOps because IT staff ranks are either remaining the same or shrinking even as IT environments continue to become more complex. Longer-term, Byrne said it’s not clear that hiring Level 1 and 2 IT administrators will be sustainable as AIOps platforms continue to automate processes.

    DevOps teams, of course, are especially keen to automate as many IT processes as possible, which is why there is a high correlation between organizations that have embraced DevOps and early adopters of AIOps.

    It may still be a while before AIOps is widely employed, but it’s now only a matter of time before machine learning algorithms are pervasive across IT. The challenge now is figuring out where the handoff between man and machine for managing IT ultimately will lie.

  • build.security Leverages OPA to Shift App Authorization Left

    build.security Leverages OPA to Shift App Authorization Left

    build.security today announced it has raised $6 million to launch a platform that promises to make it simpler for developers to centralize the management of authorization controls across multiple applications.

    The goal is to enable DevOps teams to shift control over application authorization further left to advance adoption of best DevSecOps practices using a platform based on the open source Open Policy Agent (OPA) software, which enables developers to create policies that manage authorization as code.

    Company CEO Amit Kanfer said every application deployed today requires its own unique set of authorization requirements based on roles, permission models and complex hierarchies along with relevant identity, resource and context attributes. Streamlining that process requires a platform that provides a uniform approach to managing that process that is more flexible than existing approaches based on either role-based access controls (RBAC) or attribute-based access controls, said Kanfer.

    Otherwise, developers are required to build an array of complex authorization models, policy engines and enforcement points themselves to manage identities, resources and attributes. Those tasks are not only time-consuming but also highly prone to errors that leave applications vulnerable to cyberattacks. he said.

    OPA makes it easier for developers to implement fine-grained access controls using a drag-and-drop interface that decouples authorization policy from code. That approach makes it easier to declaratively make changes and updates to applications over time as more end users are added or removed, Kanfer said.

    Designed to be deployed on-premises or in the cloud, the authorization platform also provides integrations with identity providers, databases and other services that expose application programming interfaces (APIs). It will also automatically generate policy suggestions based on the runtime interactions between services.

    Historically, authorization to access applications has been poorly managed because it’s been difficult to manage manually. Business leaders are supposed to inform IT organizations when to add and delete end users. In practice, they often forget, resulting in end users still having access to applications long after they have left the company. Even when IT is informed that an end user has left a company, it can take a day for the end user to be deleted from all the applications they have been granted access over the span of their career at that organization. Shifting control over authorization left toward developers creates an opportunity to automate a process that is often haphazard at best.

    It also should allow organizations to reduce their dependency on Microsoft Active Directory (AD), which many organizations have employed to manage access to applications and files in Windows environments in favor of a centralized approach that can be more broadly applied to multiple applications.

    Authorization management may not always be at the top of the IT management agenda. However, at a time when cybercriminals are stealing credentials with more abandon than ever, the time may have finally come for IT organizations to reconsider how those credentials are managed and updated. After all, based on the number of data breaches involving stolen credentials, current approaches are clearly not working.

  • Dynatrace Adds to Observability Capabilities

    Dynatrace Adds to Observability Capabilities

    Dynatrace today announced it has extended its observability capabilities via the launch of its PurePath 4 platform that adds support for the open source OpenTelemetry framework and W3C Trace Context specification for distributed tracing.

    In addition, the company is adding support for service meshes and serverless computing frameworks running on Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform.

    Alois Reitbauer, chief technology strategist for Dynatrace, said by adding support for open source tools to gather metrics, the total cost of achieving observability is being driven lower at a time when a transition to microservices-based applications is making the need to observe applications more crucial than ever.

    Dynatrace is adding support for OpenTelemetry and W3C Trace Context to a platform that has already supported distributed tracing since 2006 via its OneAgent software. Dynatrace is now becoming the latest provider of observability tools to embrace open source software to instrument applications as part of an effort to focus engineering efforts on advancing analytics capabilities.

    In the case of Dynatrace, those capabilities are driven by a Davis artificial intelligence (AI) engine embedded within the Dynatrace platform and SmartScape continuous topology mapping technology. The combination of those technologies makes it possible for Dynatrace to automatically discover changes to IT environments and surface the root cause of any issue that might arise, noted Reitbauer.

    Despite being widely available via application performance monitoring (APM) tools, distributed tracing is now coming into vogue. As application environments become more complex, there is a need to more deeply analyze events to discover the root cause of an outage or performance issue.

    The challenge has been that applying that level of instrumentation to applications has been costly. Many IT organizations have historically limited the use of APM tools to their most mission-critical monolithic applications. It’s also worth remembering that three-tier applications are easier to troubleshoot because the potential source of any issue is probably already well-known to the IT team, he added.

    Given all the dependencies that exist between microservices, however, Reitbauer said the need to understand what’s occurring within an application environment more deeply is becoming a bigger requirement.

    Observability, of course, has always been a core tenet of best DevOps practices. The level of observability depth required by DevOps teams has tended to vary widely. Increased adoption of microservices-based applications invariably will push more IT organizations toward embracing DevOps at increasing levels of sophistication and maturity.

    In the meantime, the observability wars among platform providers are intensifying. There’s no shortage of options to achieve that observability, so it will be up to each IT team to decide which platform might best suit their DevOps workflow. Dynatrace is betting that as more applications become instrumented, deciding which platform to employ will shift closer to the AI capabilities that enable issues to be resolved issue faster regardless of what type of application is deployed rather than the capabilities of the agent software employed to instrument applications.

  • Puppet Makes IT Automation More Accessible

    Puppet Makes IT Automation More Accessible

    Puppet today launched two efforts to make its IT automation platforms accessible to a broader range of IT teams.

    The Puppet Enterprise Platform integrates the company’s flagship Puppet Enterprise offering with Relay by Puppet, the company’s existing event-driven automation platform.

    At the same time, Puppet announced Puppet Connect, a self-service platform for executing tasks on-premises or in the cloud without requiring any agent software to be installed, managed or updated. Based on the open source Bolt orchestration engine, Puppet Connect is a commercial platform scheduled to be available in early 2021.

    Puppet CTO Deepak Giridharagopal said Puppet Enterprise Platform is unique in that it allows IT teams with varying levels of skill and expertise to employ the same platform to automate IT operations. It is the first to support task-driven orchestration, model-driven infrastructure as code and application programming interface (API) event-driven workflows in a single platform, he said, noting as such, Puppet Enterprise Platform is designed to enable organizations to better approach IT management as a team sport.

    The company is also providing access to thousands of pre-built automation modules and hundreds of API integrations for Puppet Enterprise Platform that can be downloaded from the Puppet Forge repository.

    Puppet Connect, meanwhile, is designed to make it easier for IT teams to build self-service portals more consistently. IT teams regularly build self-service portals to automate tasks to help reduce the number of help desk requests generated by end users. According to the “2020 State of DevOps Report” published by Puppet, 63% of respondents have at least one self-service internal platform. Puppet Connect streamlines the process of creating a self-service framework using the orchestration capabilities of Bolt, said Giridharagopal.

    Interest in automating IT tasks has risen sharply as IT teams cope with more end users working from home to limit the spread of COVID-19. In some cases, the size of the IT staff has been reduced because of the economic downturn brought on by the pandemic. Regardless of motivation, IT staffs that have varying levels of skills all need to find an easier way to automate tasks. The Puppet platforms provide a more accessible layer of abstraction to enable IT teams to automate tasks spanning multiple platforms, noted Giridharagopal.

    It may be a while before IT teams of all sizes automate every IT task; however, the more those tasks become automated the further down the path they move toward embracing best DevOps practices. Those organizations may not deliberately start out to embrace DevOps, but once an IT team automates its first set of tasks it’s not long before other manual tasks become automated.

    Of course, there’s no shortage of IT automation frameworks to standardize on. At this point, the more important thing for many IT teams is to simply get started at a time when no one knows for sure how many years the impact of the COVID-19 pandemic will be felt.

  • Scalyr Adds Hindsight Service to IT Analytics Portfolio

    Scalyr Adds Hindsight Service to IT Analytics Portfolio

    Scalyr has moved to reduce the cost of observability by enabling DevOps teams to retain log and event data in S3-compatible object storage systems than use a pay-per-TB-scanned pricing model to interrogate that data.

    Company CEO Christine Heckart said the Scalyr Hindsight service will make it possible for IT teams to more cost-effectively store historical event data if they so choose. This option comes at a time when they are under pressure to reduce the total cost of IT as a result of the economic downturn brought on by the COVID-19 pandemic.

    Scalyr Hindsight complements an existing real-time log analytics platform that Scalyr provides via a software-as-a-service (SaaS) platform. However, IT teams no longer are required to store data in the Scalyr cloud platform itself.

    Heckart said collectively Scalyr Hindsight and the company’s core log analytics platform provide an alternative to custom log analytics platforms based on open source Elasticsearch, Logstash and Kibana (ELK) software that can be difficult to maintain and scale. The core Scalyr log analytics cloud service, built on top of a columnar database, is now capable of ingesting more than 200TBs of data per customer per day in real-time at a price that can be lower than $5 per GB. The Scalyr Hindsight platform only charges fees when data stored is actually scanned.

    With the rise of microservices, analytics that can be applied broadly at reasonable price points has become a major issue. Previously, IT teams might have limited their usage of analytics tools to their most important monolithic applications. Because of the way many of those platforms priced their software, IT teams found it necessary to also limit the amount of data that might be stored in these platforms. Given the complexity of microservices-based applications that tend to have lots of dependencies, an observability platform that provides an affordable approach to gaining insights into modern application environments is now required.

    Heckart said Scalyr also offloads the management of the observability platform from the internal IT team. Given the massive amounts of data that needs to be stored and managed, Heckert noted it’s not uncommon for data lakes that IT teams manage to become swamps simply because there are no dedicated resources to manage them. It’s not clear how many IT teams might be willing to give up managing their own observability platforms in favor of only needing to access analytics tools, but the one thing that is clear is the cost of support open source platforms tends to climb over time. IT teams may find an ELK-based approach initially sufficient only to discover the platform needs to be replaced as the number of applications that need to managed starts to increase.

    Regardless of who manages the underlying log analytics platform, it’s no longer feasible for most internal IT teams to manage modern application environments without the aid of an advanced set of analytics applications.

  • strongDM Extends Access Management as Code Efforts

    strongDM Extends Access Management as Code Efforts

    strongDM today announced it has added an application programming interface (API) and software development kits for Go, Java, Python, Ruby and other programming languages for involving the single sign-on (SSO) capabilities of its infrastructure access management platform, in addition to adding support for command-line interfaces (CLIs) exposed by cloud service providers.

    At the same time, strongDM has integrated its namesake platform with third-party secrets management tools such as Vault from HashiCorp as well as platforms such as PagerDuty’s incident management platform and Slack messaging tools.

    The access management platform can now also support any internal web applications and database platforms including DB2i, CitusDB, Oracle 11g, 12c, and 19c, Sybase ASE and IQ.

    Finally, the company has made enhancements to existing protocols including Mongo Atlas, SSH certificate-based authentication and SSH port forwarding, and added support for Windows remote desktop protocol (RDP) network-level authentication (NLA) and integration with other SSO tools such as Auth0 and VMware Identity One.

    Company CEO Elizabeth Zalman said strongDM, unlike other SSOs, is designed from the ground up to be incorporated within a developer’s workflow. The goal is to enable developers to manage access as code in the same way they manage infrastructure as code, she said.

    In general, access management is usually the last stage of automation to be addressed by DevOps teams. However, with most DevOps teams now working from home to help combat the spread of the COVID-19 pandemic, there has been a sharp rise in the number of organizations seeking to automate access management as an alternative to managing virtual private networking (VPN) software, said Zalman.

    Historically, many IT organizations have relied on Microsoft Active Directory (AD) or the Lightweight Directory Access Protocol (LDAP) to manage access. As an alternative, strongDM has been making a case for its platform since 2015. The company recently raised an additional $17 million in funding and claims to have more than 100 customers using its platform in production environments, including Peloton, SoFi and Betterment.

    It’s unclear whether DevOps teams are extending their existing workflows to include access management. However, as DevOps teams grow weary of using custom scripts to extend those workflows, Zalman said eventually they’ll look for a more consistent approach to automating access management. A major factor in that decision is the fact that most employees won’t be coming back to the office in the same numbers they did prior to the pandemic, even after a vaccine is discovered and distributed.

    At the same time, more DevOps teams are moving to embrace best DevSecOps practices, which inevitably will encompass access management. In fact, as part of that transition, it’s probable most DevOps teams will conclude the way access is currently managed is simply too antiquated to continue.

  • Intel Readies Gold oneAPI Toolkit

    Intel Readies Gold oneAPI Toolkit

    Intel this week said a gold image of its oneAPI for unifying infrastructure as code will be available starting next month.

    Jeff McVeigh, vice president of data center XPU products and solutions Intel, said the goal is to provide a standard application programming interface (API) for accessing the compute capabilities of any processor regardless of what company may have manufactured it.

    The availability of oneAPI coincides with the launch of a series of dedicated graphical processor units (GPUs) from Intel that are intended to supplant the dominance of NVIDIA.

    McVeigh said Intel is encouraging cloud service providers and builders of systems deployed on-premises to embrace oneAPI as a way to standardize the way any CPU, GPU, field-programmable gate arrays (FPGAs) or any other class of processor is incorporated within an environment rather than continuing to relying on programming models such as CUDA from NVIDIA that are designed for only one class of processor.

    The average developer may never interact directly with a low-level oneAPI, but McVeigh said a standard set of APIs at the processor level should ultimately serve to reduce the total cost of IT as developers invoke multiple classes of processors with their applications.

    Intel, of course, also sees oneAPI as an opportunity to usurp NVIDIA as the dominant providers of GPUs that are now widely employed in the cloud to train machine learning models. While Intel CPUs are still employed most to run the inference engine that enables those models to be deployed in a production environment, NVIDIA is now seeking to leverage its success in the cloud to provide either GPUs that can be used to run inference engines or processors from Arm. NVIDIA is in the process of acquiring Arm, a provider of processors that are widely employed in embedded system and edge computing environments.

    Microsoft and the open source contributors that oversees the development of the TensorFlow framework for building artificial intelligence (AI) applications have thus far endorsed oneAPI. In addition, the University of Illinois Beckman Institute for Advanced Science and Technology this week announced it is creating a oneAPI center of excellence (CoE) that will extend the oneAPI programming model to nanoscale molecular dynamics (NAMD) environments that employ code to simulate large biomolecular systems.

    Previously, the University of Stockholm (SeRC) has committed to adding support for oneAPI to GROMACS, open source software for building simulations of biomolecules, while Heidelberg University is working on adding support for oneAPI to GPUs from other vendors. Intel is also making available software based on oneAPI that is compatible with CUDA, added McVeigh.

    Finally, Intel also announced this week that the Intel Implicit SPMD Program Compiler (ISPC) will run on top of oneAPI Level Zero, which provides low-level, direct-to-metal interface. Written in X, ISPC is used to accelerate Intel OSPRay ray tracing engine on Intel CPUs. Support for ISPC on Intel GPUs is forthcoming.

    It may a little while before DevOps teams see the benefits of oneAPI; however, as it becomes easier to mix and match processors as they best see fit, chances are it’s because the overhead on the back end has been substantially reduced using a common API.

  • Survey Sees Pace of IT Investments Continuing

    Survey Sees Pace of IT Investments Continuing

    A survey based on responses from 230 IT operations and DevOps executives in the U.S. and UK with at least 500 employees and $5 million in annual IT budgets published this week indicates large enterprises are continuing to make IT investments despite the downturn in the economy brought on by the COVID-19 pandemic.

    In the second and third quarters of 2020, 60% of IT leaders significantly or moderately increased their annual technology budgets, according to the survey results. In contrast, 22% reported their organizations significantly or moderately reduced their IT spending during the same period.

    Conducted by OpsRamp, a provider of a software-as-a-service (SaaS) platform for managing IT operations, the survey also finds nearly two-thirds of respondents (63%) have accelerated or maintained digital transformation initiatives in response to the economic downturn brought on by the COVID-19 pandemic.

    Cost optimization, however, remains a major focus. IT leaders report they are using self-service tools (60%), open source software (51%) and rationalizing IT vendors (51%) to contain costs.

    Financial managers who can help IT teams present a compelling business case for technology investments are the highest hiring priority (54%), followed by senior IT leaders who can execute digital transformation programs (47%) and cloud operators (44%).

    From a technology perspective, top priorities are information security and compliance (59%), remote work and collaboration (55%), public and multi-cloud infrastructure (50%) and monitoring and management (42%).

    More than half of respondents (57%) said they are also employing artificial intelligence for IT operations, while 50% have adopted both digital experience monitoring (50%) and network performance monitoring and diagnostics (50%) tools.

    Bhanu Singh, senior vice president of engineering and DevOps for OpsRamp, said the survey makes it clear that IT spending in large enterprises is rebounding from last spring when the pandemic first hit. Overall, IT spending for the year will still be down from its historic highs, but larger organizations are prioritizing technology projects that enable the business to adjust to fundamentally different market conditions, he said. Most of those projects involve applications being deployed on cloud computing platforms using best DevOps practices, noted Singh.

    Less clear is what the level of investments will be following the U.S. presidential election. Many business leaders are waiting to see what the stimulus spending might be approved by the new administration and Congress. That may not be known until a runoff in December for two Senate seats in Georgia, which will determine which political party controls the U.S. Senate.

    In the meantime, it does appear that larger organizations are investing in IT projects they expect will either enable them to remain competitive or pick up market share at the expense of smaller rivals that may not have as many resources to invest in IT. Historically, smaller organizations have tended to be more nimble when it comes to leveraging IT to gain a strategic advantage. However, in an era when cash flow is constrained many smaller organizations may have to be more judicious when it comes to spending on IT.

    Regardless of the outcome, the one thing that is certain is IT has never played a more crucial role in determining not only which organizations survive but ultimately thrive.