Category: Features

  • Sysdig Adds Cloud Access Controls to DevSecOps Platform

    Sysdig Adds Cloud Access Controls to DevSecOps Platform

    Sysdig announced today that it is adding a Cloud Infrastructure Entitlements Management (CIEM) capability to its Secure DevOps platform as part of an effort to better enforce least-privilege access within the context of a larger zero-trust approach to cybersecurity.

    Nick Fisher, vice president of product marketing for Sysdig, said this addition to the company’s core DevSecOps platform for securing cloud services will prevent organizations from inadvertently providing individuals with excessive permissions that result in cybercriminals gaining widespread access to applications and systems when the credentials those individuals have are compromised.

    A recent report from the Sysdig Threat Team found that nearly 80% of users have excessive entitlements due to overly permissive policies. Similarly, Gartner forecasted that 75% of security failures will result from inadequate management of identities, access and privileges by 2023, up from 50% last year.

    The CIEM offering from Sysdig enables cybersecurity and IT operations teams to resolve that issue across multiple clouds in under two minutes once installed, said Fisher. The Sysdig platform achieves that goal by making it possible to leverage templates as a set of infrastructure-as-code (IaC) tools that enable DevOps teams to integrate security within their existing workflows.

    CIEM also simplifies access control auditing by making it easier to discover who in the organization has permission to access cloud infrastructure services. As part of that capability, Sysdig also provides out-of-the-box compliance policies and access to reports that can be generated on-demand to surface a detailed audit trail of all cloud permission changes.

    Finally, CIEM will generate an alert any time someone is denied permission to access a cloud resource, noted Fisher. That capability provides cybersecurity teams with an opportunity to then review what permissions might need to be granted or whether a bad actor is attempting to compromise a cloud environment.

    Most of the cloud security issues that organizations encounter can be traced back to developers that have misconfigured one cloud resource in a way that allows cybercriminals to access, for example, a port that has been left open. CIEM extends the Sysdig platform so that, in addition to scanning for vulnerabilities and misconfigurations, it can detect and respond to attacks across containers and virtual machines deployed in a cloud environment.

    Thanks, in part, to recent high-profile breaches of software supply chains, there’s a lot more focus on cloud security as the rate at which applications are being deployed on these platforms steadily increases. While the cloud platforms themselves are fundamentally secure, the challenge most organizations are encountering is they have immature processes in place for securing the workloads running in the cloud.

    Of course, being able to limit access to those cloud resources is only the first step toward implementing a set of DevSecOps best practices to ensure the integrity of the cloud platforms being employed. Unfortunately, too many developers still tend to assume the cloud service provider is taking steps to secure workloads on their behalf. In reality, cloud service providers have adopted a shared responsibility model that, from their perspective, limits their cybersecurity responsibility to the core infrastructure they provide. It’s up to the DevOps team responsible for the workloads running on those platforms to figure out how to secure the rest of that IT environment.

  • Codenotary Uses Immutable Database to Verify Software Artifacts

    Codenotary Uses Immutable Database to Verify Software Artifacts

    Codenotary today unfurled a free notarization and verification service for open source artifacts and containers to enable IT organizations to track the provenance of the components that make up their applications.

    Dennis Zimmer, Codenotary CTO, said the Community Attestation Service is based on an immutable open source immudb database that cryptographically attaches an identity to each software artifact. Additionally, the Community Attestation Service can be integrated with the continuous integration/continuous delivery (CI/CD) platform that DevOps teams employ to build and deploy applications using those artifacts, added Zimmer.

    The identities created by the Community Attestation Service can then be used to verify a software bill of materials (SBOM) that will soon be required by not only every U.S. federal agency but also organizations that are trying to better secure their software supply chains in the wake of a series of high-profile breaches, said Zimmer. The Community Attestation Service will also encourage developers to reuse source code, builds, repositories and Docker container images that they know they can reliably employ within a zero-trust IT environment, noted Zimmer.

    Some developers have, of course, been signing their code for years, while others simply didn’t see the need. With its free service, Codenotary is trying to expand the pool of developers that will not only sign their code but do so in a way that can’t be tampered with later, noted Zimmer.

    Unlike blockchain databases that are employed to provide immutable records of transactions, Zimmer said the immudb platform was created to provide an immutable database capable of processing millions of transactions per second. That database is also being used to build a wide range of applications that go beyond the scope of the Community Attestation Service so DevOps teams can have confidence in the viability of the underlying platform, noted Zimmer.

    In the wake of some troubling security breaches, it’s now only a matter of time before security teams start to ask more difficult questions about how applications are built. Developers could wait for those cybersecurity teams to eventually create policies that DevOps teams would be required to enforce, or they can more proactively embrace a service that tracks the provenance of code that hopefully, has been scanned for all known vulnerabilities. In fact, it may not be too long before organizations decide not to employ code that has not been verified. There may even be a backlash against using code that is unsigned that could result in many existing applications either being updated or outright replaced.

    On the plus side, Zimmer noted there’s a lot more awareness of the need to sign code. The challenge is most organizations don’t have the processes in place. Fortunately, there’s a lot more interest in DevSecOps best practices to ensure the integrity of code. However, awareness of the need for something doesn’t always result in concrete actions being taken unless DevOps teams can simply make it easy for developers to do the right thing. After all, it’s not as though developers want to employ insecure code, it’s just that they are often simply too busy to make sure every last artifact employed is verified as secure.

  • CircleCI Adds Test Insights Module to CI/CD Platform

    CircleCI Adds Test Insights Module to CI/CD Platform

    CircleCI today announced it has added a Test Insights capability to its continuous integration/continuous delivery (CI/CD) platform to make it easier to identify tests that are either taking too long to run or simply exhibiting some form of anomalous behavior.

    Dawit Gebregziabher, product manager for data insights at CircleCI, said Test Insights is designed to make it simpler for IT organizations to shift more responsibility for testing further left toward DevOps teams that identify issues as applications are being built on the CircleCI platform.

    Each organization will still need to determine what tools to employ to create a test, but once it starts running, the Test Insights module will be able to surface the tests that, for one reason or another, are problematic to execute, he added.

    Test Insights, for example, will flag flaky tests and identify the longest-running and most failed tests. It will also enable DevOps teams to identify performance changes over time to pinpoint changes in application behavior.

    Finally, DevOps teams can track benchmarks at the organization, workflow and job level, as well as metrics for mean-time-to-recovery, throughput and durations for 24-hour, 7-day and 30-day windows of time.

    The challenge DevOps teams often encounter is a test may be run even when it’s apparent the application will fail. Test Insights makes it feasible for DevOps teams to manage the testing process much more efficiently, said Gebregziabher.

    It’s not quite clear how far testing is shifting left. There’s no doubt more developers are testing their code before it is loaded into a production environment to hopefully avoid issues that will only become more expensive to fix later. Other organizations have dedicated software engineers to create testing processes that are tightly integrated with CI/CD pipelines, noted Gebregziabher.

    However, many large enterprise organizations continue to also hire and retain dedicated application testing teams that are trained to identify issues that might either adversely affect the end-user experience or run afoul of compliance requirements.

    Regardless of who does what kind of testing, however, it’s clear there’s a lot more focus on addressing issues earlier when they are much less expensive to fix. The goal should be to make it easier to optimize when tests should run based on probabilities that ultimately reduce the time wasted running tests that either will fail or are, for one reason or another, flawed.

    Developers, of course, are all equally committed to running tests, so organizations will need to determine for themselves how far to shift that process left. However, at the very least, DevOps teams should be able to identify patterns that might indicate there is a need for additional developer training. After all, it’s unlikely a developer is going to understand there’s an issue until some meaningful metrics are shared.

    In the meantime, the more testing that occurs at a faster rate the more likely it becomes that an organization can provide a superior application experience the first time—and every time—an application is built and deployed.

  • InfluxData Simplifies AppDev for Time-Series Databases

    InfluxData Simplifies AppDev for Time-Series Databases

    During an online InfluxDays North America 2021 Virtual Experience conference, InfluxData today expanded the capabilities of its application development tools using its Flux query language on top of its time-series database.

    The company has enhanced its InfluxDB Notebooks tool to enable developers to quickly set up an alert, build a task or write a Flux script without having to leave their browser.

    In addition, developers can compose Flux queries to be included in client code that can then be autogenerated within applications written in more than 10 different programming languages. There is now also an extension of Visual Studio Code that makes it simpler to create Flux scripts, query InfluxDB, create Flux tasks and manage buckets.

    Finally, InfluxData added the ability to explicitly define the schema of a bucket within InfluxDB to safeguard against unwanted changes; an application programming interface (API) tool, dubbed API Invocable Scripts, to make it simpler to define parameterized Flux scripts and the ability to send notifications from Alerta, WebEx Teams and ServiceNow to the InfluxDB Cloud platform.

    Tim Hall, vice president of product for InfluxData, said the overall goal is to reduce the level of complexity developers encounter when building applications that run on a time-series database. Many of those applications are now core to digital business transformation initiatives that require data to processed and analyzed over time. In effect, time-stamped data adds yet another dimension to any application, noted Hall.

    Time-series databases are typically employed by either an independent software vendor (ISV) to create, for example, a monitoring application, or an enterprise IT team that is building an application that lends itself to data that needs to be tracked over some period of time.

    In most instances, applications built on a time-series database are being deployed alongside, for example, relational databases that are generally invoked using SQL. As a result, InfluxDB has made it simpler to join data across relational and time-series data via SQL integrations the company added to Flux, noted Hall.

    DevOps teams, of course, are not always anxious to add another database they need to support, but as applications continue to evolve, more organizations are employing a mix of purpose-built databases to develop applications as the volume of data that organizations are trying to manage and analyze continues to grow.

    In the longer term, it’s clear time-series databases will play a critical role in driving digital business transformation initiatives. The challenge most organizations are now struggling with is determining which of these initiatives to prioritize. A time-series database coupled with the developer tools provided by InfluxData will enable IT teams to build the applications that drive those processes much faster, said Hall.

    In the meantime, IT teams would be well advised to at least become more familiar with time-series databases, because knowing the time when an event occurred is becoming a bigger application requirement. After all, knowing when something happened is now just as important as the record of what actually occurred.

  • Eclipse Foundation Launches Open Source Edge Computing Initiative

    Eclipse Foundation Launches Open Source Edge Computing Initiative

    The Eclipse Foundation in collaboration with the OpenAtom Foundation today launched the Oniro project and working group to create an independent implementation of OpenHarmony, an open source operating system.

    Created by the OpenAtom Foundation, the OpenHarmony operating system is based on a HarmonyOS originally created by Huawei. It is designed to support multiple kernels but uses the Linux kernel if a device has a large amount of memory.

    It also includes a DSoftBus capability for integrating physically separate devices into a single logical super device that enables one device to control others in addition to facilitating data sharing among distributed devices.

    Huawei, in collaboration with Linaro, Seco, Array, NOITechPark and Synesthesia, has also been contributing to a continuous integration/continuous delivery (CI/CD) platform that is part of the larger Oniro project.

    Mike Milinkovich, executive director of the Eclipse Foundation, said the goal is to provide a common layer of abstraction platform that can be employed across a wide range of embedded systems that are typically being deployed at the network edge. That level of fragmentation surrounding platforms that don’t provide a lot of significant differentiated value only results in slowing down the deployment of applications, noted Milinkovich.

    As part of that effort, a dedicated working group has been created to provide a vendor-neutral structure through which contributions to the project can be made with greater transparency. The OpenAtom Foundation was created in Beijing last year with the support of Alibaba, Baidu, Huawei, Inspur, Qihoo, Tencent and China Merchants Bank. The alliance with the Eclipse Foundation brings more traditional governance rules for building open source software to the OpenHarmony project.

    In general, DevOps teams should be tracking the development of proposed de facto standards for edge computing platforms closely. More applications are now being deployed on edge computing platforms that make it possible to process and analyze data at the point where it is being created and consumed. The DevOps challenges associated with building and deploying applications across a highly distributed computing environment are considerable.

    In many cases, organizations that never really attempted or mastered continuous delivery of applications and their associated updates will be sorely challenged when it comes times to support hundreds, possibly thousands of applications, running on embedded systems based on a wide range of processors.

    In addition, many of those same DevOps teams will start to see traditional batch-oriented applications being superseded by applications based on event-driven architectures that process data in near-real-time. In most cases, those applications are going to be built on microservices-based applications that are typically more challenging to build and deploy than a traditional monolithic application.

    Arguably, IT organizations are on the cusp of a new era of IT that assumes most organizations have not only mastered DevOps best practices but are now also embracing DevSecOps best practices to secure those application environments.

    It may be a while before the open source software that will be developed by the Orino project finds its way into production environments, but it should be apparent that reducing complexity at the network edge should be a high priority for all DevOps teams heading into 2022.

  • Mona Allies with New Relic to Converge MLOps and DevOps

    Mona Allies with New Relic to Converge MLOps and DevOps

    New Relic and Mona, a provider of a platform for monitoring the models used to provide artificial intelligence (AI), announced today they have formed an alliance to help bridge the divide between DevOps teams and data scientists.

    Most data science teams today build models based on machine learning algorithms using operations platforms that enable them to manage the life cycle of those models, otherwise known as MLOps. Mona provides a set of monitoring tools that are employed to detect drift that can occur as some of the underlying assumptions that were made when building the model are discovered to be less accurate or when new data sources become available.

    New Relic, meanwhile, provides an observability platform that is widely employed by DevOps teams to monitor and observe an IT environment. IT teams that have access to New Relic One can now also view dashboards provided by Mona that detect anomalies involving data integrity issues and model performance issues.

    The integration also enables users to create custom Mona alerts that are shared via the New Relic Alerts and Applied Intelligence service. That capability reduced the volume of alerts generated by correlating anomaly detection to specific IT events.

    Yotam Oren, CEO of Mona, said that level of integration provides a bridge between DevOps and data science teams that typically have widely divergent cultures.

    Incorporating AI models within applications is the next major DevOps challenge. There is almost no application being built today that won’t incorporate some level of AI. The challenge is AI models are built by data scientists that employ a wide range of machine learning operations (MLOps) platforms to construct those models. The pace at which those AI models are developed, deployed and updated varies widely. However, all organizations will eventually need to confront the problem of how to insert AI models within applications both before and after they are deployed in a production environment.

    There are two schools of thought about how best to achieve that goal. The first is to assume an AI model is just another type of software artifact that can be managed as part of an existing DevOps process. In that context, there may be no need for a separate MLOps platform. Conversely, proponents of MLOps contend AI models are built using pipelines created by data engineering teams working in collaboration with data scientists. Developers only become part of the process when an AI model needs to be inserted into an application environment. AI models may ultimately need to be incorporated within an application, but the proponents of MLOps platforms maintain building models requires a unique platform to manage a software artifact that is built by training algorithms.

    It’s only a matter of time before compliance issues force some form of convergence between DevOps and MLOps teams, noted Oren. The longer it takes to address both the cultural and technical issues that are required to meld DevOps and data science teams, the more challenging it will become as both teams are set in their respective ways.

  • Authentication in Serverless Apps—What Are the Options?

    Authentication in Serverless Apps—What Are the Options?

    Serverless applications are growing in popularity among DevOps engineers. They provide a convenient, predictable way to run simple processes like CI/CD builds or automation scripts with no need to stand up infrastructure. They are also commonly used to deploy microservices. However, serverless applications present unique security challenges, one of them being authentication. 

    How can you ensure services communicate with other services or with the outside world only if authorized to do so? How can you ensure users can conveniently and securely access multiple instances of a serverless function?

    In serverless applications, there are many components interacting—not only end users and applications but also cloud vendors and applications. This is why common authentication methods, such as single factor, two-factor and multifactor authentication offer only a bare minimum foundation. Serverless authentication requires a zero-trust mentality—no connection should be trusted, even communication between internal components of an application should be authenticated and validated.

    To properly secure serverless authentication, you also need to use authentication and authorization protocols, configure secure intraservice permissions and monitor and control incoming and outgoing access. In this article, I’ll cover all these briefly and demonstrate a practical example—setting up authentication in the AWS serverless ecosystem.

    What Are the Most Common Authentication Methods?

    Here is an outline of the standard authentication methods used by most applications.

    Single-Factor Authentication

    Also called primary authentication, this is the most prevalent and straightforward type of authentication. Single-factor authentication asks for only one authentication method; for example, a security PIN, password or PIV card, to gain access to a service or system.

    These methods are solid in relation to familiarity and usability; however, they are generally linked to poor security and credentials can readily be stolen or guessed through phishing, data breaches or by using keyloggers.

    Two-Factor Authentication

    Two-factor authentication (2FA) adds another layer of intricacy, as it needs an additional factor to authenticate a user’s identity. Common verification methods include one-time passwords, tokens created by a registered device or PIN numbers. The presence of two verification methods significantly improves your level of security. 

    Multifactor Authentication

    This is the most complex process of authentication. It makes use of two or more independent factors to verify a user’s identity and authorize access to the system. In common scenarios, multifactor authentication (MFA) makes use of at least two of the following factors:

    • A PIN or password (something known)
    • Security token or mobile phone (something owned)
    • FaceID or fingerprint (something you are)
    • Location information, typing speed or the like (something you can do) 

    Authentication and Authorization Protocols

    An authentication protocol is a communication protocol that protects authentication data as it transfers between two entities. It enables a receiving entity to authenticate a connecting entity as well as authenticate itself to a connecting entity. This is achieved by declaring the type of information required for authentication and syntax.

    There are several types of authentication and authorization protocols, including:

    • Lightweight Directory Access Protocol (LDAP)—a software protocol that enables anyone to find data about individuals, organizations, and resources like files and devices within a network. LDAP allows this whether a user uses the public internet or a corporate intranet.
    • The Security Assertion Markup Language (SAML)—an open standard that allows sharing security information about authentication, authorization and identity across several different systems. 
    • Open Authorization (OAuth)—an open standard authorization framework designed for token-based authorization established over the internet. OAuth enables third-party services to use end-user account information without exposing user account credentials to the third party. This blog post can help you understand the differences between SAML and OAuth.
    • OpenID—enables authentication services and websites to exchange security information in a standardized manner. OpenID Connect—the newest version of the protocol—provides improved identity management, support for mobile application development, and interoperability. Learn more in the official OIDC FAQ.

    Secure Authentication on Serverless

    In serverless systems, there are four forms of access—four forms of communication that should be secured. An individual service won’t have to include all four, but the categories that it does feature should be well-protected. 

    Intraservice Access

    Creating a system by uniting serverless services has many advantages. However, you should consider the security of every additional service. 

    A service with excess permissions is a liability. Unnecessary permissions provide excessive access to data, actions or services, and a bug might let malicious users exploit that service and use its permissions to endanger your system. Hence, you should give careful consideration to the permissions required by each service.

    Effectively overseeing intraservice access permissions is often an overwhelming job. Luckily, all of the top-tier cloud providers have native access management services to make this task simpler. AWS has AWS IAM, Azure has Azure Active Directory and Google Cloud Platform (GCP) provides Cloud IAM. 

    Incoming Access

    A network is made accessible through a SaaS offering to external users. Access will be restricted, and every user will require the official credentials to achieve that access. 

    However, this brings up the same problems raised above—the secrets must be stored somewhere. You cannot manage how your users access and store the credentials that you provide them with; therefore, you should assume that their credentials are not being kept securely and that they may be compromised at any point.

    This would appear to be an unmanageable issue, but there is a solution—to provide temporary credentials. This means that even if the secrets were revealed, they would expire shortly afterwards. 

    The cloud providers have services for this. Google Cloud IAM offers short-lived credentials, AWS IAM lets you create temporary credentials and Azure Active Directory lets you configure expiration dates for its signatures that have shared access.

    Outgoing Access

    When your organization’s serverless function has access to an external third-party service, it typically needs some sort of access credentials. These credentials need to be kept somewhere, but this storage is sometimes not as secure as it should be. Today, certain web applications continue to store secrets in databases, text files or hardcoded within the function code. 

    Cloud providers have services for securely retaining credentials and secrets. AWS has AWS Secrets Manager, Azure offers Azure Key Vault and GCP offers Google Cloud Secret Manager.

    End User and Customer Access

    The issues with this access type are akin to those for incoming access, though the solution is different. Whether in B2B or (particularly) B2C environments, not many users would put up with a requirement to continuously regenerate or refresh their credentials. Thus, another approach is needed. 

    The top-tier cloud providers all have a user authentication service—Azure Active Directory B2C, AWS Cognito and Google Identity Platform. You should also consider using third-party end-user verification, as this will help you avoid vendor lock-in. Third-party offerings also tend to offer a better developer experience. The most common third-party authentication providers are Okta and Auth0.

    AWS Lambda Authentication Solution Example

    Amazon Cognito offers managed authentication and authorization that can help you build secure web services in a serverless architecture. It is typically used in combination with Amazon API Gateway when building complex serverless applications that require permission control.

    For example, a payroll system may have a manager and two additional employees. The three employees can use their unique usernames and passwords to log into the payroll system—this process is called authentication.

    To retrieve paystubs, the three employees undergo role-based authentication:

    REST API GET /api/employee/{employee_id}/paystubs

    However, the permissions they get through this process should be unique to their needs. This may involve setting up the following conditions:

    • The manager can access the paystubs of all employees.
    • Employees cannot access the paystubs of other employees, only their own. 

    You can use AWS managed services to fulfill the above requirements and also meet future scaling needs.

    Image Source: AWS

    Step One: Setting Up AWS Lambda Authorizer

    The Lambda authorizer is the brain that centrally determines whether the requester has the relevant permissions for the requested API resources.

    Here’s how you can set it up:

    • Navigate to the Amazon API Gateway and create an authorizer.
    • Select the API resources you want to secure and assign the AWS Lambda function to them.
    • Configure your new AWS Lambda authorizer.

    Step Two: Building User Access Control List (ACL)

    It is crucial to establish user permissions and keep them up-to-date. The approach might differ according to your identity provider. Here is an example of the process if you use Amazon Cognito as your IDP. 

    1. Initialize the ACL

    Every user has a unique ID, so you can describe each user’s permissions by inserting items in the permission table in Amazon DynamoDB when you register with Cognito. The DynamoDB permission table stores the access control list for all users, although the implementation may differ based on the use case. 

    For this example, let’s use a single integer ID for each employee (the UUID strings in real applications tend to be much longer). 

    Manager:

    uuid: xxxx-xxx-xxx-xxxx-xxxxxxxx

    employee: {“allow”: [“/5”,”/5/*”]}

    paystubs: {“allow”:[1,2]}

    Employee 1: 

    uuid: xxxx-xxx-xxx-xxxx-xxxxxxxx

    employee: {“allow”: [“/6”,”/6/*”]}

    paystubs: {“allow”:[3,4]}

    Employee 2: 

    uuid: xxxx-xxx-xxx-xxxx-xxxxxxxx

    employee: {“allow”: [“*”],”deny”:[“/7”,”/7/*”]} // employee_id=7 is the manager of employee 2

    paystubs: {“allow”:[1,2,3,4]}

    2. Manage the Access Control List (ACL)

    The user permissions in a complex project are constantly changing. Employees can be promoted, demoted or terminated, with different access requirements depending on their changing roles. A new manager, for example, needs access to all pay stubs, which means you have to create a new pay stub object for every employee. 

    You can manage user permissions using the Amazon DynamoDB stream:

    Lambda functions can easily be integrated when you insert or modify items in the permissions table. A Lambda function might include code that modifies permissions according to changes to the role of a user.

    3. Build an ACL policy

    Once you’ve established an ACL with user permissions for every object, you should create an authorization policy document. The Lambda function authorizer uses this policy to evaluate whether user API requests should be allowed or denied.

    Amazon API Gateway offers a mechanism for evaluating policies internally. internal policy evaluation mechanism. The Gateway will automatically evaluate any policy JSON returned by the authorizer. For details about Lambda authorizer’s output standard, see this document.

    In this article, I covered the basics of authentication in serverless applications:

    • Single-factor authentication
    • Two-factor authentication
    • Multifactor authentication
    • Intraservice access
    • Outgoing access
    • Incoming access
    • A real-life example with Amazon Cognito

    I hope this will be of help as you gain a better understanding of authentication and authorization techniques that help secure serverless applications.

  • New Relic Acquires CodeStream to Close Observability Gap

    New Relic Acquires CodeStream to Close Observability Gap

    New Relic today revealed it has acquired CodeStream as part of an effort to make observability data more accessible to developers.

    Buddy Brewer, group vice president (GVP) and GM at New Relic said New Relic CodeStream makes application telemetry data collected via the New Relic One platform directly available within an integrated development environment (IDE), now including Microsoft VS Code. In addition, there are also integrations with Microsoft Teams and the GitHub repository.

    The goal is to close an observability gap that exists today between developers and IT operations teams that stems from a lack of developer visibility into production environments, he noted.

    Finally, New Relic is also announcing a core user pricing option for New Relic One that is aimed at developers. Starting at $49 per month, developers gain access to a core set of capabilities such as CodeStream integrations, errors inbox, logs, GraphQL application programming interfaces (APIs) and the ability to build and run custom applications on New Relic One.

    Peter Pezaris, CodeStream co-founder and CEO, now GVP and GM, strategy and developer tools at New Relic, said becoming part of New Relic will make it easier to bridge the divide that exists between most developers and IT operations teams by making it much easier for developers to see how their code is actually executing in a production environment. That capability will eliminate scenarios where code that runs just fine on a laptop executes poorly in a production environment, he added.

    Developers can also add inline code comments, trigger pull requests, request feedback and track issues directly from their IDE via integrations with a wide range of developer and communication tools. Developers also can switch from New Relic One into an IDE in a single click to navigate, triage and remediate production errors in the context of the code they wrote.

    New Relic is initially making New Relic CodeStream available for free—no credit card required—during a preview period that extends to January 12, 2022. Afterward, some of the more advanced features will require a paid user license once free tier limits are reached. Similarly, IT teams can also sign up for a free New Relic One account without entering a credit card to instrument their applications.

    As the responsibility for managing applications in production environments continues to shift left, Pezaris said it’s critical to give developers the tools they need to optimize code in a production environment.

    It’s not quite clear exactly how much responsibility for applications is shifting left toward developers; it differs from one organization to the next. Many are simply trying to reduce as much friction as possible between developers and their IT operations teams by providing more ‘connective tissue’ from New Relic that IT teams would have to otherwise write themselves, noted Pezaris.

    It may take some time to bridge the observability gap between developers and IT operations teams. Most IT operations teams are still coming to terms with what observability provides that existing IT monitoring tools don’t. Monitoring, at its core, focuses on predefined metrics to identify when a specific platform or application is performing within expectations. The metrics tracked generally focus on, for example, resource utilization. In contrast, observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without having to rely solely on a limited set of metrics that have been pre-defined.

    In theory, at least, that level of observability should ultimately reduce the total number of incidents an IT operations team needs to fix by identifying issues long before they become an actual disruption.

  • How do we Bring APIs Into Decentralized Web3?

    How do we Bring APIs Into Decentralized Web3?

    Blockchain-based applications are growing in popularity and underscore the decentralized Web3 movement. Simultaneously, web APIs are helping port data and reusable functionality across application types and are driving microservices development practices. So, how do we bring these two paradigms together?

    There are many cases where a blockchain-based smart contract may require data updates that only a web API request can fulfill. For example, perhaps the smart contract needs to access the latest price of a given stock ticker. Or, perhaps on-chain activities are contingent on the weather at a particular time. The ubiquity of APIs means there are thousands of helpful features available for the task at hand. Yet, integrating them into Web3 applications is tricky.

    Interestingly, the API3 Alliance is making strides to introduce an industry-standard solution to bring APIs to Web3 applications. API3 provides a vendor-neutral standard gateway to apply on top of existing APIs that helps them appeal to decentralized apps. The vendor-neutral API Alliance already consists of many APIs across various verticals and appears to be gaining more traction.

    I recently met with API3’s co-founder, Heikki Vänttinen, to learn more about the challenges of introducing traditional web APIs into blockchain development. According to Vänttinen, whereas previous solutions relied on a third-party middleware or ‘oracle,’ the best way to incorporate an API request and response system into smart contracts is through a node operated by the API provider themselves. Such a system could help blockchain-based applications leverage myriad software-as-a-service solutions, essentially uniting the two worlds.

    The State of APIs in Web3

    The number of APIs is growing exponentially. As companies reinvent themselves with digital transformation, APIs have become the glue for interconnecting the applications we use on a daily basis. When it comes to APIs, there are a growing number of potential use cases for blockchain—the technology is not just relegated to cryptocurrency; rather, it can power a number of scenarios that require smart ledgers.

    Since APIs drive much of the functionality in modern software development, it makes sense that they may be useful in decentralized app development. For example, a blockchain-based decentralized finance (DeFi) app that handles lending and borrowing may want to call an API to collect raw data for a liquidation price, says Vänttinen. Or, a smart contract may wish to acquire the cost of an asset as a conditionality for making a purchase. Hooking into public APIs to check local weather or flight arrival information could affect how smart contracts respond.

    There are many use cases for incorporating APIs into a distributed ledger system. However, “blockchain applications are not able to call APIs natively,” Vänttinen explained. The incompatibility is due to the nature of blockchain technology, said Vänttinen, as it requires consensus from multiple parties to maintain the smart contracts. For this reason, blockchain applications require a middleware piece to connect a blockchain node to an API, relaying data responses from outside of the chain.

    Incorporating APIs Into Blockchain

    As Vänttinen described, previous attempts to connect APIs to Web3 architectures have revolved around a third-party middleware node. Yet, externalizing integration responsibility in this way creates a centralized point of failure, which defeats the purpose of a decentralized environment altogether.

    Instead of relying on such a dependency, Vänttinen advocates for a “first-party oracle approach,” in which API providers operate a layer in front of their APIs. This enables a request-response mechanism for smart contracts, allowing blockchain applications to initiate API requests from on-chain and incorporate the responses into the contracts. This method retains immutability and trustlessness, which is inherent to the distributed ledger.

    One solution enabling such an architecture is Airnode, a Web3 middleware that connects any web API directly to any blockchain application. Companies that have adopted Airnode make up the API3 Alliance, which is “a collective of API providers who have signed up to enable this first-party use of their API for Web3 applications,” said Vänttinen. “Adopting Airnode is a pledge from API companies to make their API blockchain-compatible.”

    The API3 Alliance is one significant initiative aiming to extend the capabilities of smart contracts and blockchain applications. This could enable such environments to call an API to send a text message or email, or grab sports scores, collect maps data and so on. Thus far, participating APIs in the API3 Alliance come from sectors such as agriculture, biotechnology, communications, open banking, insurance, supply chain management and many others. The initiative also accepts applications from API providers that want to join the marketplace.

    Benefits of a Common Web3 Integration Layer

    So, what are the benefits of embracing an industry-standard middleware for web-API-to-blockchain connectivity? Well, according to Vänttinen, it is the best way to enable complex off-chain computation. Enacting intricate computing on the chain is quite resource-intensive, resulting in high transaction fees. By extending particularly complicated calculations externally, blockchain applications could reduce operational overhead. “The industry is long overdue for connection between the real world and blockchain,” said Vänttinen. (By real-world, I assumed he meant other architectural styles that pre-date blockchain.)

    Having a standard implementation could also reduce the development burden for the API provider to support nuanced technologies. Whereas other solutions would require someone in-house to operate blockchain nodes, Airnode is more of a plug-and-play API gateway that could be deployed on AWS as a Lambda function, Vänttinen said. This usability makes it easy for API-based services to appeal to Web3 applications, thus quickly opening a new potential growth segment.

    Of course, Vänttinen admitted, the concept of syncing with off-chain processes is still at a very early stage. There are competing philosophies about how to address it on the market, and some foundational issues have not yet been resolved. For example, most efforts have focused on Read operations to acquire raw data. The next step will be having the middleware ‘oracle’ incorporate off-chain Write capabilities, too. Furthermore, more PubSub-like formats will be important to enable event listeners capable of reactive qualities.

    Final Thoughts

    The Web3 trend is bringing more decentralized, blockchain-based applications to fruition. But, such applications can’t operate in a closed-off world. To experience the full breadth of digital possibilities, they’ll likely need to figure out how to incorporate off-chain computations and mesh with alternative communication software libraries. Naturally, this will require integrations with web APIs, which have become an important method of cloud-based communication.

    The API3 Alliance is “here to propose a solution to the API connectivity problem for Web3,” said Vänttinen. “When a data provider is blockchain-compatible, you can apply the reputation of the data provider in this decentralized data feed … The importance of this becomes more clear when you talk about connecting off-chain services and microservices with smart contracts.”

    Airnode and the surrounding API3 Alliance is one initiative spearheading the connection of APIs and smart contracts. But, this is just one method in a maturing field. What do you think about bridging the two paradigms? Comment on your experience below!

  • Atlassian Survey Shows Lack of Collaboration in Software Development

    Atlassian Survey Shows Lack of Collaboration in Software Development

    A survey of 750 IT and engineering leaders found that while a full 83% of respondents agree that interdepartmental communication is critical for successful software development, only 41% of respondents view cross-collaboration to be a priority.

    The survey gathered responses from within organizations that generated more than $50 million in annual revenue and was conducted by Forrester Consulting on behalf of Atlassian.

    Overall, more than two-thirds of respondents (68%) said more departments than ever are involved with software development. Overall, 80% said they need to improve collaboration with other departments, but 82% don’t feel particularly confident about their ability to accomplish that goal. Three quarters (75%) also noted the need to improve collaboration within their own department.

    Well under half of the respondents (42%) said their culture, processes and tools enable the real-time teamwork and free-flowing information that help leaders make sound decisions.

    Among the subset of respondents that are highly collaborative, however, more than (53%) said they see greater customer satisfaction, with 59% reporting increased revenue because of that increased customer satisfaction.

    Suzie Prince, head of product for DevOps at Atlassian, said the challenge organizations face is finding a way to successfully employ a mix of synchronous versus asynchronous communication mediums to better enable collaboration. In fact, collaboration as part of any DevOps culture needs to be a C-level discussion to consistently accelerate the rate at which software can be developed and deployed, she added. The core issue is that not every management team is equally comfortable with open communications between the various departments that make up an organization, said Prince.

    It’s critical for C-level executives to set that tone and lead by example; DevOps best practices can’t only be adhered to by development teams. The entire business needs to embrace DevOps best practices as an operating model for the business to really attain the maximum benefit.

    Unfortunately, DevOps practices within development teams have been unevenly adopted, so it’s often very difficult to get the rest of the business to embrace DevOps principles. Organizations that do embrace DevOps, however, tend to be much further down the path toward becoming a digital business and can more adroitly take advantage of business opportunities as they arise or shift strategy as changing business conditions may warrant. In fact, organizations that embraced DevOps best practices arguably fared better during the economic decline that followed the onset of the global COVID-19 pandemic that took every business and IT leader by surprise.

    It’s not clear from the Atlassian survey results if a DevOps divide is starting to emerge that makes some organizations more competitive than their rivals. In theory, at least, if every organization is now a software company that happens to build something or deliver a service, then the faster they can reliably build and deliver applications the greater the competitive advantage they can attain and sustain.

    There are, of course, a whole host of factors that determine how competitive any organization might be. However, in the age of digital business transformation, it’s clear that being able to build and deploy software at a fairly regular cadence is the new table stakes required for simple survival.