devopsy.kinsta.cloud

Tag: API

  • Our API Mess is Coming

    Our API Mess is Coming

    APIs are the satisfaction of a long and deep need to be able to create consistent and reliable integrations between disparate systems, operating systems and datasets. As we started to use REST-based APIs, we also realized they fulfilled a previously little-addressed gap in automation. Frankly, they are a force multiplier in DevOps delivery.

    Early efforts at system-independent APIs were not great, but we learned lessons and are steadily getting better. While REST is the default solution for APIs, it probably isn’t the end of the road, as we continue to learn and improve. That’s what we in tech do – get excited about an idea, drive it to broad use and then iteratively improve it.

    And that’s the root of the problem. We have been, and are, saturating our applications with APIs. Internal, external, external-that-rely-on-another-external. The thing that comes to mind every single time I look at an API dependency map for even a medium-sized application is “spaghetti code.” Those lines are going all over, quite frequently returning to themselves.

    And we’re going to have to maintain this creation. Maybe not us, individually, but us as in IT and DevOps teams. The only constant is change, and far too many of us are not only failing to plan for API changes, but failing to even think about them. Because things are generally going swimmingly and most IT managers and DevOps teams have more urgent things to think about.

    But risk management – particularly broad risk – is an essential part of future planning. If you lose access to external API X, what will you do to keep your application running? If you lose an entire internal system that hosts API Y, how can you recover in a timely manner? While most organizations will identify and repair single points of failure, the solution is a pool, and the impact of losing Z servers in the pool needs to be understood, if nothing else – questions like, “At what point would user experience be impacted?” need to be asked, and a plan formulated to keep that from happening.

    We are already seeing issues created by using external APIs. All of them are predictable – an API going offline, or even disappearing completely; an API interface or attendant data format change; a change to the algorithms underlying the API, causing a change in data results, etc.

    API usage is far easier than it has historically been – and companies are working hard to make it easier to access APIs, both yours and others’. This is a good thing, as it means less time spent figuring out and tweaking API calls and request data. But it means our API dependence will only grow, and the number of lines in our dependency graphs – each representing some amount of risk – will only grow.

    “But we’re doing things in the cloud, so we can auto-scale, and failures are no longer and issue for us!” some shout. While that can help some scenarios, it doesn’t help others – if a vendor goes under and takes an API you use with it, or a preferred vendor (or inside component) becomes a security risk, you’ll still need to take steps. Knowing what and where your weaknesses are before they become issues is important, and worth your time.

    Most of you are killing it, cranking out solutions faster, iterating to make them better, using APIs to extend functionality and reduce delivery times. I’m just saying, stay aware of the technical debt you’re creating, because debts always come due, and risks eventually become issues. So know what’s a risk in your infrastructure, and have a plan to react when it does cause problems.

  • Languages and DevOps: API-First

    Languages and DevOps: API-First

    One of the places development has made significant changes is in the land of APIs. These days, a program almost isn’t considered a proper application unless it offers APIs to do most of what can be done via UI (and even command line). This is a big deal for several reasons; first and foremost, the ability to subdivide the work and then test each subdivision. A proper API does one thing and does it well. Testing can validate that this statement is true for any given API. It also allows for distribution of processing. It is far easier to use a load balancer to scale APIs than it is to scale a monolithic application. And finally, it helps with portability, because with API-first, developer users don’t care what platform it’s running on, just that it can be reached via URL and responds as expected.

    Everything is a trade-off, and there are several places that API-first has issues. Integration testing is objectively harder. Loosely coupled applications, in general, are harder to test, and requiring multiple portions of an application be available to run testing becomes onerous, particularly as the layers of APIs start to build up. Second, security is another issue – particularly if you’re offering a public-facing API, but also for internal usage. Once an attacker has access to the network, internal-only APIs are no different to them than external. Smart IT departments design internal APIs with the same controls that external requires. Third is DevOps in general. API-first grew up in the agile/DevOps world, but because there are cascading dependencies, is actually harder to manage DevOps for than most monolithic applications. Finally, documentation – the bane of API-first’s existence. The number of poorly documented APIs vastly outnumbers the astoundingly well-documented ones. Even well-documented APIs, like Google Places, require some thought and testing to use well. Poorly documented ones – my favorite is, “It’s mysite.com/users – everyone knows what’s in there!” – are next to useless without a learning curve.

    With all of that said, what about languages? That greatly depends on what you’re doing. Users don’t care what language it is implemented in – as noted above, they care that the API can be reached via URI, and that it behaves as expected. So you do have options. Indeed, at this point in the evolution of APIs, the language options are so open that you’ll probably be choosing an API definition language instead of a programming language, and just instruct your definition tool to spit out the language of choice.

    Wait, what?

    For those who haven’t followed API development closely over the last few years, it goes something like this. APIs became standardized, then APIs became a big deal, then smart people started to think in terms of code generation á la ancient 4GLs. What if you could use a specification language to describe what you want to do, then pick your language and tell the tool to generate the API? What if you could do this for the entire API hierarchy? That’s where we are. I’m most familiar with OpenAPI and RAML, but there are a large number of specifications out there, and a larger number of tools to develop in them. Before my API friends freak out, of course this is an over-simplification of what happened. I only have 1000 words, max, and my editor would like less. Relax a bit.

    And then you generate.

    I’ll rely on my own experience here, but there are literally dozens of tools available to get you from idea to API interface (doing the work behind the interface is where you come in). With Swagger CodeGen, the tool I’m most familiar with, you specify the API, then generate server interfaces (they call them stubs), and generate client libraries. Finally, you develop what goes on behind the interface on the server side, and (you or others) use the API on the client side. When I say “generate,” I mean “Tell it your favorite language, and click generate.” It spits out the code you can then use to flesh out your API. It does not spit out the info to get the API into your DevOps toolchain, so you’ll have to do the work for that. Which means the more APIs you can generate in a batch, the less DevOps lifting you’ll have to do (more on that if/when we cover microservices). Since most languages can easily be placed into DevOps via their build tools, I don’t want to make this sound too onerous – security and testing will be a bigger problem, by far.

    API security is slowly getting automated and placed right into generated code, but all the usual bits, like certificate management, still have to be handled. The interesting thing, that hasn’t (yet) gotten a ton of attention, is security of clients. It is far harder to verify that the client library is secure when the purpose of that library is to grant access to your API. I expect we’ll eventually have to get there for some applications, but today, it appears to be largely individual orgs figuring out how best to lock down client libraries, and most of the limiting factors end up server-side.

    I’ve generated Python, Java and Ruby APIs via various toolsets, and considering the problem domain is still relatively small, they work fine. And it saves one piece of the puzzle … but does introduce another piece – coordinating client and server updates via DevOps.

    Keep making the apps that run the world, and use API generators to further standardization, spitting out whatever language you need on the backend, and a portable interface for clients.

    You. Rock.

    Whatever your language of choice, in the end, it will likely be your build tool that links it to DevOps. Your biggest concern for DevOps will probably be portability, at least in the near future. Meanwhile, keep rocking it. You’re doing the work that needs doing, even if it isn’t the part of DevOps everyone is currently buzzing about.

  • HAProxy Updates Data Plane API to Simplify Service Discovery

    HAProxy Updates Data Plane API to Simplify Service Discovery

    HAProxy has updated the open source HAProxy Data Plane application programing interface (API) to include service discovery capabilities alongside native support for the Consul service mesh being developed by HashiCorp.

    Version 2.2 of the HAProxy Data Plane API also adds support for file handling, SSL certificates, map files and configuration files for its Stream Processing Offload Engine (SPOE).

    HAProxy is best known for the open source load balancing software of the same name that is widely deployed by DevOps teams. As an extension of that platform, HAProxy Data Plane makes it possible to dynamically add and configure front ends and back ends and then route application traffic between them.

    Daniel Corbett, director of product and marketing for HAProxy, said the HAProxy load balancing software coupled with the Data Plane allows IT organizations to create a gateway through which they can manage their APIs.

    Corbett said the company is currently rounding out a suite of offerings, dubbed HAProxyOne, that are extensions to its load balancing software and that add a control plane and a service mesh to its portfolio. The company has already added a Kubernetes ingress controller to its portfolio.

    At the same time, however, HAProxy Data Plane will make it simpler for DevOps teams to integrate other offerings, such as Consul, within the control plane being developed by HAProxy, Corbett says.

    Corbett said, collectively, HAProxy will provide a suite of tools through which DevOps teams will be able to more easily manage application delivery across IT environments that span multiple platforms.

    HAProxy is not the only load balancing software provider with similar ambitions. The issue that HAProxy will address is the need for a framework that can apply equally well to both monolithic applications and microservices-based applications, Corbett says.

    In 2021, a major battle is brewing for control over the infrastructure that DevOps teams use to deploy applications at scale. Alongside proxy servers and load balancing software, IT organizations are starting to use API gateways, service meshes and ingress controllers as the number of APIs through which data flows rapidly increases. In effect, these technologies provide the connectivity infrastructure required to route data between the applications that make up a distributed computing environment.

    It’s still early days, though, as advocates for competing open source projects make their cases for the hearts and minds of developers. However, given the profound impact these decisions are likely to have on IT operations, it’s likely many organizations will soon standardize on one suite of offerings to minimize complexity.

    Whatever the result, applications environments are, in fact, becoming more complex. IT organizations will soon find themselves running monolithic applications alongside a raft of microservices-based applications, all of which need to use APIs to integrate with each other. Decisions made today about how best to integrate those APIs will have a profound impact for years to come.

  • Internal Makes On-Premises Edition of No-Code Tool Available

    Internal Makes On-Premises Edition of No-Code Tool Available

    Internal today launched an on-premises edition of its eponymous no-code development platform that promises to simplify application development backlog reduction for enterprise IT organizations.

    Arisa Amano, CEO of Internal, said many professional developers regularly find themselves building simple applications that, if given access to the right tools, could be built instead by end users. Internal was created to provide that capability either through a cloud service or, now, through an on-premises edition.

    The on-premises edition makes it simpler for developers to work with databases, application programming interfaces (APIs) and applications that reside behind a corporate firewall, Amano said.

    As organizations accelerate digital business transformation initiatives, professional developers are pressed to build and deploy more complex applications faster. Many lower-priority application development projects are sidelined simply because developers don’t have time to take them on. Internal, and other no-code and low-code application development tool providers, are making a strong case for enabling end users to build their own applications using graphical tools, or encouraging professional developers to use these tools to build less complex applications more quickly.

    A survey published by Internal finds 62% of respondents report their developers spend more than 25% of their time fulfilling requests to build internal tools. More than three-quarters of respondents (79%) said engineers use any extra time they can find to develop customer-facing features or other applications.

    Most of the projects filling application backlogs tend to involve workflows that span multiple applications running in a data center. The challenge organizations face is low-code and no-code tools that reside in the cloud can’t easily access that data. As such, Amano said there’s a clear need for a no-code tool that can be deployed in an on-premises IT environment.

    Of course, regardless of who builds the application, it still must be deployed. Challenges arise as developers build more applications, which must be concurrently deployed in production environments. The addition of no-code and low-code tools, and the applications created with those tools, will require organizations to rethink and refine their DevOps processes.

    Historically, professional developers have tended to view no-code and low-code tools as the application development equivalent of a black box preventing them from building reliable, robust applications that can scale. Many low-code and no-code applications that organizations build, however, are meant to used internally by a finite number of end users. At the same time, no-code and low-code tools become more robust with each major update.

    Whatever bias may exist, the current application development backlog in most organizations is only going to increase. Professional developers may prefer to write code at a lower level than a no-code or low-code tool allows; however, most business executives are more interested in application performance than in how it was constructed.

  • Low-Code Opens API Integration Potential for Citizen Developers

    Low-Code Opens API Integration Potential for Citizen Developers

    APIs have an accessibility problem. Could a low-code API approach make them more inclusive and accessible for citizen developers?

    Most software today integrates with third-party SaaS. TomTom for geolocation. Stripe for payment. Twilio for SMS. Kairos for facial recognition. AWS for NLP … the list goes on. Whatever function you’re looking for, there’s probably an API for that.

    These specialized utilities help software teams avoid reinventing the wheel for things outside of their core competency. They’re standardized, too — most services are programmable following REST or GraphQL guidelines. Serving integrations over HTTP aids in decoupling the business logic from the client interface. Furthermore, OpenAPI Specification, now part of the Linux Foundation, has risen as a standard description format, making APIs more defined and interoperable.

    However, for all this progress, there’s still friction in the integration process, even for seasoned developers. Each API is a bit unique. It takes time to read through the documentation to authenticate and discover methods, parameters and accepted fields. It requires effort to test requests within Postman, decipher error messages and set up calls within your unique environment. On top of all this, downtime and breaking changes are legitimate concerns for over-the-wire dependencies.

    APIs require effort and maintenance, and just integrating third-party APIs alone won’t build you a fully functional application. Yet, organizations find themselves scrambling to find and hire engineers, and increasingly the onus is on non-tech folks to create new digital workflows. Naturally, companies are currently searching for even more low-code solutions.

    So, what’s the glue holding together API products? How can we better connect APIs to the software applications we’re already building and allow citizen developers to reap the benefits?

    My theory: If low-code platforms decrease the API integration hurdle, more problem-solvers can stitch together business utilities to create more advanced internal applications. Making third-party APIs more inclusive for citizen developers could boost the API economy at large and enable creative solutions to flourish throughout enterprises.

    The Need for Low-Code API Integration

    Perhaps the best way to demonstrate the need for low-code is to scare you with some code. Let’s say you want your application to initiate a direct deposit into a bank account.

    You don’t want to build a financial service from scratch, so you turn to Stripe API. You set up an account, grab a key, and dig in to the ‘Getting Started’ materials. You peruse their comprehensive documentation, and find out how to create a payout.

    The docs demonstrate a sample HTTP curl request for creating a payout. Other parameters to pass include destination, methodor source_type.

    curl https://api.stripe.com/v1/payouts \
 -u sk_test_4eC39HqLyjWDarjtT1zdp7dc: \
 -d amount=1100 \
 -d currency=usd

    If there are no errors with the request, the response will be a payout object, formatted in structured JSON. It details the balance, data created, currency type and other fields. It looks something like this:

    {
 "id": "po_1HmNKQ2eZvKYlo2CuRNBRWUW",
 "object": "payout",
 "amount": 1100,
 "arrival_date": 1605116514,
 "automatic": true,
 "balance_transaction": "txn_1032HU2eZvKYlo2CEPtcnUvl",
 "created": 1605116514,
 "currency": "usd",
 "description": "STRIPE PAYOUT",
 "destination": "ba_1HmNKQ2eZvKYlo2CzlT6m5gk",
 "failure_balance_transaction": null,
 "failure_code": null,
 "failure_message": null,
 "livemode": false,
 "metadata": {},
 "method": "standard",
 "original_payout": null,
 "reversed_by": null,
 "source_type": "card",
 "statement_descriptor": null,
 "status": "in_transit",
 "type": "bank_account"
}

    Now, you can parse the response into data your application can ingest. You can also send other requests to update, return and cancel this payout object.

    For specialists familiar with HTTP calls and JSON, this is pretty straightforward. Stripe docs typically generate sample requests for many languages, like PHP, Ruby, Python and others to help within your own environment.

    But what if your users are more comfortable with UIs? Integrations are pretty opaque to the non-technical. Even for seasoned developers, APIs can quickly become too dense, and not all have a great developer experience.

    Hurdles In API Integration

    There are many issues preventing non-tech-savvy users and novice programmers from inserting API-driven business logic into their applications:

    • Onboarding time: Setting up an account, acquiring authentication, reading docs and testing calls to familiarize yourself all takes time.
    • Data Types: Data passed in headers must conform to explicit data types, or else requests may fail.
    • Parsing: Responses are often too verbose; developers must parse the JSON object responses to retrieve the data they need.
    • Programming language: Each language handles network communication differently, meaning requests look different in every language.
    • Breaking Change: Third-party services change often, so if you’re not actively maintaining an integration, you may face breaking change.
    • Differing formats: There are various API standards. REST, SOAP, GraphQL all are designed a bit differently. Plus, webhooks, event-driven and publish-subscribe patterns have nuanced communication models.
    • Security and legal: If APIs aren’t appropriately secured, hackers could easily steal user data or cause outages.

    As you can see, a simple programming task like “make a bank deposit” becomes extremely complex the more closely we look at the logistics of actual implementation.

    Furthermore, just because you’ve made an integration work, that doesn’t mean you have a fully-functional application, securely deployed in the cloud, with a friendly UI  that’s universally accessible. A single application may have numerous integrations, meaning the integration process must be duplicated for every external service you connect.

    API Categories

    Overcoming these challenges is a lot to ask of non-professional developers, who may not be comfortable consuming business applications from the command line. This issue is further compounded as integration extends beyond public APIs, too.

    In a recent interview with Divyesh Kharade, co-founder/CEO at DronaHQ, he identified three main API categories:

    • Ready-made SaaS APIs: Microsoft Teams, Slack, Sendgrid, Twilio, Google Sheets, etc. These are the connections to third-party operational business utilities and applications.
    • Connecting to databases: Many companies require secure connections with custom internal databases or legacy internal MySQL databases.
    • Internal custom software: These are private or partner APIs. They may be new API-first microservices or custom-built legacy internal apps. Such middleware may expose many APIs.

    If a low-code application wants to become a holistic software development add-on, it must anticipate all potential environments. This includes integration realities for various scenarios.

    Citizen Developer vs. “Non-Tech”

    So who, exactly, benefits from more accessible integrations? The term citizen developer is often used to refer to any non-professional software developer with minimal programming training. “When most people speak of citizen developers, they mean functional guys that can build tech,” says Kharade.

    As I’ve said previously, fostering a culture of citizen developers is becoming a bigger priority amid dwindling developer talent and a combination of other forces impacting software development requirements.

    Yet, citizen developer tends to means something a bit different to everyone involved in the low-code space. Kharade seems to prefer the term “non-tech.” Non-tech users may include designers who may not have a software background but the relevant UX/UI knowledge to assemble workflows. “Design is a big part of any application,” said Kharade.

    Masking the Complexity of APIs

    If low-code tools mask complexity and make APIs more accessible for these non-engineers, it could inspire many creative solutions. With low-code, “creativity is limitless, and integrations really help with that,” said Kharade. “Once they see that power, they aren’t going back again.”

    For example, if an internal application requires chat functionality, Slack could be a great addition. Low-code layers could hide complexity in the backend, to the point that it “doesn’t feel like a request in background,” noted Kharade.

    The solution works both ways. Utilizing low-code platforms to manage data and expose APIs could enhance visibility and extensibility. Kharade described an industrial manufacturing use case, in which shift workers manually entered machine operations data on paper. That was before COVID; once lockdowns were mandated, the company started digitizing this record-keeping process.

    Access to real-time data, and the ability to filter and organize historical data, is already beneficial.  “Six months down the line, they may consider how to make this data accessible using APIs,” said Kharade. “The journey has just started — it will only get smarter and better.”

    Trigger-Based Workarounds Were A Start

    Of course, tools like IFTTT and Zapier have already been making APIs accessible to laymen for years. These platforms use friendly, UI-based front ends over the foundation of complex integrations, allowing users to set up trigger-based automation. If this particular service does this, this other service should do that.

    These are cool tools for stitching together service-service communications. However, these workflows are a bit isolated from the larger application development picture. For example, an application for hiring employees may require more than connecting a form API’s input to a Google Sheets field. This could require a custom database, a way to upload documents and the ability to deploy an application to the cloud.

    Since low-code already has standard components for drag-and-drop, adding a common method to insert third-party API integrations feels like a natural feature enhancement for these platforms.

    Making Integration More Accessible

    ProgrammableWeb records 23,000 APIs as of this moment. Clearly, many innovative, programmable SaaS offerings are ripe for the picking. Some paid, some open, all with slightly different integrations.

    “It’s difficult for a non-tech user to register and interact with an API,” said Kharade. Figuring out authentication, OAuth, headers, payloads and responses requires expertise. Though a professional developer may be required to “register them once,” says Kharade, once third party integrations are set up, they “should be drag-and-drop and click-to-use.”

    Solving integration problems is just one of the many arenas where low-code platforms could shine. If platforms can successfully establish (and maintain) connectivity, consumers could more easily access the core logic they are searching for. Democratizing robust solutions for traditionally non-tech companies is especially important amid the COVID-19 crisis.

    To be clear, I don’t believe it’s the API provider’s job to cater to end-users. Instead, they should focus on creating excellent developer experiences. Naturally, the type of business application you are trying to construct and your access to software engineering talent will, to an extent, determine a project’s technical stack. The JAMstack style, for example, may be a better fit for your team’s makeup and project objectives.

    That said, there seems to be room in the market for additional low-code layers to open up integration. In the same way APIs avoid reinventing the wheel for specialized utilities, low-code layers could standardize the pains of integration itself.

  • SAP Makes Available Free Cloud Tier to Build Applications

    SAP Makes Available Free Cloud Tier to Build Applications

    SAP announced that next year it will make available a free tier on SAP Cloud Platform for developers that are building applications.

    Announced at the SAP Tech-Ed conference, the free tier comes at a time when SAP is encouraging developers to use its applications as a foundation for digital business process transformation initiatives.

    As part of that effort, SAP is also extending the capabilities of the low-code tools it makes available to customize SAP Cloud Platform Workflow Management and SAP Intelligent Robotic Process Automation (RPA) 2.0 to make it easier to integrate these offerings with other applications. SAP beginning in January is also promised to include a limited edition of SAP Intelligent RPA in every SAP S/4HANA Cloud subscription.

    Finally, SAP is beta testing SAP Ruum, a set of no-code tools it plans to make available alongside the low-code tools it already includes in its portfolio of tools for building custom applications that extend the packaged applications it provides on-premises and in the cloud.

    At the heart of the SAP digital business transformation strategy is SAP Business Technology Platform, which makes it possible to share data across SAP applications via a common data and object model. SAP makes available SAP Cloud Platform to provide a distinct platform-as-a-service (PaaS) environment through which developers can build extensions to its applications.

    Most recently, SAP has made SAP Graph tools in beta offering as a complement to a revamped SAP API Business Hub that will be available in beta this month.

    Thomas Grassl, vice president and global head of developer relations and community for SAP, said the company is also embracing continuous integration/continuous delivery (CI/CD) platforms as it continues to encourage developers to move away from waterfall-based approaches to building and deploying applications.

    The free tier of resources SAP is providing developers will encourage organizations to experiment more at a time when many organizations are now quite certain what digital process may ultimately gain traction with their end customers and suppliers. SAP is also focusing on no-code and low-code tools to enable organizations to accelerate the development of those applications. SAP already has an existing partnership in place with Mendix, a unit of Siemens, to make available low-code tools.

    SAP’s biggest concern, of course, is that digital business transformation initiatives will revolve more around application platforms from Oracle, Salesforce, ServiceNow and Workday. Every major provider of enterprise applications offers low-code and no-code tools to customize their applications. Many enterprise IT organizations are employing these tools more widely to accelerate digital business initiatives as an alternative to building entire applications from the ground up.

    The challenge these organizations will soon encounter is that as the backlog of applications that need to be deployed starts to build, they will have to modernize their approaches to deploying applications to keep pace.

    As a dominant provider of enterprise applications, SAP is in a position to drive digital business transformation in a way that extends the reach and scope of its enterprise resource planning (ERP) portfolio. The issue is finding a way to accomplish that goal when organizations thanks to the COVID-19 pandemic have a pressing need to transform much sooner than later.

  • Apollo GraphQL Offers Free API Management Service

    Apollo GraphQL Offers Free API Management Service

    Apollo GraphQL has announced the general availability of a free Development Graphs cloud service to simplify management of GraphQL application programming interfaces (APIs) at scale.

    GraphQL APIs have been gaining traction as an alternative to REST APIs because they provide developers with more control over who can access data via an API and are easier to build and maintain.

    Apollo GraphQL CTO Matt DeBergalis said that approach makes it possible to build APIs using the Agile development methodology versus the waterfall processes that typically are employed today to construct REST APIs.

    Originally developed by Facebook, GraphQL APIs are being extended by Apollo GraphQL via a set of Studio tools based on a graph that makes it easier to discover those APIs and search GraphQL schemas. IT teams can also use the tools to launch queries against those APIs using an integrated development environment (IDE) dubbed Explorer. Developers are not required to have a deep knowledge of GraphQL syntax or the shape of a schema to launch a query. They can also copy links to share their queries with other users to copy a version of a query that can be run in the terminal as a CURL command.

    DevOps teams can also search for types and fields in a schema using tools that guide them through the process of querying any field in their schema. They also can find and add fields to a query without needing to know in advance where those schemas are actually running.

    The tools also make it possible to view samples of results to better gauge what data will be returned by a query, along with latency rates for each field in a query to better understand the overall cost of a query.

    DevOps teams can also view response data from their queries as JSON or as tables and create mock responses to queries to test operations, Finally, each request and view of every query can be viewed alongside the data returned.

    Connections to a private sandbox in the cloud for each local GraphQL server or any development environment are enabled using the Development Graphs tool.

    DeBergalis said he doesn’t expect most DevOps teams will rip and replace the REST APIs that are already strewn across most extended enterprise. Instead, most new applications will make use of GraphQL APIs that over time will become the de facto standard for building APIs. However, developers are clearly voting with their feet to abandon REST APIs in favor of a more agile approach, he said.

    Apollo GraphQL is looking to become the software development lifecycle (SDLC) platform for managing GraphQL APIs at scale, DeBergalis added.

    The challenge organizations face now is finding a way to integrate the management of those GraphQL APIs within the context of a larger set of DevOps workflows. Of course, that may wind up being more challenging than initially anticipated when they consider REST and GraphQL APIs could be employed alongside one another for years to come.

  • OpenTelemetry and the Future of Monitoring Instrumentation

    OpenTelemetry and the Future of Monitoring Instrumentation

    OpenTelemetry is an open source project to provide a de facto standard trace API and a metric and log agent to end vendor lock-in

    It’s no surprise that understanding the internal health of applications and systems is a high priority for developers and SREs. It’s a well-understood problem with dozens of vendors focused on helping companies diagnose application performance in real-time.

    While the actual platform doing the metric, log or trace analysis tends to see most of the focus, a critical component of the data pipeline is, and has always been, the actual instrumentation. That includes APIs and SDKs for application traces as well as installable agents to gather metrics and logs. In the past, this has been a highly fragmented space with both open source and proprietary agents and SDKs. Most APM vendors have provided their own proprietary instrumentation libraries that, intentionally or not, create a vendor lock-in at a code-level that isn’t ideal for most organizations.

    To address this challenge, an open source project was created with a highly aspirational goal of providing a de facto standard trace API along with a metric and log agent that would eliminate this fragmentation, create a unified standard and effectively end vendor lock-in.

    The project, OpenTelemetry, was officially started in May 2019. It is a Cloud Native Computing Foundation sandbox project and came out of the merger of the OpenCensus and OpenTracing projects. They’ve since expanded their goal to include not only application tracing and code instrumentation, but also metric and log collection. They are essentially covering all the critical telemetry points in a system or application.

    Imagine if, regardless of the platform you ultimately sent the data to, you could instrument your code with the same libraries, the same agents and a single set of binaries and libraries. It would dramatically simplify the deployment process, internal training and make it trivial to change or add analytics platforms as needed. This is the promise of the OpenTelemetry project.

    A project like this doesn’t get far without significant support. This is where OpenTelemetry shows incredible promise to realize widespread adoption quickly. Nearly all the largest monitoring and cloud vendors including AWS, Google, Microsoft, Splunk, New Relic and many more have thrown their weight behind OpenTelemetry. They’re actively contributing and supporting the project. One by one, these organizations have come to the agreement that the telemetry component is not the differentiating feature. Instead, they are working together to provide a best-of-breed experience at the telemetry layer.

    The project is in beta, and the potential is huge. OpenTelemetry modernizes every aspect of the telemetry system. It eliminates vendor lock-in. It will outperform the vast majority of existing agents. It simplifies installation and training by reducing dozens of agents and libraries to just one.

    So where to get started?

    It’s still early, but depending on your platform of choice, it may already support OpenTelemetry. If so, look into the existing libraries and test them out in your internal sandbox environments. You’re likely to find improvements over your existing telemetry tools, although there will certainly be gaps in things such as development language support and metric/log integrations. It should give you a sense of the scope, potential and maturity of the project and help to determine whether this is a viable solution for your next deployments.

    The big question mark is whether this will live up to its potential. With the high-quality team assembled, the support of the community and industry and with a truly impactful mission, it will. Ideally, within a few years, this will be how we instrument and gather data from our applications regardless of where we send that data.

  • Kong Previews Application Connectivity Service

    Kong Previews Application Connectivity Service

    Kong Inc., at its online Kong Summit 2020 conference, unfurled Kong Konnect, a cloud service through which developers can leverage application programming interfaces (APIs) and microservices to integrate distributed applications.

    Available as a private beta, Kong Konnect promises to make it easier for developers to create complex workflows spanning API gateways, Kubernetes Ingress controllers and service mesh runtimes.

    Company CTO Marco Palladino said Kong Connect makes it possible for developers to consume integration runtimes as a service via an interface that allows them to invoke only the specific services they require.

    Scheduled to be generally available early next year, Kong Konnect is based on a ServiceHub platform that creates a single source of truth for all services via a catalog that any organization has developed. Kong Konnect also provides access to a runtime manager that enables self-service provisioning based on roles in addition to enabling IT teams to manage Kong runtimes across hybrid cloud computing environments.

    Kong Konnect also integrates the company’s API Gateway, Ingress Controller and Kong Mesh, a service mesh based on open source Envoy proxy software, and Kong Vitals, a tool for monitoring Kong services and transactions in real-time. Kong Konnect is also integrated with Insomnia to design APIs using a tool that integrates with any Git-based repository and the Kong Developer Portal, through which developers can publish their own services.

    Palladino said the goal is to provide a turnkey service that automatically provides all the Layer 4 through 7 networking services that developers require to integrate both monolithic applications running on virtual machines as well as emerging microservices-based applications running on Kubernetes clusters. Kong Konnect makes available universal service connectivity on-demand via a single API to any class of developer, he said.

    As application environments become more distributed, developers are spending more time than ever on basic connectivity issues. The company is making a case for a cloud service that unifies all those services under a common control plane it manages on behalf of developers. That service will then free up developers to spend more of their time writing code versus integrating application services, said Palladino.

    That capability will be especially critical as IT organizations begin to deploy more applications on edge computing platforms that need to be integrated with a wide array of application services running on public clouds and on-premises IT environments, added Palladino.

    It’s too early to say to what degree IT organizations may be ready to rely on a service versus continuing to manage connectivity themselves. In effect, Kong is making a case for abstracting all the underlying complexity of connectivity and networking using an API to invoke everything from a simple API gateway to a service mesh designed to integrate hundreds of microservices.

    Regardless of the path chosen, the one thing that is clear is a new application connectivity as a service era has finally arrived.

  • Strengthen API Security With These Tips and Patterns

    Strengthen API Security With These Tips and Patterns

    If you haven’t noticed, digital organizations are building more and more APIs. ProgrammableWeb tracks more than 23,000 public web APIs at the time of writing, and the API market is estimated to be worth $5.1 billion by 2023. Building with APIs increases internal interoperability, reduces development time and can extend product functionality tremendously. In short, the value of APIs is rising. However, opening up with APIs brings security caveats that, if not addressed, could result in serious breaches that negate these benefits.

    I recently spoke with Roey Eliyahu of Salt Security to get his take on the top security concerns for API-enabled companies, and how organizations can safeguard their services. In short, he recommends thoroughly cataloging and understanding your API inventory, designing for legal challenges and mitigating OWASP’s top 10 API security vulnerabilities.

    Challenges for API Security

    APIs have been called “the next frontier in cybercrime.” Rightly so, as API breaches continue to pop up nearly every day. Take the recent API vulnerabilities at Cisco Systems, Shopify, Facebook, U.S. presidential campaign apps, and GCP as evidence. The most infamous was likely the Equifax breach—not enforcing formats on incoming API calls resulted in a massive data breach, which cost the company a $700 million lawsuit.

    So, why have API breaches increased recently? Well, a few years ago, there were simply fewer APIs. And, these older services exposed less sensitive data. Nowadays, companies typically have many more APIs. Banking applications, for example, might expose hundreds of highly valuable endpoints that are very appealing to hackers.

    Companies also build APIs for different audiences: A 2020 study by SmartBear found 72% of companies develop both internal and external-facing APIs. These APIs could easily hold hundreds of parameters, and with every new method added, a company’s potential attack surface grows considerably.

    3 Steps To Secure APIs

    To mitigate the above API security issues, Eliyahu suggested three main steps: Understand your API inventory, audit against privacy policies and data privacy regulations, and track the most pressing API vulnerabilities like a hawk.

    Know Your API Inventory

    “You need a great inventory of APIs to know your attack surface,” said Eliyahu. Many API providers document their APIs with OpenAPI definitions. But, these files don’t always match the production realities—they may be manually generated or leave out certain methods. “In every organization, there is up to a 40% gap between APIs that are actually deployed versus what has been inventoried,” he said. By maintaining a clearer API inventory, organizations can avoid unprotected shadow IT.

    Include Legal Auditing

    A change to an API could result in a change in how it collects user information. Updates involving sensitive data could break privacy policies or regulations such as GDPR. “After you have an inventory, you need to map all parameters manually,” said Eliyahu. He recommended performing a legal audit against your API inventory and establishing a repeatable testing process that plugs into a continuous delivery pipeline.

    As API teams release new updates, they must consider security and legal implications earlier on in the development flow. “Now, with CI/CD, these changes have to happen very frequently,” he noted.

    Watch the OWASP API Security Top 10

    There are many known API security gaps. For example, there may not be proper authorization in place, keys could be exposed, unexpired tokens could be hijacked and so on. Thankfully, OWASP has compiled a resource of top API security vulnerabilities, which Eliyahu and many others in the API economy have rallied behind. “The world should move to adopt the OWASP issues,” he said. These vulnerabilities are as follows:

    1. Broken Object Level Authorization
    2. Broken User Authentication
    3. Excessive Data Exposure
    4. Lack of Resources & Rate Limiting
    5. Broken Function Level Authorization
    6. Mass Assignment
    7. Security Misconfiguration
    8. Injection
    9. Improper Assets Management
    10. Insufficient Logging & Monitoring

    Using AI To Go Beyond ‘Whac-A-Mole’

    Plugging these above holes is a good start. However, these are only defensive measures against known patterns; they aren’t helping you keep guard for nuanced attack types. Ideally, security systems should stop cyberattacks before they even get started.

    Yet, Eliyahu recognizes a big gap in how APIs are monitored. Organizations “don’t have proper tools to know who is looking for vulnerabilities,” he said. “Tools are typically not that advanced. They mainly look for injections and known patterns.”

    Imagine a hacker probing an API. They will likely do so by trial and error, testing undocumented endpoints, sending malformed requests and so on. “They need to probe for hours or days before they find something,” said Eliyahu. If an AI could detect these odd attempts in minutes or seconds, IT could significantly reduce risk.

    Security solutions should thus leverage big data and AI to create baselines of typical behavior, then deter malicious activity the millisecond any nefarious probing begins. Eliyahu said such a security AI must consider dozens of behaviors such as, How is the API being accessed? What parameters are being used? What are the relationships between parameters? What is the flow of API calls? What type of data can be exposed?

    Instead of waiting for a breach to occur, an AI-enabled API security monitoring could instantly pause behavior deemed “out of the norm” by referencing a knowledge base that’s constantly refined using machine learning. Because, if you are only blocking malicious actors without knowing how you need to improve API security posture, “you are just playing Whac-A-Mole,” said Eliyahu.

    DevSecOps Requires API Security Forethought

    Mark O’Neil, analysis at Gartner, predicted that by 2021, 25% of organizations with public APIs will have discontinued or rebooted their public API strategy, citing security concerns as a common breaking point. APIs are now at the core of many application developments. Yet, without the proper DevSec proposition, teams risk wasted efforts and costly breaches.

    To summarize, here are our tips on API security:

    • Ensure everything is well-documented: API inventories cannot be audited if undocumented. Shadow APIs increase the likelihood of an attack.
    • Watch the top 10 OWASP vulnerabilities: These points are foundational to API security. Though not a cure-all, they present a starting point.
    • Build a security and legal review process: Ensure APIs are never deployed without the proper security review. Create validation systems that plug into continuous development processes, so new versions are never breaking privacy policies.
    • Each API is unique: Another thing to keep in mind is each API is unique. Therefore, API security monitoring also requires more context into unique business logic.
    • Continuously monitor your APIs: Continuous search for anyone looking for vulnerabilities.
    • Remove deprecated APIs: Remove old API versions. “You need a mechanism to identify and remove old API versions.” Otherwise, “zombie APIs will bite you!”
    • Learn from the attacks you block: Don’t just play Whac-A-Mole. “You need a mechanism to learn from attacks you block.”