devopsy.kinsta.cloud

Tag: APIs

How Supergraphs Help Consolidate Data Access
GraphQL, a query language for APIs, has become more of a common tool for new web-based development. GraphQL can help streamline data access by providing all the resources a developer requires in a single request. Compared to the traditional REST approach to web-based data access, GraphQL has a more standardized and traversable schema and arguably better developer experience.

But GraphQL is not just great for improving developer usability for a single data model. If used to aggregate various data sources and APIs into a unified graph, GraphQL could become ‘one schema to rule them all‘ for enterprises. Some refer to this architecture as a supergraph.

I recently caught up with Tanmai Gopal, CEO of Hasura, to explore the concept of a supergraph and see how it can help make data more accessible for enterprises. Below, we’ll consider the expansion of the supergraph concept and what the benefits are of deploying such a strategy.

Understanding the Need for Supergraphs

In large software architectures, product teams need access to various data sources from different domains. “There is a need for integrative experiences, which are becoming more and more important for the end user,” explained Gopal. For instance, an e-commerce application might be composed of product catalog data, login functionality, payment data, shipping information and other necessary information.

However, there are hurdles in the way of pulling in data from these multiple domains. Different domains are often built as microservices, supported by a dedicated team and using a bespoke developer portal and API schema to externalize the services, said Gopal. The friction involved in API integration and the lack of a common registry of services could slow down development.

Not only do microservices that expose discrete endpoints have issues, but the lines between internal and external are blurring, necessitating a zero-trust approach. And if we begin to consider data as a product, explained Gopal, we need to access multiple domains in a secure, consistent and scalable way.

Benefits of Adopting GraphQL and Supergraph

A supergraph can solve some of these difficulties, explained Gopal. By exposing graphs from different domain services, you can automatically unify data into a supergraph, which could be accessed from various teams. And the great thing about using GraphQL is that the act of documenting is the act of building the API, meaning that the description and schema are more standardized.

By composing disparate services into a centralized registry, you could work across domains in a systematic but federated way, said Gopal. Implementing a layer here could also be a good area to enforce role-based identity or authentication.

Andrew Carlson, principal field architect at Apollo GraphQL, similarly proposed using GraphQL to centralize data access control. “When used as a layer to aggregate and orchestrate existing APIs,” Carlson said. “It’s an ideal location in our architecture to centralize access control and authorization down to the field level, providing field-level observability into which clients request what data.”

Gopal shared a handful of other specific advantages for using GraphQL in this context:
- Helps you write less code: Since GraphQL enables you to fetch whatever field you want, aligning on GraphQL as a common interface could reduce the number of requests required to integrate data. Also, certain developer experience perks, such as auto-complete, could enhance schema discoverability and have exponential benefits when sources are intertwined in a combined graph.
- Increases agility for new development: The need to quickly understand the overall schema and what it looks like is becoming especially relevant, especially for AI applications, said Gopal. There’s a massive agility boost when using a supergraph schema, he added, such as enabling software teams to fetch the context they require to quickly put things into motion.
- Improves data and service discoverability: Software teams are facing tooling sprawl, and the number of APIs in use within an organization keeps increasing. A supergraph can enable a traversable data mesh, greatly aiding discoverability and, in turn, the ability to innovate. “The benefit from the ability to understand and use data from different domains is massive,” said Gopal.
- Unifies teams from various domains: Another potential benefit of the supergraph concept is improved inter-departmental collaboration. Helping teams work together could enhance reporting and reduce risk. This new shared operating model helps to clean things up when updating code and the schema, said Gopal.
Next Step: Protocol Agnosticism

The idea of wrapping disparate data sources and formats in a standard, unified layer is an alluring concept. “The agility impact is massive,” said Gopal. Especially, he adds, if you have the ability to automate the creation of a supergraph around various styles of legacy databases.

However, investment into a supergraph can be time-intensive. The benefits are unlocked down the line, but you will need an internal champion to initially advocate for it and bring it to fruition, he cautions. Furthermore, enterprises may face hesitancy in going all-in on GraphQL due to past investments in other technology types.

While GraphQL is becoming more popular, various API styles are still commonly applied and will continue to be supported for some time. For instance, a 2023 study by API management and testing company Postman found that GraphQL use had eclipsed the mainstay API format of SOAP. Yet, REST is still the dominant API style, used by 86% of respondents, followed by Webhooks (36%), GraphQL (29%) and SOAP (26%).

As such, the next step is for the supergraph to be agnostic to GraphQL, said Gopal. Instead, it should be able to work with various protocols since most enterprises are utilizing multiple styles.
January 11, 2024
GraphQL API Visibility Ensures Enterprises Can Escape Lock-In

While GraphQL is still early in its enterprise adoption curve, the runtime query language (and replacement for REST APIs) has already built an impressively robust ecosystem for jumpstarting major enterprise-grade deployments. Among the appeals of GraphQL are its open source implementations and breadth of available server types, which combine to offer enterprises flexibility and the benefits of best-fit solutions.

However, operating GraphQL in production, maintaining GraphQL APIs at enterprise scale and achieving the requisite GraphQL visibility and security require complementary tools and services. This is where enterprises can fall prey to vendor lock-in and lose the flexibility to change and control their own deployments if they aren’t careful.

Enterprises need vendor partners to manage their GraphQL deployments effectively, and vendors are glad to offer comprehensive SaaS solutions. But enterprises must understand that a vendor can seek to capture their long-term business by coercion—straitjacketing customers by hosting their entire GraphQL infrastructure and GraphQL gateway and making it, well, not exactly easy to let go. They can unleash particularly nefarious practices, using proprietary features to force customers to migrate to certain technologies or purchase expensive upgrades to keep their crucial GraphQL deployments afloat. Enterprises must be wary of allowing this vendor lock-in to happen for their own good.

Knowledge is essential to maintaining the balance of power in vendor relationships. Enterprises must understand their GraphQL graph and key measures to correctly assess current vendors and maintain agility and control over deployments by not entrusting their entire fate to any one partner.

These are the specific areas where enterprises should seek out intelligence and reliable metrics in order to choose their own future with GraphQL:

Understand Your Overall Graph (and Subgraph) Utilization

Establishing an accurate overview of your graph utilization—and harnessing actionable insights—are crucial, foundational steps that lay the groundwork for deeper analysis and better decision-making. You need to know what graph and subgraph objects and operations your most important API users heavily engage with and which objects and operations are underused or not used at all. You need visibility into who is using GraphQL APIs and how. Getting to actionable intelligence should also tell you where errors are most likely to occur and what’s causing them, as well as where major performance bottlenecks pop up during heavy load events.

With this overview available, the next steps call for building a depth of knowledge in each of these five areas:

1) Understand areas of heavy graph and subgraph usage

Knowing exactly where heavy usage occurs in your graph puts you in a position to upgrade or migrate your GraphQL API server or gateway on your terms while mitigating outage and error risks. Specifically, you want to know which operations in your graph are executed most and how often, which graph objects and fields are consumed most often, and which subgraphs support those most heavily burdened areas in your supergraph. This knowledge and some careful planning help to put successful GraphQL server or gateway changes within your power and also reduce risk when introducing changes to areas of the graph under heavy consumption.

2) Understand where your graph is underused

GraphQL migration and modernization efforts are simpler—and carry less risk—when you first remove any unused functionality and reduce your scope as much as possible. To do so, you need visibility into your graph’s least executed operations, objects and fields, and subgraphs, as well as any operations with zero recent usage. Then, flag low-use graph areas you can deprecate, replace or just remove to simplify your deployment before making changes.

3) Understand your GraphQL API users and usage

You want a seamless experience for your GraphQL API users throughout any changes you make, especially for the users and graph-usage patterns most crucial to your business. That means knowing the frequent users of the areas of your graph under the most heavy usage. You also want to recognize and bolster any GraphQL operations that your most important users frequently call, in addition to the objects and fields those users consume.

4) Understand graph errors and potential error hotspots

The right actionable intelligence will prepare enterprise teams to anticipate and address graph errors. This requires visibility into where, when, why, and how often graph errors occur and error hotspots (for example, high P95 latencies indicative of potential timeouts). Insights into which users feel the brunt of graph errors are also valuable in shaping mitigation strategies.

5) Understand graph performance bottlenecks

Bottlenecks caused by your GraphQL server or gateway itself may call for a shift in your graph’s underpinnings. Bottlenecks can also arise from costly queries of substantial depth and height, especially when these queries see intensive use.

To foresee and address bottlenecks, actionable insights include the average and P95 latencies of your graph’s most-called operations and the heights and depths of your graph queries with the worst performance metrics. You also want to know the number of times that bottlenecked operations get called each day, the subgraphs at the root of bottlenecks, the latency added by supergraph query planning, and which users are impacted the most by bottlenecks.

Graph Knowledge = The Power to Avoid Vendor Lock-In

Implementing the GraphQL API visibility and actionable intelligence strategies covered here help ensure that an enterprise has the means to modernize or migrate as necessary to preserve its strategic independence. Those unfortunate organizations already in the grip of vendor lock-in may need to bide their time but should still use these same techniques to improve and decouple their fundamental GraphQL services from their vendor-controlled GraphQL technology. There’s no acceptable reason why organizations relying on free and open source GraphQL servers and gateways should find themselves stuck in proprietary software arrangements. Thankfully, the right knowledge can deliver enterprises a more flexible future as they modernize API development with GraphQL.

October 4, 2023
ngrok Adds Load Balancing to Programmable Network Service

ngrok today added a global server load balancing capability to its service for managing ingress via an application programming interface (API).

Alan Shreve, ngrok CEO, said the ngrok Global Server Load Balancing (GSLB) service reduces latency and increases overall resiliency by automatically rerouting traffic across a global network of points-of-presence ngrok manages.

The overall goal is to provide a higher level of abstraction for managing network operations (NetOps) that can be programmatically invoked. NetOps takes the place of what are essentially the equivalent of Assembly programming tools to provision networking equipment, he added. The long arc of computing is moving toward higher levels of abstraction to make IT capabilities more accessible, noted Shreve.

That approach eliminates the need for IT teams to assign network operations teams to manage appliances in favor of a service that also provides a demilitarized zone (DMZ) via a service that accesses reverse proxies and firewalls, noted Shreve. The ngrok service ensures unauthorized requests are instantly blocked.

The ngrok service provides a programmable approach to managing network services that makes it simpler for DevOps teams to invoke them. Instead of waiting for a network operations team to provision networking equipment, an API makes it possible to integrate the provisioning of networking services within the context of a larger DevOps workflow.

One of the challenges that IT teams continue to encounter is that while virtual machines can be spun up in minutes, the provisioning of network services can require anywhere from minutes to days, depending on the age of the networking equipment. The ngrok service doesn’t eliminate the need for network professionals, but it does make it simpler for them to collaborate with application development and deployment teams, said Shreve.

It’s not clear if NetOps teams will one day simply become an extension of a DevOps team, but as applications become more distributed, the need to minimize network latency is becoming more critical. A service like ngrok makes it simpler for DevOps teams to ensure the appropriate level of bandwidth is available so distributed application performance doesn’t slow to a crawl.

Of course, the bigger challenge will be bridging the cultural divide that currently exists between many NetOps and DevOps teams. Both groups have their own distinct nomenclature for describing services that, in many cases, the other simply doesn’t comprehend. As distributed application development accelerates, each organization will need to determine how to demolish those silos so both teams can more easily achieve a common goal.

In the meantime, IT leaders might to reconsider their strategy as networking services become increasingly programmable via APIs. Most modern networking equipment today is software-defined in the sense that network operations teams have access to graphical tools to provision networking equipment. However, as APIs become more accessible, the need for those graphical tools may not be nearly as high as it is today as networking becomes just another service to be managed as code.

September 14, 2023
SmartBear to Acquire Stoplight to Advance API Governance

SmartBear announced today it plans to acquire Stoplight, a provider of tools for editing and governing application programming interfaces (APIs).

Sean Butler, vice president of product management for SmartBear, said the deal, once finalized, will add a set of complementary tools and open source software that extends the core API catalog platform that SmartBear provides to manage APIs. The open source software provided by Stoplight includes Prism, an HTTP mock server that emulates API behavior; Spectral, an API style guide enforcer and linter; and Elements, an API Documentation tool.

The governance capabilities are especially critical as organizations begin to deploy hundreds, even thousands, of APIs that are used to integrate the microservices that are at the core of modern cloud-native applications, noted Butler.

As those applications are deployed, the management of APIs is naturally becoming more challenging. Organizations, for example, might have a different set of rules for APIs that are externally facing than they do for internal APIs. However, it’s not uncommon for internal APIs to be made externally facing without updating them when a new use case is discovered.

SmartBear is advocating a “design first” approach to the building of APIs that ultimately serves to make them simpler to manage, said Butler. In fact, good governance provides the foundation upon which API security can be achieved and maintained at a time when the number of cyberattacks seeking to exfiltrate data via API endpoints is starting to increase, noted Butler.

Unfortunately, too many developers are still prone to building and deploying rogue APIs, many of which turn into zombie APIs that developers forget to remove long after the function they were designed to enable is no longer required.

It’s not clear how many organizations are adopting a design-first approach to building APIs before they write application code, but as more APIs are deployed, the way IT is managed is rapidly changing. Application services are essentially becoming more disposable, so long as the APIs used to expose them to other applications remain stable. In fact, it’s now only a matter of time before most organizations fully appreciate the degree to which their entire IT strategy now revolves around their APIs.

The challenge is that it’s not always clear who within an IT organization is responsible for API management. Although an API is created by a developer, that developer eventually moves on to other projects. Therefore, responsibility for managing and updating APIs often winds up shifting to DevOps teams that were not part of the API development process.

One way or another, the management of APIs will need to improve once legislation that holds organizations more accountable for how data is managed becomes law. The issue that many organizations will soon discover as they move to address those requirements is: A general lack of visibility into both the number of APIs they have and, just as importantly, how they are being employed.

Hopefully, advances in artificial intelligence (AI) will one day soon make it simpler to manage APIs. In the meantime, however, a modicum of governance applied today should go a long way to preventing a problem that otherwise is all but inevitable.

August 22, 2023
How to Prevent Zombie APIs

There is a growing reliance on web APIs. A full 92% of organizations plan to increase their investments into APIs over the next year, the 2023 Postman State of the API Report found. APIs are powering everything from internal microservices strategies to partner strategies and fully-fledged products.

Yet, this newfound API sprawl has consequences; a looming threat that might rise from the grave to haunt you …

I’m talking, of course, about zombie APIs. Zombie APIs are endpoints that are no longer maintained yet are still active. They may be unused endpoints, old features never officially deprecated or forgotten development or testing environments. Regardless, zombie APIs are a type of technical debt that could pose a legitimate threat if left to rot.

The sudden growth of APIs, along with immature inventory management and a lack of internal communication, can contribute to creating countless shadowy endpoints. Below, I’ll review some of the issues that zombie APIs might cause and consider some actionable steps DevOps teams can take to prevent them.

Understanding Zombie API Risks

Forgotten APIs can cause a number of issues. For example, suppose an insecure testing environment was accidentally left exposed to the public. An attacker using automated endpoint scanning could find authentication and authorization holes to exploit, potentially exfiltrating sensitive data.

It’s also a bad idea to let any piece of technology go unmaintained, given the daily influx of new exploits and vulnerabilities. This point is accentuated by the connected nature of APIs, which may plug into a vast network of databases and allow both read and write privileges to the backend infrastructure.

Insecure APIs can cause a business both economic harm and damage reputation. Just consider the high number of API-related data breaches in recent years, which occurred at companies like Optus, Dropbox, Twitter and Zendesk. It’s even estimated that somewhere between $12 billion to $23 billion was lost in 2022 due to compromises in web APIs.

Zombie APIs are an easy target for these sorts of cyberattacks because they tend to have insecure designs that are easier to crack. They also tend to use immature monitoring systems, which means hackers can act under the radar more easily than with actively used endpoints.

Tips to Avoid Zombie APIs

Keep an Active Inventory of APIs

As they say, you can’t secure what you don’t know about. In fact, OWASP lists improper inventory management as one of the ten top API security risks. Therefore, proper inventory management is a good first step to avoiding zombie APIs. Scan your service portfolio for forgotten endpoints, catalog them and update the catalog regularly.

It’s also a good practice to thoroughly document your services to aid in internal discoverability. If possible, adopting specification-first practices can also ensure the documentation and implementation are based on a single source of truth, thus avoiding mismatched versions and documentation drift.

“The key to preventing zombie APIs is visibility,” said Jeremy Snyder, CEO, FireTail. “Many organizations lack ongoing, up-to-date inventories of all the APIs that are running in either production or staging environments.”

Use Proper Versioning and Life Cycle Strategies

Another method to reduce the chance of zombie APIs is to adopt mature API life cycle management practices. The chosen API versioning strategy should be clearly defined and codified. And versioning changes should be clearly communicated to all stakeholders.

On that note, it’s just as important to plan out a sunsetting, retirement and deprecation strategy from the onset. Then, remove deprecated services when they have reached their end of life! This is not a glamorous task, but it should be baked into someone’s assignments.

“Good cyber hygiene is an often overlooked strategy,” explained Snyder. “A second requirement is to have a policy around versioning and how many versions of an API can exist at any given moment. For instance, that policy may state that only the latest version can exist or the latest plus the previous. However, this relies on having visibility in order to maintain compliance.”

Other areas of proper life cycle management include employing governance around development and deployment and utilizing automated testing in continuous deployment (CI/CD) pipelines. Both of these could help set policies and security requirements, which should also bolster API safety.

Share Knowledge of Internal and Third-Party Services

Tribal knowledge is the antithesis of DevOps culture. As such, it’s a good practice to de-silo and share internal knowledge of API-powered services to limit the propagation of rogue endpoints. This could be done through private network repositories or even an internal Postman Collection or Workspace.

“Internal processes like organizational coordination and communication are critical,” added Snyder. “If teams don’t tell each other about new APIs being developed and launched into production, security teams can’t assess or audit them. Conversely, if app teams don’t know about network infrastructure resources designed to cover various API security aspects, developers don’t know what security controls need to be baked into the API code.”

Visibility is Required to Avoid Zombie APIs

A recent CISO survey revealed that API security was the number one concern for CISOs. For many reasons, web APIs are still low-hanging fruit for attackers. This is underscored by the 400% spike in malicious API attack traffic recorded by Salt Labs earlier this year.

Many of these attacks, we’ve also learned, are occurring over authenticated APIs. This shows that greater accountability and visibility are required to begin to stem the tide of API-related security incidents.

To reach an adequate level of visibility, leaders should consider adopting tooling that helps determine if there are outdated APIs or APIs with multiple live versions, said Snyder. “These tools should help display what security live APIs have and provide recommendations around how to fix them, if not providing the necessary code or modules to secure the APIs directly.”

August 4, 2023
Postman Acquires Akita Software to Gain API Observability

Postman today announced it has acquired Akita Software to gain access to a platform for monitoring and observing application programming interfaces (APIs).

The first integration of the respective products offered by both companies will arrive in beta this fall.

Postman CEO Abhinav Asthana said the acquisition of Akita Software will make it much simpler for organizations to discover what APIs are being employed across an enterprise. A recent Postman survey found that 28% of developers are now deploying APIs at least once per week.

The Akita Software platform will make it simpler for organizations to discover both rogue APIs that have been created by a shadow IT project and zombie APIs that have been inadvertently left exposed long after they are needed.

Akita Software’s CEO Jean Yang said the integration with Postman’s API management platform will provide a rich source of data that can be used to add artificial intelligence (AI) capabilities to Akita’s API observability platform. For example, generative AI capabilities should make it simpler to create summaries in natural language that explain the calls an API is making to specific data sources.

In addition, the combined company will be in a better position to enable organizations to better secure their APIs after deployment by surfacing anomalous behavior indicative of an API that is being used to exfiltrate data, she added.

The merger with Postman will, in effect, enable the combined entity to provide a complete ecosystem for managing APIs, noted Yang.

Most APIs are used primarily to interconnect internal-facing applications and systems, with a subset used to integrate partners. Only a relatively small percentage of APIs are externally facing in the sense that they expose data to the internet. Many organizations, however, fail to appreciate how quickly an internally facing API can become externally facing over the course of its life cycle. As such, IT teams need tools and platforms that can manage and secure APIs as they evolve.

As APIs continue to play a crucial role in advancing business goals, there is a clear need for increased visibility as the types of APIs employed also continue to expand. Most APIs in use today are based on the REST format, but there are also APIs based on GraphQL, event-driven architecture and various messaging platforms. None of these newer APIs will supplant REST APIs any time soon. Rather, they are being applied to drive additional processes alongside REST APIs.

In the longer term, it’s not clear how many APIs will be managed by the developers that created them versus a centralized DevOps team. However, the more APIs that are deployed in a production environment, the less feasible it becomes for developers to manage their life cycle. The challenge is making sure there is some type of structured workflow in place that, at the very least, makes everyone involved in the process aware that a new API has been created and deployed.

July 19, 2023
State of API Report Highlights Productization and Generative AI
In today’s software landscape, an API-first strategy is increasingly popular. It’s the bedrock of how applications are built and connected. APIs help promote internal tech reusability, helping developers do more with less. Due to this increasing API reliance, maintaining quality control is increasingly important for internal and partner integrations and ensuring the reliability and security of these connections remains a top priority.

Due to the rising API ubiquity, nearly half of the total development energy is spent working with APIs. And to continually improve these interfaces, taking a product mindset is proving helpful. In fact, 60% of API developers now view their APIs as products. Generative AI has also shown incredible promise and is now being used in tandem with traditional tools to increase API code quality.

Postman recently released its fifth annual State of the API Report. The 2023 edition, which surveyed thousands of developers and API professionals, found subtle changes in tooling preferences and styles. It also sheds light on the ongoing API productization movement and API developers’ use of generative AI. Below, I’ll focus on these two key areas and review the report for its other key takeaways.

API Productization is Hot

The majority of API developers now view their API as a product, and this figure rises to 68% in large companies. A full 65% of respondents said their API generates revenue, too. This news is significant, as it demonstrates the industry is maturing its view of APIs. They are no longer offshoot developer toys but fully functional services contributing to overall business growth.

But how much growth? Well, of those monetizing APIs, 43% now say APIs generate a quarter of company revenue. For some companies, APIs generated more than 75% of total revenue! There also appears to be a parallel between API revenue generation and large companies.

Do you view your APIs as products? Most developers say yes.

Other industry reports unearthed similar API productization potential. Rapid’s 2022 State of APIs report found that 42.6% of organizations are now monetizing APIs. And investment into API-first products remains strong even amidst a broader contracted tech investment climate. These findings come as social media giants like Reddit and Twitter attempt to monetize their APIs.

That said, for 28% of companies, API revenue still equates to less than 10% of total revenue. APIs still don’t get the lion’s share, perhaps because they’re packaged with an existing platform or because of the many other indirect ways APIs might benefit the business. Furthermore, consumers are becoming more cost-conscious, as 60% cite pricing as a factor before integrating with an API. Still, the numbers are significant firsts for the burgeoning economy.

Most API Developers are Using Generative AI

Generative AI interfaces like ChatGPT and Bard have certainly taken the tech industry by storm due to their ability to generate code snippets, debug code and more. And for the first time, the State of the API Report spotlighted generative AI and its use within the API economy.

Today, 60% of API professionals are using generative AI on the job. API engineers commonly use generative AI to find mistakes in code, generate machine-readable code and increase learnability around their programs. Some even use generative AI to suggest potential security flaws in their APIs.

Compared with other app development endeavors, developers are by far most excited about building generative AI-powered applications in the year ahead. Yet, the efficiency gains are modest—many engineers forecast productivity improvements of 10% to 25%.

How are API developers using generative AI in their work?

Generative AI is set to accelerate many areas, including DevOps and API operations. Yet, contrary to some headlines, generative AI can’t automate all aspects of the software development life cycle. Rather, the technology is more likely to augment the API developer’s workflow than replace the need for seasoned professionals altogether.

Other Key Takeaways

Similar to last year’s report, integrating between internal applications, programs or systems remains a top concern for integrating an API or producing an API. The good news is that these integrations continue to increase in quality.

The report found that deployment failures and times-to-recovery are improving on average. More than half (56%) of respondents said API security incidents occur less than once a year within their organization. This shouldn’t overshadow the significant impact API security breaches can have but signals that DevSecOps is beginning to respond to persistent API vulnerabilities.

There are more API-first leaders than ever before. And API owners continue to evolve how their services are built and deployed. For example, REST is still by far the most common API style, used by 86% of developers. Webhooks made second place at 36%. However, GraphQL rose in usage, bumping SOAP into fourth position.

REST is still the dominant style, but GraphQL makes gains as SOAP slips.

Other Quick API Factoids
- More non-developers are becoming interested in APIs, as evidenced by 53% of survey takers being non-developers.
- Most APIs are private—61% percent of businesses’ APIs are for internal use only.
- After the technology group, business/IT services, financial services and health care are the most common sectors to adopt APIs.
- Microservices is still the dominant architectural style of APIs, at 42%.
- Performance, security and reliability are the top factors evaluated when integrating with a new API.
- Improper authentication, authorization or access control is the most commonly cited API risk.
- Common API tools include Postman, SwaggerHub, Datadog, SoapUI and Insomnia. Top gateway and API management platforms include AWS API Gateway, Azure API Management, Apigee and Kong.
- JSON Schema is by far the most popular API specification.
- The financial industry is a leader in API-first strategies—32% of respondents spend more than 20 hours a week working with APIs.
- Most developers deploy APIs to production once a week to once a month.
- When developers leave a company, their departure can cause outdated documentation, zombie APIs and a loss of institutional memory.
API Obstacles Remain

Looking to the future, investment in APIs is projected to climb. In fact, 92% of developers said their organization plans to invest the same or more time and resources into APIs in the coming year. Yet, some obstacles remain—a lack of documentation and poor service discoverability are still prominent roadblocks to API consumption. Furthermore, a lack of time and talent are major concerns that might stunt API development.

The Postman State of API Report surveyed over 40,000 developers and API professionals, making it the largest study conducted on APIs to date. It should be noted that, given Postman’s specific focus in the market and its surveyed pool, figures may be skewed to represent increased API enthusiasm. For deeper insights, you can read the entire report here.
July 17, 2023
Adopting a Low-Code GraphQL Approach to Build a Core Data Platform

Our education platform has relied on an outdated monolithic legacy application for years. It’s an outdated system built of a single service layer on top of a database designed for use cases that have evolved over the last decade. But we’re using the wealth of data in the monolith as a foundation for our current projects, thanks to our newly adopted tools.

With our product line continually expanding, our need for a centralized API that can sit over commonly used data has become even more critical. This includes data related to our customers’ organizational units, purchased products, licensing information, and student, teacher and class rosters, among other things.

The stakes are high since we have multiple downstream clients depending on us. Our goal is to move some functionality and data out of the main system while making sure our design is flexible enough to adapt to future product changes. We envision a hub-and-spoke model where all our products interact through a central hub to access shared data. Efficiency, security and reliability are all top priorities. Because we’re a small team, we need to move fast and be open to new ideas. That’s why we’re using a low-code backend solution to minimize manual coding and maximize our productivity.

A few months ago, we adopted a tool called Hasura. Its core value proposition is that it places a production-grade GraphQL API over any database, saving teams months of recurring effort in building, shipping and maintaining their APIs. We put the open source Hasura community edition to the test and found that it lived up to its claims. In this article, we’ll show you how it worked for us and how it can work for you too.

Converting GraphQL to SQL

Before a team can convert GraphQL to SQL, they must undergo a data modeling phase. After this data modeling phase, backend teams must complete some unavoidable tasks. One of these tasks is to build an API that allows clients to read and mutate data. Hasura provides this instantly, freeing engineers from the toil of writing basic CRUD APIs. Hasura compiles a GraphQL string into a GraphQL abstract syntax tree (AST), which is then compiled into a SQL AST. This abstraction lets us do powerful things, like using a bulk GraphQL mutation to transactionally and atomically insert or update data. GraphQL provides a flexible way for us to query our data in the form of a graph. This means that, as engineers, we get to focus more on business logic.

API Federation, Aggregation and Composition

Hasura’s support for remote GraphQL schemas enables us to design a service-oriented architecture in which Hasura functions as an API federation/composition layer and delegates to other Go microservices that provide specialized business logic for a particular domain. This ability to federate doesn’t just end at remote GraphQL schemas—Hasura also supports remote data sources, opening multiple possibilities for how we traverse through the graph and explore relationships in our data, including how we might join remote databases to each other or to remote APIs (REST or GraphQL).

In greenfield projects, teams typically spend months crafting a fully-featured API, a common language that all clients can share. With Hasura, we get this in seconds and can spend that saved time writing automated tests or implementing business logic.

Developer Dashboard

Hasura’s Dashboard/Console improves developers’ experience. Using the dashboard, developers can craft experimental GraphQL queries and mutations against our local instance of Hasura. The embedded GraphQL playground makes it a breeze for new developers joining the team to introspect and discover our API. We can also visualize our database schema in the UI directly and make changes to it with a few clicks of a mouse (if needed), generating source-controlled SQL migration scripts like you’d find in other web frameworks.

Observability

No production-ready system would be complete without observability, or the ability to gauge a system’s internal state based on external output, usually in the form of logs, traces and metrics. Hasura ships logs and traces to our New Relic telemetry platform for each database operation, allowing our team to identify bottlenecks and receive early alerts of unexpected or degraded system performance.

Authorization

Hasura’s role-based authentication support allows us to check an additional box on our production-readiness checklist. Clients with certain roles may have permission to execute specific GraphQL API operations or access particular database tables. Hasura can derive a client’s role in one of two ways: Either the client will send Hasura a JWT auth header (which includes a signature containing claims providing the client’s role), or Hasura will refer to an external webhook which, in turn, makes the auth decision. Without Hasura, we would need to write and maintain the code to perform all of this work.

Change Data Capture

The last way Hasura enables us to build our core platform is through change data capture (CDC), or the process of tracking changes to a database. Hasura provides this in the form of event triggers, which function like a transactional outbox system. When a record is inserted, updated or deleted, a bona fide database trigger will record this event in a table. Then, a reliable process in Hasura will guarantee that the event is delivered at least once to its intended webhook (one of our microservices). This is tremendously useful for our team since we need to sync data from our new database to the older legacy database.

There are many tools we could use for CDC, such as AWS SQS or Kafka (some of which we use downstream to integrate our platform with individual products), but having this feature baked into Hasura makes the process simpler, as it frees us from having to provision any new infrastructure or deal with the complexities of bringing on additional tools.

Conclusion

For the Teaching Strategies platform modernization team, we’re just getting started. As we continue to evolve and improve our product, we’re considering Hasura for finer-grained authorization, such as attribute-based access control (ABAC) over coarser role-based access control (RBAC). We will also investigate real-time streams so other systems, such as users’ browsers, can see data changes as fast as possible.

Using the functionality and features of Hasura community edition outlined above, our team is well on the way to building a strong core platform. By leveraging this tool, we can meet our milestones faster and provide even more support for educators and caregivers, making their lives easier.

Interestingly, Hasura is a combination of the programing language Haskell and the Sanskrit word asura—or demon. As far as tech choices go, this one feels much more like a guardian angel.

July 13, 2023
Must You Collect All That?

It is interesting that we no longer consider teams that super-tool their apps as being slightly ‘off’ or, working in a field where systems were mission critical, even life-and-death. Now, we all are collecting and reporting data points in real-time across not just the SDLC, but across the entire environment. It is wild the amount of data available to the average DevOps team about what is going on with their applications. But it’s not all being used.

No, seriously, it is unreal. Let me offer two anecdotes that explain why I know you’re not using it all.

Most of my regular readers know that I worked at F5 for years, the first couple on the DevCentral team, doing the Dev part. Their flagship product, the BIG-IP, collected an incredible amount of super-granular data. I am not exaggerating when I say an incredible amount. They had data points from the wire up to the traffic routes through the load balancing algorithm to nodes. And it was (and likely still is) all available to users via APIs. Whatever data was needed about traffic passing through the BIG-IP or the nodes behind the BIG-IP and the connections in between users and those nodes, you could get.

One of my projects was to offer a simplified interface to the APIs. They were complex and granular, and customers wanted fast and easy, so I was publishing code that worked as an intermediate layer.

Note: F5’s APIs, like everyone else’s, are far more advanced today than they were then, and it is easier to do what you need to do because they took the “create a user layer” part seriously. It’s complex machinery so it still has complex APIs, but accessibility is way up from those early days.

Anyway, BIG-IP collected so much data that I could not name all of the fields, let alone when each one was pertinent. I gave up trying after a while, because if a board-level engineer needed a data point, the idea at the time was “Why not expose it?” so ‘number of connections to a given node’ under this algorithm was hidden among things like ‘microseconds to connect to DNS on that connection’ (I’m making those up; it’s been years). Some were terribly useful to end users and others were not. All were accessed in the same way, and learning by experimenting was the only real way to tell the difference.

Users did not need all of that data. Users did not need the complexity and obfuscation created by supporting all of that unnecessary data.

My other example is a massive system of interconnected IoT devices that were centrally connected/controlled with a private, specialized network to manage them (actually, there were several different networks on several different mediums, and that matters where data points are concerned).

When setting this system up, we estimated about half a million IoT devices with capabilities that ranged from passive monitoring to active controllers of physical devices. Four different communications mechanisms, a couple dozen different devices, every major (and a couple minor) databases and command-and-control software on every major OS, managing from a handful to hundreds of thousands. And all of these were super-tooled because the IoT network was mission-critical. Our databases and the infrastructure to support/protect them were a major part of our data center budget because of the amount of data.

The amount of data was a flood. The management software was developed by different companies with different transports and endpoints in mind, meaning each had a different dataset. Once all of the networks were running satisfactorily, our next immediate project was to create a unified dataset for reporting and troubleshooting. I was in charge of IT, but the top-level person–because each of those half a million devices was a physical install–was a physical engineer. We fought non-stop about what was important. I was trying to map and convert datapoints between systems; he was trying to maximize data available for reporting up. He often said, “Well no, we don’t need that today, but we might.” This put us in a zone where “number of devices on this system” was on par with “attenuation of the communications line to the nodes on this physical subnet.” Totally unworkable, and our compromise was to put together a small IT team to put a layer over all that raw data and offer an intelligent interface side-by-side with data dumps.

We are there again today. I hear many of my friends in the enterprise space saying, “Because of ML, we might need that data and storage is cheap.” I disagree. Storage is an overhead that is wasted if you are not actively using the data. We are past the “all data is good data” craze; it’s time to adapt. Useful data that expands insights is all you need tooling for. Report data that will help solve problems, and if the team finds tomorrow that it needs a data point that is not currently collected, it can be added in.

There is room for debate here–the rate of change for ML systems (please stop calling them AI; they’re not, yet, and we’ll need that word when they are cognizant) means that there is a reasonable argument for “all of the data.” And I’m not proposing limiting data collection. Honestly, application security specifically and enterprise IT security overall has benefitted from datapoint sharing between formerly separate market spaces. If the SCA scan shows X framework is used, then the IAST can narrow down tests to those that look at X and not its competitors, for example. But honestly, don’t drown yourself in a data lake when all you need is a fish tank where you can see all the guppies.

And keep rocking it. Even if you entered IT two years ago, you are now a seasoned vet and things have changed–beyond all of our expectations, I’d say. Yet you’re still plugging away, keeping the enterprise running. Don’t stop—just try to keep the data hoarding to a reasonable level and make sure the data you have is being used, whether as useful feeds to an ML tool or as consumable information for other people/systems.

June 28, 2023
Noname Security Looks to Shift API Security Testing Left

Noname Security today made generally available an update to a tool for testing application programming interface (API) security that promises to make it easier for DevOps teams to ensure APIs are secure.

Filip Verloy, a field CTO for Noname Security, said Active Testing V2 is purposely designed to make it simpler to integrate API testing within a larger DevSecOps workflow.

Version 2 of Active Testing, for example, provides additional integrations with continuous integration/continuous deployment (CI/CD) platforms along with automated API discovery tools.

Other capabilities included in the latest release are tools to automate API response inspection, troubleshooting, classification and vulnerability discovery.

Finally, Noname Security has also added support for GraphQL-based APIs alongside REST APIs.

In total, there are now more than 160 API security tests that can be employed on an API testing tool that Noname Security developed to enable it to build Active Testing, said Verloy. In effect, Active Testing is an extension of that core API testing tool, he noted.

The biggest issue DevSecOps teams encounter when testing API security is that most legacy tools are not designed to provide any visibility into the business logic used to construct APIs. As such, there is a need for API security tools that can be easily integrated within DevSecOps workflows to discover issues before APIs are exposed in a production environment, he noted.

While a lot of progress has been made toward securing software supply chains, APIs have generally not received enough attention. Developers typically create APIs, but they don’t typically assume responsibility for managing and securing them. As the number of APIs within organizations continues to proliferate, there is a growing need to centralize API management and security.

Historically, developers have assumed that cybersecurity teams were making sure APIs are secure. Cybersecurity teams, conversely, assumed that the developers that created APIs would be responsible for securing them. DevSecOps best practices make it possible to now extend responsibility for security further left toward developers in a way that enables cybersecurity teams to define the policies and controls that should be used, noted Verloy.

It may a while before every organization masters the all the nuances of DevSecOps, but as pending legislation that requires organizations to better secure software supply chains becomes law it’s only a matter of time before organizations revisit API security—if they have not done so already.

In the meantime, cybercriminals have already discovered that insecure APIs provide a convenient mechanism for exfiltrating data that they can then sell or hold for ransom. The number of rogue and zombie APIs that are unknown or that have been forgotten has become a major cybersecurity issue. Ideally, DevSecOps teams would not only test the security of APIs before they are deployed but would also address a significant amount of technical API security debt that has already accumulated.

One way or another, API security is going to become a significant issue in the weeks and months ahead. The issue now is how to address it before it gets any worse.

June 21, 2023