Tag: GraphQL

  • Why GraphQL is a Better Choice for Building Microservices

    Why GraphQL is a Better Choice for Building Microservices

    There’s been a long debate about whether it’s better to use REST or GraphQL for building microservices. Both technologies have their proponents and critics, but when it comes to the specific needs of microservices architectures, GraphQL emerges as the clear front-runner. Here’s why.

    Understanding the RESTful Concerns

    While REST has been the go-to API style for many years, lauded for its simplicity and general applicability, its limitations become glaringly apparent in the context of microservices. These limitations include:

    • Over-fetching of data
    • Multiple HTTP requests for related data
    • Complex versioning strategies

    Such issues can impede microservices architectures’ performance and scalability.

    Over-fetching of data: REST APIs are designed to return a fixed set of data, which often results in over-fetching. This redundancy is particularly problematic in mobile networks, where every extra byte of data can lead to slower applications and degraded user experiences.

    For example, imagine we have a REST API endpoint /api/user/{userId}, which returns the following data for a user:


    In scenarios where mobile apps only require specific user details like name and email for a profile overview, fetching comprehensive user data means over-fetching.

    This excess data usage can be costly for users with limited data plans and result in slower app performance and degradation of user experience.

    Latency and the N+1 problem: Microservices often require data spread across multiple services.

    Suppose you have an ecommerce application where the order domain handles product, customer, order and return entities, and the stock domain manages product, stock, warehouse and delivery entities. A common operation might be to display order details along with the current stock status for each product.

    Let’s say we have an order microservice and a stock microservice to handle orders and stocks.

    In our scenario, a customer places three orders (N=3) and each order contains four different products. To display the order details with the stock status for each product, the application initially makes one request to fetch the orders. Then, for each product in each order, it makes additional requests to the stock microservice to retrieve stock information. This results in one initial request, plus 12 subsequent requests (three orders times four products per order), summing up to 13 API calls. This multiplicative increase in requests leads to higher latency due to multiple round-trip times (RTT) and increased load on the network and servers, embodying the N+1 problem.

    Versioning challenges: Maintaining REST APIs over time involves complex versioning strategies, such as introducing new endpoints or embedding version numbers in the API path. This can lead to bloat and confusion, complicating API development and consumption.

    The GraphQL Advantage

    GraphQL addresses many REST weaknesses.

    Domain-Driven Design: Microservices thrive on a domain-driven design, where each service is built around a specific business capability. GraphQL’s schema-centric approach aligns perfectly with this, enabling a more organized and coherent structure for microservices.

    Unified schema: In a microservices architecture, each service may have its schema, representing a portion of the business domain. GraphQL stands out by allowing these schemas to be combined or stitched together, presenting a unified interface to clients. This means that clients can query data from multiple services in a single request, drastically reducing the complexity and number of network calls.

    Solving the N+1 problem: GraphQL’s ability to fetch data with a single request, regardless of how many underlying services need to be called, directly addresses the N+1 problem inherent in REST architectures. This not only improves performance but also simplifies client-side data fetching logic.

    For the N + 1 problem described earlier, if the services support GraphQL, it can be used to fetch nested data in a single query, effectively solving the N+1 problem.

    Efficiency and performance: By allowing clients to specify exactly what data they need, GraphQL eliminates over-fetching and under-fetching issues common in REST APIs. This leads to more efficient data retrieval, reduced bandwidth usage, and faster applications, particularly noticeable in mobile environments.

    In the earlier scenario where mobile apps only require specific user details like name and email for a profile overview, clients can request name and email alone.

    Simplified versioning: Unlike REST, GraphQL reduces the need for versioning by allowing new fields and types to be added to the schema without impacting existing queries. This forward compatibility means that clients and servers can evolve more smoothly over time. GraphQL stands today as a mature technology that addresses specific needs in modern web development that are not fully met by traditional REST APIs. Its design promotes efficiency, flexibility, and developer productivity.

    Looking ahead, the future of GraphQL is bright, with community-driven efforts focused on addressing its current limitations, particularly around security, performance and standardization. As these efforts bear fruit, GraphQL’s adoption is expected to widen, consolidating its position as a key technology in the API landscape.

  • GraphQL API Visibility Ensures Enterprises Can Escape Lock-In

    GraphQL API Visibility Ensures Enterprises Can Escape Lock-In

    While GraphQL is still early in its enterprise adoption curve, the runtime query language (and replacement for REST APIs) has already built an impressively robust ecosystem for jumpstarting major enterprise-grade deployments. Among the appeals of GraphQL are its open source implementations and breadth of available server types, which combine to offer enterprises flexibility and the benefits of best-fit solutions.

    However, operating GraphQL in production, maintaining GraphQL APIs at enterprise scale and achieving the requisite GraphQL visibility and security require complementary tools and services. This is where enterprises can fall prey to vendor lock-in and lose the flexibility to change and control their own deployments if they aren’t careful.

    Enterprises need vendor partners to manage their GraphQL deployments effectively, and vendors are glad to offer comprehensive SaaS solutions. But enterprises must understand that a vendor can seek to capture their long-term business by coercion—straitjacketing customers by hosting their entire GraphQL infrastructure and GraphQL gateway and making it, well, not exactly easy to let go. They can unleash particularly nefarious practices, using proprietary features to force customers to migrate to certain technologies or purchase expensive upgrades to keep their crucial GraphQL deployments afloat. Enterprises must be wary of allowing this vendor lock-in to happen for their own good.

    Knowledge is essential to maintaining the balance of power in vendor relationships. Enterprises must understand their GraphQL graph and key measures to correctly assess current vendors and maintain agility and control over deployments by not entrusting their entire fate to any one partner.

    These are the specific areas where enterprises should seek out intelligence and reliable metrics in order to choose their own future with GraphQL:

    Understand Your Overall Graph (and Subgraph) Utilization

    Establishing an accurate overview of your graph utilization—and harnessing actionable insights—are crucial, foundational steps that lay the groundwork for deeper analysis and better decision-making. You need to know what graph and subgraph objects and operations your most important API users heavily engage with and which objects and operations are underused or not used at all. You need visibility into who is using GraphQL APIs and how. Getting to actionable intelligence should also tell you where errors are most likely to occur and what’s causing them, as well as where major performance bottlenecks pop up during heavy load events.

    With this overview available, the next steps call for building a depth of knowledge in each of these five areas:

    1) Understand areas of heavy graph and subgraph usage

    Knowing exactly where heavy usage occurs in your graph puts you in a position to upgrade or migrate your GraphQL API server or gateway on your terms while mitigating outage and error risks. Specifically, you want to know which operations in your graph are executed most and how often, which graph objects and fields are consumed most often, and which subgraphs support those most heavily burdened areas in your supergraph. This knowledge and some careful planning help to put successful GraphQL server or gateway changes within your power and also reduce risk when introducing changes to areas of the graph under heavy consumption.

    2) Understand where your graph is underused

    GraphQL migration and modernization efforts are simpler—and carry less risk—when you first remove any unused functionality and reduce your scope as much as possible. To do so, you need visibility into your graph’s least executed operations, objects and fields, and subgraphs, as well as any operations with zero recent usage. Then, flag low-use graph areas you can deprecate, replace or just remove to simplify your deployment before making changes.

    3) Understand your GraphQL API users and usage

    You want a seamless experience for your GraphQL API users throughout any changes you make, especially for the users and graph-usage patterns most crucial to your business. That means knowing the frequent users of the areas of your graph under the most heavy usage. You also want to recognize and bolster any GraphQL operations that your most important users frequently call, in addition to the objects and fields those users consume.

    4) Understand graph errors and potential error hotspots

    The right actionable intelligence will prepare enterprise teams to anticipate and address graph errors. This requires visibility into where, when, why, and how often graph errors occur and error hotspots (for example, high P95 latencies indicative of potential timeouts). Insights into which users feel the brunt of graph errors are also valuable in shaping mitigation strategies.

    5) Understand graph performance bottlenecks

    Bottlenecks caused by your GraphQL server or gateway itself may call for a shift in your graph’s underpinnings. Bottlenecks can also arise from costly queries of substantial depth and height, especially when these queries see intensive use.

    To foresee and address bottlenecks, actionable insights include the average and P95 latencies of your graph’s most-called operations and the heights and depths of your graph queries with the worst performance metrics. You also want to know the number of times that bottlenecked operations get called each day, the subgraphs at the root of bottlenecks, the latency added by supergraph query planning, and which users are impacted the most by bottlenecks.

    Graph Knowledge = The Power to Avoid Vendor Lock-In

    Implementing the GraphQL API visibility and actionable intelligence strategies covered here help ensure that an enterprise has the means to modernize or migrate as necessary to preserve its strategic independence. Those unfortunate organizations already in the grip of vendor lock-in may need to bide their time but should still use these same techniques to improve and decouple their fundamental GraphQL services from their vendor-controlled GraphQL technology. There’s no acceptable reason why organizations relying on free and open source GraphQL servers and gateways should find themselves stuck in proprietary software arrangements. Thankfully, the right knowledge can deliver enterprises a more flexible future as they modernize API development with GraphQL.

  • Adopting a Low-Code GraphQL Approach to Build a Core Data Platform

    Adopting a Low-Code GraphQL Approach to Build a Core Data Platform

    Our education platform has relied on an outdated monolithic legacy application for years. It’s an outdated system built of a single service layer on top of a database designed for use cases that have evolved over the last decade. But we’re using the wealth of data in the monolith as a foundation for our current projects, thanks to our newly adopted tools.

    With our product line continually expanding, our need for a centralized API that can sit over commonly used data has become even more critical. This includes data related to our customers’ organizational units, purchased products, licensing information, and student, teacher and class rosters, among other things.

    The stakes are high since we have multiple downstream clients depending on us. Our goal is to move some functionality and data out of the main system while making sure our design is flexible enough to adapt to future product changes. We envision a hub-and-spoke model where all our products interact through a central hub to access shared data. Efficiency, security and reliability are all top priorities. Because we’re a small team, we need to move fast and be open to new ideas. That’s why we’re using a low-code backend solution to minimize manual coding and maximize our productivity.

    A few months ago, we adopted a tool called Hasura. Its core value proposition is that it places a production-grade GraphQL API over any database, saving teams months of recurring effort in building, shipping and maintaining their APIs. We put the open source Hasura community edition to the test and found that it lived up to its claims. In this article, we’ll show you how it worked for us and how it can work for you too.

    Converting GraphQL to SQL

    Before a team can convert GraphQL to SQL, they must undergo a data modeling phase. After this data modeling phase, backend teams must complete some unavoidable tasks. One of these tasks is to build an API that allows clients to read and mutate data. Hasura provides this instantly, freeing engineers from the toil of writing basic CRUD APIs. Hasura compiles a GraphQL string into a GraphQL abstract syntax tree (AST), which is then compiled into a SQL AST. This abstraction lets us do powerful things, like using a bulk GraphQL mutation to transactionally and atomically insert or update data. GraphQL provides a flexible way for us to query our data in the form of a graph. This means that, as engineers, we get to focus more on business logic.

    API Federation, Aggregation and Composition

    Hasura’s support for remote GraphQL schemas enables us to design a service-oriented architecture in which Hasura functions as an API federation/composition layer and delegates to other Go microservices that provide specialized business logic for a particular domain. This ability to federate doesn’t just end at remote GraphQL schemas—Hasura also supports remote data sources, opening multiple possibilities for how we traverse through the graph and explore relationships in our data, including how we might join remote databases to each other or to remote APIs (REST or GraphQL).

    In greenfield projects, teams typically spend months crafting a fully-featured API, a common language that all clients can share. With Hasura, we get this in seconds and can spend that saved time writing automated tests or implementing business logic.

    Developer Dashboard

    Hasura’s Dashboard/Console improves developers’ experience. Using the dashboard, developers can craft experimental GraphQL queries and mutations against our local instance of Hasura. The embedded GraphQL playground makes it a breeze for new developers joining the team to introspect and discover our API. We can also visualize our database schema in the UI directly and make changes to it with a few clicks of a mouse (if needed), generating source-controlled SQL migration scripts like you’d find in other web frameworks.

    Observability

    No production-ready system would be complete without observability, or the ability to gauge a system’s internal state based on external output, usually in the form of logs, traces and metrics. Hasura ships logs and traces to our New Relic telemetry platform for each database operation, allowing our team to identify bottlenecks and receive early alerts of unexpected or degraded system performance.

    Authorization

    Hasura’s role-based authentication support allows us to check an additional box on our production-readiness checklist. Clients with certain roles may have permission to execute specific GraphQL API operations or access particular database tables. Hasura can derive a client’s role in one of two ways: Either the client will send Hasura a JWT auth header (which includes a signature containing claims providing the client’s role), or Hasura will refer to an external webhook which, in turn, makes the auth decision. Without Hasura, we would need to write and maintain the code to perform all of this work.

    Change Data Capture

    The last way Hasura enables us to build our core platform is through change data capture (CDC), or the process of tracking changes to a database. Hasura provides this in the form of event triggers, which function like a transactional outbox system. When a record is inserted, updated or deleted, a bona fide database trigger will record this event in a table. Then, a reliable process in Hasura will guarantee that the event is delivered at least once to its intended webhook (one of our microservices). This is tremendously useful for our team since we need to sync data from our new database to the older legacy database.

    There are many tools we could use for CDC, such as AWS SQS or Kafka (some of which we use downstream to integrate our platform with individual products), but having this feature baked into Hasura makes the process simpler, as it frees us from having to provision any new infrastructure or deal with the complexities of bringing on additional tools.

    Conclusion

    For the Teaching Strategies platform modernization team, we’re just getting started. As we continue to evolve and improve our product, we’re considering Hasura for finer-grained authorization, such as attribute-based access control (ABAC) over coarser role-based access control (RBAC). We will also investigate real-time streams so other systems, such as users’ browsers, can see data changes as fast as possible.

    Using the functionality and features of Hasura community edition outlined above, our team is well on the way to building a strong core platform. By leveraging this tool, we can meet our milestones faster and provide even more support for educators and caregivers, making their lives easier.

    Interestingly, Hasura is a combination of the programing language Haskell and the Sanskrit word asura—or demon.  As far as tech choices go, this one feels much more like a guardian angel.

  • Hasura Unfurls Data Delivery Network Based on GraphQL

    Hasura Unfurls Data Delivery Network Based on GraphQL

    At its HasuraCon 2023 conference today, Hasura launched a Data Delivery Network (DDN) based on a set of GraphQL application programming interfaces (APIs) that makes it simpler to integrate distributed sources of data.

    In addition, Hasura is adding Hasura Schema Registry to streamline API governance using graph technology, a MongoDB connector, a Native Data Collector (NDC) for REST APIs and support for the Open Domain Data Specification (OpenDDS), an open framework for modeling APIs originally developed for the travel industry.

    Finally, Hasura is adding support for Native Queries and Logical Models capabilities that convert SQL requests into API calls.

    The Hasura DDN leverages the GraphQL API to enable sub-second integrations across a network that, in concept, is similar to how a content delivery network (CDN) is constructed.

    Hasura CEO Tanmai Gopal said the goal is to enable sub-second integration of data at a global scale across a network with more than 100 regions. Developers can integrate their APIs in less than a second regardless of the number of data models connected. Metadata changes can also be used to create builds that can be independently tested and applied to projects to enable API version control and collaboration among development teams.

    The Hasura DDN is accessed via Hasura Cloud, an instance of a data integration platform that can be deployed on multiple clouds, including Microsoft Azure, and will soon be made available for private clouds managed by internal IT teams.

    GraphQL is gaining traction as an alternative to REST APIs for integrating data, but it’s more complex to implement and manage. The Hasura DDN extends the reach of a Hasura Cloud platform that provides a simpler method for adding GraphQL capabilities to an IT environment in a way the also conserves network bandwidth, noted Gopal.

    GraphQL is a query language for APIs and a runtime for fulfilling queries that provides a complete and digestible description of the data in an API. It can also be used to pull data from multiple sources because it uses a schema that defines the different types of data available. REST APIs, in contrast, can only fetch data from a single source. As such, many organizations that are trying to drive adoption of reusable digital services are relying on GraphQL.

    It’s not always clear who within an organization is assuming responsibility for managing the APIs that developers create, but as the number of APIs increases, DevOps teams, in many cases, are moving to centralize API management. In fact, the more data an organization creates, the greater the corresponding use of APIs to programmatically access that data becomes.

    Hasura is betting that as organizations reach a critical mass of API integration requirements, it will make more economic sense to rely on a purpose-built data integration platform accessed as a cloud service versus relying on a set of tools and platforms they need to configure and manage. Regardless of approach, the one thing that is certain is the number and types of APIs that need to be managed are rapidly exceeding the ability of IT teams to keep pace.

  • Data APIs: Realizing the Future of Data Warehousing

    Data APIs: Realizing the Future of Data Warehousing

    Data is the currency of business in today’s digital economy, with organizations collecting mountains of data from their customers, products and services. Enterprises are increasingly turning to data warehouses to store this valuable enterprise data and to make it useful and actionable. Data APIs–both GraphQL and REST–are emerging as a tool to power frictionless and high-quality integrations between applications/services and data in enterprise data warehouses. But what exactly is driving this shift toward using APIs on top of data warehouses? In this article, I will talk about the drivers behind the rise of data APIs on data warehouses, challenges with current approaches to building, operating and scaling data APIs and how organizations can address those challenges.

    The Rise of Data APIs

    Data APIs enable organizations to create powerful integrations between applications without having to go through lengthy development cycles or write complex code. As a result, businesses can deliver improved user experiences faster than ever before by quickly connecting disparate applications and services together using intuitive REST or GraphQL endpoints. Because these endpoints are standardized across multiple systems (e.g., REST or GraphQL), developers don’t have to learn new languages or coding techniques every time they want to connect two different applications together. This makes it much easier for developers to work with multiple systems simultaneously while still ensuring that all applications remain connected via one interface–the API endpoint itself.

    Challenges With Current Approaches

    Despite all these advantages provided by Data APIs, there still remain a few challenges that organizations need to consider when building out their own solutions. For example, many existing approaches require manual setup processes, which can be extremely time-consuming and tedious for developers working with multiple systems at once. In addition, many existing solutions only allow for limited query functionality, which means that more complex queries may not be possible without additional coding or manual setup processes being performed. Finally, some existing solutions require an extensive amount of organizational resources in order to build out a full-fledged solution—time that could be better spent on more important tasks like product development or customer service initiatives.

    How Organizations Can Address These Challenges

    Fortunately, there are now tools available that make it easy for organizations to quickly build out powerful data API solutions with minimal effort required from their developers or IT staff members. These tools provide pre-built templates that can be used as a starting point for any integration project while also allowing users to customize their own solution based on specific needs and requirements without having to write any code themselves. Additionally, these tools allow users access to advanced query functions such as joins across multiple tables so they can get the most value out of their databases without compromising either speed or security either. Finally, these tools also include powerful monitoring capabilities so users can ensure that their data APIs remain running smoothly at all times, no matter how much traffic they receive.

    Benefits of Using an API for Integrating Cloud Data Warehouses

    When it comes to integrating cloud data warehouses into your application or service, there are several advantages that come with using an API as opposed to traditional methods such as manual coding or ETL tools:

    • Increased Flexibility: By leveraging an API, you are able to access data from multiple sources quickly and easily without having to manually code every integration yourself. This also allows your application or service more freedom when it comes to accessing additional data sets in the future; all you need is an existing API connection in place.
    • Improved Security: Since APIs are designed specifically for secure communication between different systems, they offer enhanced levels of protection against malicious attacks such as DDoS attacks or unauthorized access attempts. You can also add additional layers of security by implementing authentication protocols like OAuth or JWT tokens on top of your existing API connection.
    • Enhanced Performance: APIs can be designed with specific performance optimizations in mind; this means that you can create custom solutions tailored towards the exact needs of your application or service which will ultimately result in improved end-user experiences and higher engagement rates from customers who interact with your product or service regularly.

    Data warehouses have become an essential part of modern digital businesses due to the sheer amount of valuable enterprise information they store. However, traditional approaches can often lack the flexibility or scalability needed in order for businesses to stay competitive in today’s ever-changing market environment. This is why smart organizations are using data APIs on top of them for quick integration among disparate applications/services along with advanced query capabilities.

    By leveraging tools designed specifically for this purpose, companies can easily overcome current challenges associated with building out their own solutions while still getting maximum bang for their buck when it comes down to successful implementation and operation over the long term.