Author: Brecht De Rooms

  • Defining the Database Requirements of Dynamic JAMstack Applications

    Defining the Database Requirements of Dynamic JAMstack Applications

    Zero-configuration hosting solutions such as Zeit Now and Netlify are taking the frontend community by storm due to their ease of use. If we add static site generators such as Gatsby/Next.js/Nuxt.js/Hugo/Jekyll to the mix and deploy our statically generated pages to a content delivery network (CDN), we get something highly desired with minimal effort: distributed auto-scaling applications with zero configuration.

    The usage of static site generators and CDNs became quite popular, so much so that Netlify decided to give it a name: JAMstack. At first, this might not strike you as something very newsworthy. Haven’t we all deployed static sites such as blogs or documentation on CDNs? What is different about this approach?

    To answer that, let’s first look at the definition that is typically used to explain JAMstack: The JAMstack is a static HTML site, rebuilt automatically every time you update the content, and deployed directly to a CDN.

    Of course, that still does not explain why this approach is different. The first time I read this definition, I wondered how I could ever build something more dynamic than a blog with such a stack. However, note that nothing in the previous description says that every element of our site has to be statically generated. In fact, the JAM acronym stands for JavaScript, APIs and Markup, where the letter A is the answer to dynamic content.

    The idea is to pregenerate markup where you can and then enrich these statically generated pages with dynamic content fetched from APIs. The APIs of choice in a JAMstack application are typically APIs from third-party services (SaaS platforms) or custom-built with Functions as a Service (FaaS). Both Netlify and Zeit have great zero-configuration FaaS offerings to write APIs for your JAMstack.
    In essence, developers in the JAMstack community are often looking for a specific set of requirements:

    • Zero configuration—easy setup, easy deploy, no server management.
    • Scaling—your application can handle sudden spikes of increased traffic without extra effort. 
    • Speed—low latencies and fast page loads (thanks to the distributed nature of CDNs).
    • Pay as you go—only start paying when your page actually serves customers.

    By combining easy-to-use hosting solutions, static site generators, CDNs and scalable APIs in one stack, we can relatively easily get these requirements.

    What If My Application Requires Highly Dynamic Data?

    Imagine that you are building a social network for professionals on which professionals can create profiles. Professionals create profiles to promote their skills, and companies search the network to hire professionals for specific projects. For such an application, you need to store, search, filter and paginate data.

    To build such an application, you need a database, but not just any database. Ideally, you want to avoid losing the zero-configuration, pay-as-you-go, scaling and speed benefits that a JAMstack application typically delivers. There are several features a database can possess that make it a perfect fit for a JAMstack application.

    In this article, we will describe these features and explain how each of them makes a database a better candidate for a JAMstack application. Since such applications often rely on serverless functions, many of these properties also apply for applications built with FaaS.

    Database Properties that Work Well with JAMstack and FaaS Applications

    Multi-Region, Globally Replicated

    To understand why multi-region distribution is desirable, let’s revisit why static websites on CDNs are incredibly fast. A CDN is fast to deliver your content because it contains copies of your content at different locations. When content is requested from the CDN from a specific location, it will attempt to deliver that content from the closest location to the requestor. In order to get an idea of how much that matters, take a glance at the Zeit CDN status page which shows you the difference in latency between your current location and other locations. By deploying our applications to a CDN, our pages automatically load from the closest location to the user, which results in low loading latencies. And low latencies result in a great user experience.

    In order to keep this user experience, the dynamic data that will be loaded from our APIs has to exhibit low latencies as well, and the best way to achieve this is to use a distributed database. Databases that feature multi-region replicas such as FaunaDB, Google Spanner or Amazon Aurora not only provide the dynamic querying that you need, but also deliver your data from the closest location, just like a CDN.

    Connectionless

    Many traditional databases are not connectionless, which means that they require persistent connections with the process that queries the database. There is often a limit on the number of connections that a database can handle, and setting up a connection takes time and puts more strain on the database. A database that supports a JAMstack application will either be called from the frontend (if it’s possible to do securely) or from serverless functions. An increase in the number of users increases the number of running functions or frontend calls.

    Having a limit on connections and/or creating/destroying many connections then quickly becomes a problem. An ideal JAMstack database behaves like an API; it does not require connections. There is an excellent article on this topic that describes the connection problem from a FaaS perspective.

    Flexible Security

    Providing security out-of-the-box takes away tedious configuration from developers. If this security layer can also provide a means to securely access the database from the frontend, we can potentially write our application backendless, eliminating an extra hop and reducing the low latency we’ve gained from distribution even more. When a database is connectionless and can be accessed securely from the front-end, we consider it to be frontend-friendly. However, security requirements can be complex. If we completely rely on the database security layer, it has to be flexible enough to express custom rules. Firebase Cloud Firestore, FaunaDB and the upcoming Userbase offer direct access via clients, like browsers.

    Scaling and Pay-as-You-Go

    Before we add a database, our typical JAMstack application is completely auto-scaling and pay-as-you-g(r)o(w), which is considered an advantage for many startups. We only pay for the CDN bandwidth and function calls that our application actually uses. If our database is also pay-as-you-go and scales automatically, we do not lose this advantage.

    Strong Consistency

    Strong consistency is quite a technical matter, so we’ll explain it briefly, and for the technical details, this article is a great resource. Strong consistency is important in distributed databases when your application stores and updates data. In that case, there are two requirements to preserve a great developer experience:

    • You never lose data: When a write or update is acknowledged, it can no longer be dropped.
    • Data updates are predictable: When you save or update data, a consequent read will reflect these changes. 

    Strong consistency (or strict serializability) is the opposite of eventual consistency. By adopting a database that does what you expect it to do, you can focus entirely on your application. Only a few distributed databases provide strong consistency without limitations.

    Conclusion

    JAMstack applications are not limited to pages that are entirely static; they can be partly static and partly dynamic by retrieving data from APIs. A distributed database can be one of these APIs and is essential in case your application contains dynamic pages with filtered or paginated data. Ideally, such database fulfills the six described requirements so that we can build dynamic JAMstack applications without losing speed, scalability, pay as you go or the zero-configuration approach that we love.

    In case you wonder how a partly static, partly dynamic JAMstack can be created in an elegant way with modern frontend frameworks, take a look at hydration, which might be the topic of a forthcoming article.

    — Brecht De Rooms

  • Serverless and GraphQL: Build and Launch at the Speed of Thought

    Serverless and GraphQL: Build and Launch at the Speed of Thought

    Websites and apps are now, often, one and the same. Traditional development methods for an online or downloadable app can take months, given the backend development aspects that need to be managed—infrastructure provisioning and management, security, scalability, consistency and more, particularly for companies that have users around the globe and potentially very high usage demands.

    In addition, once the app is built, it can take several days to launch and ensure global availability. Once launched, the app can require ongoing maintenance and supervision to ensure availability and performance, as well as constant fine-tuning of the balance between over- and under-provisioning to manage costs.

    Serverless computing, when combined with GraphQL APIs, is the Holy Grail of application development. These technologies make it possible for developers to build and launch apps lightning fast by removing server provisioning and management concerns. Developers need only worry about building a compelling app with usability features that optimize engagement. Additionally, with serverless and GraphQL, an app can essentially be launched to a global user base within hours or even minutes, rather than days, while avoiding the need to manage servers. Let’s take a look.

    What is GraphQL?

    GraphQL is an API standard for building modern, data-driven applications based on the widely known HTTP Protocol. First created by Facebook in 2012, GraphQL was key to the social media giant shifting its mobile app from HTML5 to a native mobile app. Based on its success, Facebook made GraphQL available to the public as an open-source project in 2015.

    With GraphQL, you define an agnostic Domain Specific Language (DSL) around your data structure in terms of:

    • The programming language you use. As an API standard, GraphQL is open for implementations in many languages, such as JavaScript, Java and C#. You can essentially use the standard with your language of choice.
    • The database engine you use. GraphQL is only concerned with having the data ready, rather than the source of the data and where it is coming from, so you have the freedom to implement a GraphQL API with almost all database engines.

    What’s All the Fuss About Serverless?

    Serverless represents a major paradigm shift in the way we create and run web services and apps. Serverless computing operates in the cloud, freeing developers from the burden of thinking about back-end tasks, such as managing infrastructure, scaling servers and provisioning capacity and resources.

    Below, I’ve listed some specific benefits that serverless provides, looking specifically at serverless databases—many of these advantages are part of other serverless offerings as well. The overall impact is that with serverless technologies, developers only pay for what they use, or what the app uses depending on user demand. Developers can scale up to virtually infinite disk space and compute loads in seconds when traffic spikes hit, and scale back down when users go offline. The NoOps aspect of serverless means maintenance costs and burdens are removed. Additionally, global availability zones allow for replication between different locations and allow you to keep data where it’s needed—close to the user to minimize latency. Built-in encryption means developers can secure their apps with only basic security know-how.

    • Consumption-based billing.
    • Auto-scaling.
    • No operations overhead.
    • Global low latency.
    • Secure by default.

    When these benefits are combined with the advantages of GraphQL, one will notice a perfect synergy between the two technologies. Not only does the process become more automatic, the standard facilitates communication between internal teams and makes it easier for clients to integrate with your software, through:

    • API discoverability and automatic documentation.
    • Consolidation of data.
    • Language and data source-agnostic.
    • Type safety.

    Serverless Plus GraphQL Equals Developer Nirvana

    There are numerous examples of companies that have managed to build faster and cut costs with serverless. One of the most impressive demonstrations is when students in Australia managed to rebuild the census application and handle loads that the official $9 million system could not handle, at a fraction of the cost using serverless technology. GraphQL takes these advantages a step further.

    Airbnb, a company already known for its serverless adoption, claims to be moving 10x faster thanks to GraphQL, while Coursera noticed that their frontend developers found it easier to discover and query new data as their API evolves. These benefits are not just limited to a developer’s own company. For example, GitHub clients can now fetch data more efficiently and more easily thanks to the GraphQL-based public-facing API. GraphQL is even competing with SQL—when databases such as FaunaDB offer a native GraphQL API, users move to GraphQL at a rapid rate.

    Given the advantages of serverless in removing server provisioning and maintenance, and the freedom GraphQL gives developers when it comes to using programming languages and database engines, developers are essentially able to build and launch apps on a global scale as quickly as they think of them. And in the end, don’t we all want our customers to have a better experience?

    — Brecht De Rooms