So long REST, hello GraphQL
"I am happy as a clam. It's that good!" exclaimed one of the frontend engineers in our team at travel audience. At that very instant we realized the transition from a REST backend to GraphQL backend was worth the pain. In this post, we will briefly describe our journey.
What is GraphQL?
GraphQL is a query language similar to SQL, but for product APIs. Like SQL you can use GraphQL to specify the fields you are expecting your query to return. This significantly reduces the amount of data transferred over the network and increases the responsiveness of the client. GraphQL was originally born at Facebook because of frustrations caused by REST and FQL (Facebook Query Language).
Why GraphQL?
Roundtrips and bloated payload- At travel audience, we have a host of microservices written in Go that communicate with each other predominantly via gRPC. One of these microservices— the backend microservice, exposes REST endpoints which are consumed by a web frontend. As this frontend app evolved, the pages became complex and we had to perform multiple roundtrips to the server to display a single page. Furthermore, the payload contained unnecessary data which this frontend would never use. Let us consider an example of fetching a single blogpost and its author. As seen in the below diagram, to fetch a blogpost and its author using REST, we had to perform 2 roundtrips to the server and back.
Compare that to the 1 roundtrip to fetch the same amount of data using GraphQL. That’s twice as efficient as REST. The savings would be much higher if we had to fetch multiple authors for a single blogpost.
Type safety — The frontend team at travel audience uses typescript which is type safe. The backend team uses Go which is type safe. Our microservices communicate via gRPC which is yet again type safe. The only component in this setup which wasn’t type safe was our REST service. GraphQL gave us an opportunity to have an end to end type safe system.
Evolving microservices — Another major pain point for us is constantly evolving microservices. When any of the microservices, which the backend service depended on changed for e.g. addition/modification of a field, then the backend service had to be adjusted to reflect those changes.
Moving from REST to GraphQL would not only help alleviate performing multiple roundtrips to the server to display a single page but also achieve end to end type safety. A sweet bonus was that the GraphQL library we used enabled us to solve the ever-evolving microservices issue to some extent.
How?
Take One — graphql-go/graphql
Since most of our microservices are written in Go, and none of us have any experience writing javascript for which GraphQL has first-class support, it was quite natural for us to use a Go library for GraphQL to kickstart our journey. The following reasons prevented us from going ahead with this library:
- No compile-time type safety & reflection — graphql-go/graphql relies on empty interfaces and hence any type mismatch would not be caught during compile-time. Also, type inference happens at runtime via reflection.
- Verbosity — Since schema, queries, and mutations are declared in code, this made our codebase very verbose and clunky.
- Declaring Circular Dependency — We ran into an issue while defining types which reference each other. Defining types with circular dependency would cause the library to panic. The right way to do it was by declaring circular dependencies at the initialization of the package and this was counterintuitive for us.
Take Two — samsarahq/thunder
- Interface and union support — While we were researching on the various GraphQL libraries, we wanted the most feature-complete, stable and easy to use the library. Thunder was missing support for GraphQL interfaces and unions and hence we were a bit wary to use it.
- Thunder relies on reflection.
Take Three — 99designs/gqlgen
This was our last go at implementing our GraphQL endpoint using Go. As we began our foray into gqlgen, the following features impressed us immensely:
- Schema-first approach — Our team is cross-functional and using the Schema-first approach made collaborations easier because understanding GraphQL schema language is far easier than understanding code.
As you can see in the above code snippets, the schema-first version is far more readable and easier on the eyes then the code-first version.
- No reflection — One of the Go proverbs is “Reflection is never clear” which is why gqlgen does not use any reflection for type inference. Rather, it generates code which unfolds all the fields within a type and uses switch cases as shown below
- Most feature-complete — Among the libraries we were experimenting with, gqlgen was the most feature complete and had an active community of contributors.
- Code generation — Generating code free developers from the burden of writing boilerplate code. Another advantage of code generation is consistency. The code generator always generates code which you expect it to generate.
- Binding custom types to GraphQL types — The backend service which we spoke about earlier is essentially a REST wrapper for our microservices. It essentially talks to the various microservices via gRPC and exposes the fetched data to our web frontend via REST. If any of these microservices changed, the backend service needs to be updated to reflect these changes. This was very cumbersome.
The default behavior of gqlgen is to generate models for types specified in the schema file. But, you could also specify existing models in your codebase (e.g. database models) in the config file and gqlgen would re-use this model rather than generating models for the types specified in the schema file. In our case, we just had to specify the protobuf definitions (gRPC uses protocol buffers as the definition language and message format) and viola our protobuf definitions were bound to the types in GraphQL. Hence when any of the mircoservices changed, all we had to do was add/delete a field in the schema file and possibly a few lines of code to handle type conversion if required.
However, we faced a couple of problems by binding the gRPC definitions to the GraphQL endpoint:
- It exposed all the fields present in the protobuf definitions. This wasn’t something we intended to do.
- We needed some “extra” fields in GraphQL that were not present in our protobuf definitions.
To overcome these problems, all we had to do was specify the fields we wanted to expose along with the “extra” fields in the schema file. If the fields in the protobuf definition and the schema file were incompatible (e.g. int in schema and int64 in protobuf), a few lines of code would have to be added to perform the type conversion.
Our migration plan was quite simple. We first migrated a few of our GET endpoints to GraphQL queries to test the waters before going all in on GraphQL. Since we did not face any issues with this, we slowly started transitioning all our REST endpoints to GraphQL. Currently, our frontend consumes a mix of REST and GraphQL endpoints, but we are slowly moving to a GraphQL-only setup.
DataLoaders and performance tracing
Once we got the basics working, we wanted to improve the performance of our queries. Enter DataLoaders. Simply put, a DataLoader is a utility to prevent repeatedly fetching the same data in the context of a single request. Let us consider a simple query to fetch all blogposts along with their authors.
query{blogposts{name,author{name}}}The naive way of processing this query would be to fetch all blogposts and then fetch the author for each of the blogposts. But many of the blogposts might have the same author and assuming that we store the data in some sort of a relational database, it would be wasteful and unnecessary to repeatedly query the database for the author information in the context of a single request, once we have already fetched it. This is exactly the kind of problem which DataLoaders aims to solve. The way DataLoaders achieves this is by memoization. DataLoaders contain a load() function and after load()is called once with a given key, the resulting value is cached to eliminate redundant loads.
In the case of some complex queries, DataLoaders have helped us reduce our request processing time by almost half. We use dataloaden Dataloader library from vektah who also happens to be the maintainer of gqlgen and like gqlgen, dataloaden also uses code generation.
gqlgen supports performance tracing via opentracing and apollo tracing standards. The way it achieves it is by providing a tracer middlewarewhich accepts any type that adheres to the tracer interface. We use performance tracing to keep track of query response times.
In Conclusion
GraphQL has helped us achieve end-to-end type safety along with considerable performance improvements while significantly reducing the size of our codebase in comparison to REST. An added bonus has been the increased collaboration between the frontend and backend teams during API design.
This is just the beginning and we hope to incorporate more GraphQL features to help us solve problems at scale.
If you enjoyed reading this and would like to help us in our journey, we are always looking for curious minds to join our team.
