How to federate remote data sources into GraphQL APIs

In this article, you will learn to use Hygraph and GraphQL to build a platform that federates data from multiple sources and exposes it through a single GraphQL API. But before you dive into the details of how to build such a platform, it's important to understand what content federation is and why it can be useful.

Content federation is a practice of aggregating and displaying content from multiple sources on a single platform or website. This can be accomplished through the use of APIs. Content federation has a variety of use cases, including e-commerce, travel, and stock management systems.

• It can facilitate e-commerce by consolidating product listings from multiple suppliers onto one platform.
• In travel, it can integrate flight, hotel, and rental car information from numerous providers.
• In stock management, it can provide real-time inventory data from various warehouses, enabling efficient tracking and management.

By the end of this article, you will have a deeper understanding of what content federation is and the benefits of using it. You’ll also put things into practice by completing a tutorial to build a restaurant blog that consumes a single GraphQL endpoint that exposes blog posts and restaurant information from the Yelp API.

#What is Content Federation?

As mentioned in the introduction, content federation involves aggregating and displaying content from multiple sources on a single platform or website. This can be accomplished through the use of APIs such as REST or GraphQL. Content federation is usually achieved using a platform that acts as a middleman, making all the API calls and presenting the data as a single unit to all clients that require it.

Hygraph, a headless content management system (CMS), allows you to federate content from both REST and GraphQL APIs. It refers to such APIs as “remote sources.” As a CMS, Hygraph enables you to create models of your content, which the content team can then use as a blueprint to add data to your system. These models contain fields with different data types, such as string, text, integer, float, and boolean. So a model for a blog post can have the following fields:

In addition to normal fields that contain normal data, Hygraph allows you to add fields for remote data—that is, data that doesn’t originate from your system. Fields that contain this data are referred to as “remote fields.”

What is a Remote Field?

A remote field is a part of a Hygraph model that contains data from a remote source. These fields allow you to define arguments that may come from other parts of your model or are hardcoded. These arguments are then passed as an API call to the remote source, and the returned data is returned with the rest of the data in your model.

The Yelp GraphQL API contains data about businesses like phone numbers, operating hours, exact location, and star rating. This data can be returned from a search query based on arguments like country and city. You can model this arrangement with a remote field and return search results based on country and city data that is present in your model. The image below shows the selection of the search query and the location, country, and categories arguments:

After defining this remote field, you can make queries, such as the one provided below. This query fetches a blog post's title and body content based on its slug. It also fetches details of restaurants that are in that country and city.

query FetchBlogPostsAndRestaurants($slug: String!) {
    blogPost(where: { slug: $slug }) {
      title
      body {
        html
      }
      restaurants {
        business {
          name
          alias
          display_phone
          photos
          rating
          is_closed
          location {
            city
            address1
          }
        }
      }
    }
  }

So essentially, remote fields allow you to spice up the data from your content database with data from external sources. All the infrastructure and retries have already been handled by Hygraph. You simply need to define your arguments and get your data.

Benefits of Content Federation

There are a number of benefits to using content federation. One of the main benefits is that it provides easier access to third-party APIs, allowing you to easily integrate and display data from a variety of sources. This can save time and effort compared to building custom integrations for each individual source.

Another benefit of content federation is that it promotes efficient workflows. By centralizing data from multiple sources in a single platform, it becomes easier for users to access and manipulate the data they need. This can be especially useful for businesses and organizations that need to manage large amounts of data from multiple sources.

#Building a restaurant blog with Hygraph and Yelp API

The remainder of this article is a tutorial that will walk you through the steps of building a restaurant blog using Next.js, Hygraph, and the Yelp API. The blog displays information about the different foods and lifestyles of various cities around the world. It also fetches the most highly rated restaurants in each city.

Prerequisites

To follow along with the tutorial, be sure you meet the following prerequisites. You’ll need:

1. Node.js version 14 or higher. You can download the latest LTS (Long Term Support) version from the official website.
2. NPM version 6 or higher. Most Node.js installations come with NPM, but if yours didn't, you can download it from its GitHub repo. To verify the presence of NPM, run npm --version in a terminal.
3. A Hygraph account. Sign up here.

Getting the starter code

This project comes with some starter code in a GitHub repository. The repo contains the files needed to run the restaurant blog, but some files are missing. You'll follow along to add the missing code and get the project running. To begin, you’ll need to clone the project and install dependencies. The commands for achieving this are listed below:

git clone https://github.com/vicradon/hygraph-crunch.git # clone the repo
cd hygraph-crunch# change-directory into the repo
npm install # install dependencies
cp .env.example .env.local # create an environment file by copying the example environment file

After running all the commands above in a stepwise fashion, you can go ahead to get your auth key and content URL from Hygraph.

Obtaining the auth key and content URL from Hygraph

Go to your Hygraph account and create a new project by clicking on the Add project button:

A modal appears when you click on the plus button for you to fill in the details of your project. You can set the project name as “Hygraph Crunch” and the description as “A restaurant blog that pulls in data from Yelp API.” Choose a region that is closest to your users, then select Add project.

Once the project has been created, you will be taken to the project’s dashboard. Go to the settings page and find the content URL.

Copy the High Performance Read-only Content API endpoint and paste it as the value of the CONTENT_API key in your .env.local file. Since you won’t be writing data from the frontend, you don’t need to use the normal content API that allows for read and write.

Now, scroll down to the “Permanent Auth Tokens” section and click on the Add token button.

In the modal that appears, set the name as “Draft Content Auth Token” and the description as “This token has access to the draft content of the project.”

Once you fill in the details, select “Draft” as the default stage of the token and click on the Add & configure permissions button.

Now that the token has been created, you will be navigated to the token’s settings page. You can go ahead to copy the token and paste it as the value of the AUTH_TOKEN key in your .env.local file. Also, be sure to select the Yes, initialize defaults option that gives the token read-access to the draft content in your project.

Your API key should now have the permission to fetch all the draft content you create.

Running the project

Now that you have your environment variables set up and the starter code in place, you can run the project and piece-in the missing details. Open a new terminal at the root directory of your project and run the command below to start the development server:

npm run dev

Your project should now be running on http://localhost:3000. If there’s already something running on port 3000, it’ll choose the next available port (i.e., 3001 and so forth).

You should see the following web page when you open port 3000 on your browser:

Creating the blog post model

The blog post model contains the fields required for the blog post. Navigate to the Schema section of your project’s portal and click on the Add button next to the MODELS dropdown. This should bring up a modal where you can set your model’s details.

You can set the display name as “BlogPost” and the description as “Create content for the restaurant blog. Add the country and city to see available restaurants.” Afterwards, click on the Add Model button.

You need to add five fields to the BlogPost model:

1. A single-line text field that will serve as the title of the blog post.

2. A slug field for accessing the blog posts through readable URLs.

3. A rich text field that will act as the blog post body.

4. A single-line text field that will contain the country name. (The country field that identifies the country the city is located in.)

5. A single-line text field that will contain the city name (The city field that identifies the city the content editors are writing about.).

Note: The country and city fields are single-line text fields that are are identical. The only difference is their names.

After adding all the required fields, your blog post model schema should have five fields, as pictured here:

Add content to your blog post model

Now that your blog post model is ready, you can add content to it from the content editor. Navigate to the content editor from the left side bar and click on the Add entry button in the top right corner:

This should bring up a form where you can fill in content:

You can use example content from the misc/content directory in your cloned repo. It contains five example blog posts. Fill in the details of your first blog post and click on the purple Save button to save the post as draft. Once you’re ready for production, you can promote a draft post to Published.

Querying data from Hygraph

After adding the example blog post content provided, you can now write queries to fetch that data. The frontend can then use these queries to display the data.

To write your first query, navigate to the “API playground” page linked on the left sidebar. When the playground loads, replace the existing query with the query below:

query BlogPosts {
  blogPosts {
    title
    city
    country
    body {
      html
    }
  }
}

This should return the specified fields of all the blog posts you’ve added.

Setting up a Yelp account

To set up a Yelp account, you will need to go to the Yelp developer website and click on the Sign Up button. After signing up, go to the app management page and create a new app.

When your app is created, you will be provided with a client id and an API key. Take note of your API key as you will use it on Hygraph.

Scroll to the bottom of the page and click on the Join the developer beta button. This will enable the GraphQL API for your API key.

Defining the remote source

Select the Schema link on the sidebar, then click the Add button next to the "Remote Sources" menu option. A form will appear, where you can set the display name, API type, base URL, and headers of the remote source.

• Select "GRAPHQL" as the API type
• Set the base URL as “ https://api.yelp.com/v3/graphql ”
• Click the Add headers button, then set a header with a key of "Authorization" and a value of "Bearer YELP_APP_API_KEY". Your YELP_APP_API_KEY is the key generated when you created your Yelp App.

Defining the restaurants Remote Field

Now that you have the remote source in place, you can define the remote field that will query the remote source. You can add a remote field in the same way you added other fields. This time, you will select “GraphQL” as the field type (GraphQL is the last field in the field selection sidebar).

Select the Search option and set location as “doc.city”, country as “doc.country”, and categories as “restaurants”. You also need to click on the Business option under “FIELDS” so that the businesses that are restaurants in the selected country and city are returned. After selecting all the required fields, click on the Add button.

Fetching the Remote Source with the queries

You can now modify the query you ran earlier to include the restaurant remote field:

query BlogPosts {
  blogPosts {
    title
    city
    country
    restaurants {
      name
      phone
      photos
      rating
    }
  }
}

Running this query returns all the blog posts and restaurants within the designated country and city. Only four fields of the restaurant subquery were selected, but you can return as much or as little as you need.

Adding the queries to the frontend

The last sections of this tutorial involve adding the queries so the frontend can display the data. The first step in doing so is to replace the content of the file blog-posts.ts in the src/api/queries folder with the content below:

import { gql } from "graphql-request";

export const FETCH_BLOG_POSTS = gql`
  query FetchBlogPosts {
    blogPosts {
      id
      title
      slug
    }
  }
`;

export const FETCH_BLOG_POST_AND_RESTAURANTS = gql`
  query FetchBlogPostsAndRestaurants($slug: String!) {
    blogPost(where: { slug: $slug }) {
      title
      body {
        html
      }
      city
      restaurants {
        name
        alias
        display_phone
        photos
        rating
        is_closed
        location {
          address1
        }
      }
    }
  }
`;

The FETCH_BLOG_POSTS query in the file content above fetches the title and slug of all the blog post content you created so that they can be displayed on the landing page of your restaurant blog. The FETCH_BLOG_POST_AND_RESTAURANTS query returns the in-depth details of a single blog post, like the HTML content and the restaurants that match the specified city and country.

Displaying the blog posts on the home page

To display the blog post’s titles and make them accessible through their slug, you need to add this TSX to your pages/index.tsx file.

First, add the component for fetching the data:

...
                        Featured Blog Posts
          </Heading>

          {/* Add the code below to the file */}
          <Grid
            width={"100%"}
            maxW={"800px"}
            margin={"0 auto"}
            columnGap={6}
            templateColumns={{
              base: "repeat(1, 1fr)",
              md: "repeat(2, 1fr)",
              lg: "repeat(3, 1fr)",
            }}
            rowGap={6}
            justifySelf={"center"}
          >
            {blogPosts.map((blogPost) => {
              return (
                <BlogPostCard
                  key={blogPost.id}
                  title={blogPost.title}
                  slug={blogPost.slug}
                />
              );
            })}
          </Grid>
          {/* Add the code above to the file */}
        </Box>
      </Box>
        </>
...

Then update your imports to match the new additions. Find the updated import definitions in the snippet below:

import { Box, Flex, Heading, Text, Grid } from "@chakra-ui/react";
import Head from "next/head";
import { FETCH_BLOG_POSTS } from "api/queries/blog-posts";
import client from "api/client";
import { BlogPost } from "utils/types";
import BlogPostCard from "Components/BlogPostCard";

Finally, fetch the data using getStaticProps and import this data into the page. The updates you need to make are outlined below:

...
import BlogPostCard from "Components/BlogPostCard";

const Home = ({ blogPosts }: { blogPosts: BlogPost[] }) => {
    return (
    <>
...

export default Home;

export async function getStaticProps() {
  const data = await client.request(FETCH_BLOG_POSTS);
  const blogPosts: BlogPost[] = data.blogPosts;

  return {
    props: {
      blogPosts,
    },
  };
}

If you check the browser, you should now see the blog posts.

Display the blog post content when a post Is clicked

If you click on one of the cards, you should see the title and content of the blog, but without the restaurants. Add the code below to display the restaurants:

...
<Box
        className="blog-post-content"
        dangerouslySetInnerHTML={{ __html: body?.html || "" }}
        mb={"2rem"}
      />

      {/* Add the code below to the file */}
      <Box>
        <Heading mb={"1rem"} as={"h2"}>
          Restaurants in {city}
        </Heading>

        <Grid
          gridTemplateColumns={{
            base: "repeat(1, 1fr)",
            md: "repeat(2, 1fr)",
            lg: "repeat(3, 1fr)",
          }}
          columnGap={6}
          rowGap={6}
          justifyContent={"center"}
        >
          {restaurants?.map((restaurant) => {
            return (
              <RestaurantCard
                key={restaurant.alias}
                name={restaurant.name}
                display_phone={restaurant.display_phone}
                photos={restaurant.photos}
                rating={restaurant.rating}
                isClosed={restaurant.is_closed}
                location={restaurant.location}
              />
            );
          })}
        </Grid>
      </Box>
      {/* Add the code above to the file */}
    </Box>
  );
}
...

Now, import the Grid component from Chakra UI and the RestaurantCard component from the components folder:

import { Box, Heading, Grid } from "@chakra-ui/react";
import RestaurantCard from "Components/RestaurantCard";
...

Finally, destructure the city and restaurant data from the blogPost prop:

...

function SingleBlogPost(props: Props) {
  const {
    blogPost: { title, body, city, restaurants },
  } = props;

After adding all the code in the right places, you should now get the restaurants returned with the blog post data.

#Deployment considerations

Now that you have completed the tutorial, you may want to deploy your app. You should set all the draft content you want to move to production as Published, then define an Auth token for Published content.

Since the app was written in Next.js, you can easily use Vercel for deployment. You simply need to set the CONTENT_API and AUTH_TOKEN environment variables on the production server.

#Conclusion

In this article, you explored the concept of content federation and learned how it can be used to aggregate and display data from multiple sources on a single platform. You also looked at how to build a platform that federates data from multiple sources using Hygraph and GraphQL, including how to define your schema, link external APIs, manage content, and develop the frontend. By following these steps, you can create a platform that offers easy access to a variety of data sources for your users.