React rich previews using CloudFront and Prerender

But what we want is something more like this, where specific detail from the page is exposed in the link preview.

Most websites and services conform to Facebook's Open Graph protocol when deciding what to show in a preview.

The solution: Prerender + CloudFront = 😍

To solve this problem, we need some way of providing these services with access to our page-specific meta tags. One way to do this would be to implement server-side rendering which we decided not to pursue given the time and infrastructure this would require us to set up and maintain. Another way to achieve this to use a service like Prerender to cache static HTML vesions of our pages and return them to known social bots. We decided to go down this path because it worked well with our existing stack and was an easy two-way door in a way that moving to SSR would not have been.

Our stack currently runs on AWS infrastructure, in particular CloudFront as a CDN and S3 as a file store. CloudFront can be customised using Lambda functions that allow us to freely redirect and rewrite requests and responses. At a high level, we write a lambda function that:

1. Looks at the user-agent of a request to determine whether or not we think this request is coming from a known social bot.

2. If it is, we rewrite the origin to point to Prerender

3. Prerender takes the request, renders it on their side and returns the HTML after our Javascript has finished rendering. If a previously cached version is available, Prerender will return that immediately.

4. CloudFront caches the response and will return it to any other requests that have a matching user-agent.

The code and configuration

Below is an example of how to achieve the above using CloudFront.

In addition to creating our function, we also need to configure it within AWS. For more on how to manually configure that check out Cloudonaut's original post. At Hatch, we had a few extra steps we needed to go through given that we use Serverless to manage our deployments to AWS. We needed to configure our serverless.yml file to forward header values in our CloudFront distribution. Our DefaultCacheBehaviour ended up looking like (notice the Headers in ForwadedValues):

‍

‍

We want to trigger our Lambda function using the CloudFront origin-request event (read more about CloudFront event types here). This may seem counter intuitive as these events are only triggered when the requested resource is not present in the CloudFront cache (as opposed to viewer-request which is triggered for every request). However, our method works because we configure CloudFront to maintain its cache based on the user-agent sent with the request.

Testing

Now that you've written your function and configured CloudFront, you should be able to see static HTML returned when using one of the user agents we specified. A quick and dirty way to test this is using cURL and the -A flag to set the user agent:

You can also use a tool like Metatags to preview a number of different social sites at once. Most social sites also provide their own debuggers (Facebook, Twitter, LinkedIn) which may be useful as well.

Bits and pieces

• Cloudonaut's original post is a little more explicit about their logic. They have checks for the Prerender user agent (obviously, we don't want to cause an infinite redirect) and for non-html files. This is captured in our code above, but it's a little less explicit.

• We use the i flag when creating the RegExp object so that we check case-insensitvely. Some agents use uppercase and others do not (e.g. Slack uses Slackbot whereas Twitter uses twitterbot). I figured it was easier just to make the check case-insensitive.

• /_escaped_fragment/ is a Google thing to handle hash fragments. In our case, it's not strictly necessary, but it may be for you. Read more about it here.