Using PhantomJS to Capture Analytics for a Rails Email Template

Every Sunday Bark sends parents a weekly recap of their children’s activity online. The first iteration was pretty basic, things like “Your children sent X number of messages this past week” and “You have 10 messages to review”. But we wanted to go deeper…

Using PhantomJS, we were able to take screenshots of a modified version of the application’s child analytics page and include the image in the email sent to the parent. The email now contains everything the parent can see from the application, all without leaving their inbox.

The Problem

If you’ve every attempted to style an HTML email with anything more than text, you’re sadly familiar with its limitations. Tables and other elements from the 90’s are the only tools we have to maintain cross-platform compatibility. One of those tools, the subject of this post, is images.

Our weekly recap email contained a line chart illustrating the number of messages the child exchanged during the past week. While this was somewhat helpful to parents, it didn’t tell the full story.

"First version of the Bark weekly recap email"

While this email does include a graph, it’s the result of calling out to a service that rendered the graph, stored it, and returned the URL to include as an image. While this service worked well for simple illustrations, it didn’t provide us the flexibility we had with modern web tools and visualizations. Aside from that, the styling of the charts is limited.

Elsewhere on Bark, we had already built the full story through other lists and illustrations.

"Bark analytics with interactions"

"Bark analytics with activities"

"Bark analytics over time"

Recreating the same lists and charts just for the email felt like a duplication nightmare and vulnerable to becoming stale. We wouldn’t be able to use the same rendering because most of the charts rendered SVGs, which aren’t compatible with most email clients. Additionally, there were a handful of CSS styles needed for the page that while possible to include in the email, felt excessive.

Stepping back from the problem, we realized we wanted the majority of the analytics page, just embedded in the email. Was there a way to do that without rewriting it for email clients?

The Solution

We could take a screenshot of the analytics page and embed it as an image in the recap email.

wkhtmltoimage

Our first attempt was using wkhtmltoimage and the IMGKit ruby gem. Aside from the headaches of installing a working OSX version of wkhtmltoimage due to a regression, getting a working configuration was non-trivial.

wkhtmltoimage doesn’t parse CSS or JavaScript, so those would have to be explicitly included. Since Bark uses the asset pipeline, we’d have to get the latest version of the compiled assets both on development and production. This proved to be extremely difficult under the default configuration given how each group is compiled. We use Nginx to serve our assets in the production and it felt weird to have a configuration we would hope worked when we pushed to production.

After spending almost a full day trying to get the right combination of settings, we gave up. There had to be a better way…

Saas FTW

Frankly, our next step was to look for a Saas service that provided this functionality. Certainly I should be able to send a URL to an API, and they’d return an image, perhaps with some configuration options for size and response. To our surprise, there were none (based on a 15 minute internet search. If you know of one, we’d love to hear about it). There were plenty of services focused on rendering PDFs geared towards invoices and other documents one would want to email customers.

PhantomJS

We were reminded of Capybara’s ability to capture screenshots on failed test runs. After poking around this functionality, phantomjs emerged as a potential solution.

If we installed phantomjs on to the server and ran a command line script to capture the screenshots prior to sending the email, we could inline include those images in the email.

Installation of phantomjs was simplified using the phantomjs-gem ruby gem, which installs the appropriate phantomjs binary for the operating system and provides an API (#run) to execute commands.

Script the Screenshot

Using a screenshot example from the PhantomJS github repo, we put together a script (vendor/assets/javascripts/phantom-screenshot.js) to capture the analytics page:

#!/bin/sh

var page   = require('webpage').create();
var system = require('system');
page.viewportSize = { width: 550, height: 600 };
page.zoomFactor = 0.85;

page.onError = function(msg, trace) {
  var msgStack = ['ERROR: ' + msg];
  if (trace && trace.length) {
    msgStack.push('TRACE:');
    trace.forEach(function(t) {
      msgStack.push(' -> ' + t.file + ': ' + t.line + (t.function ? ' (in function "' + t.function +'")' : ''));
    });
  }

  console.error(msgStack.join('\n'));
};

page.open(system.args[1], function(status) {
  if (status !== 'success') {
    console.log('Unable to load the address!');
    phantom.exit(1);
  } else {
    window.setTimeout(function () {
      page.render(system.args[2]);
      phantom.exit();
    }, 2000);
  }
});

Note: a variety of the settings (viewPortSize, zoomFactor, and timeout) were found using trial and error for our particular situation.

We use Sidekiq to enqueue the thousands of recap emails sent to parents each week. Because this approach relies on using our existing website as the source data for the screenshot, we have to be conscious of spreading the job processing over a certain period of time, so we don’t overload the application for regular users.

Create the Screenshot

With this script in hand, now we can use the following class to create the image for each child:

class RecapAnalytics
  ScreenshotError = Class.new(StandardError)

  def initialize(analytics_url:)
    @analytics_url = analytics_url
  end

  def file_path
    unless create_screenshot
      raise ScreenshotError.new("Unable to complete analytics screenshot")
    end

    temp_file_path
  end

  def create_screenshot
    Phantomjs.run screenshot_script, analytics_url, temp_file_path
  end

  private

  attr_reader :analytics_url

  def screenshot_script
    Rails.root.join('vendor', 'assets', 'javascripts', 'phantom-screenshot.js').to_s
  end

  def temp_file_path
    @temp_file_path ||= begin
      file = Tempfile.new("child-analytics")
      file.path + ".png"
    end
  end
end

For each child, we’ll provide the URL to the child’s analytics page and run the following file_path method to return the path of the screenshot:

RecapAnalytics.new(analytics_url: "https://www.bark.us/children/XXX/analytics").file_path

Adding as an Inline Email Attachment

With an image for each child, we can iterate through each child and inline include the image in the mailer:

file_path = RecapAnalytics.new(analytics_url: "https://www.bark.us/children/XXX/analytics").file_path
attachments.inline["#{child.first_name}.png"] = File.read(file_path)

Then in the email template, we can include the following to render the image:

 <%= link_to image_tag(attachments["#{child.first_name}.png"].url), child_url(child) %>

"Bark weekly recap email with interactions"

"Bark weekly recap email with activities"

"Bark weekly recap email over time"

Conclusion

PhantomJS proved to be the simplest solution for the job. With a small script and no further configuration, we were able to lean on the analytics page we’d already built to improve the Bark recap emails.

Parents will now have more visibility in to their child’s online activity without leaving their inbox.