This is one of the final post leading up the the launch of the Build a Ruby Gem Ebook, which is now available for sale in 3 packages, including 14 chapters of code and over 2 hours of screencasts.

One of the simplest and most common application of background processing is sending emails outside of a web request. And while background processing is pretty common, most existing Ruby background processing libraries require an additional process to execute these jobs, resulting in increased infrastructure costs.

At the time, I was working on an application hosted on Heroku, and the cost of an additional dyno ($35/month) wasn’t justified. The background jobs did very little more than send emails out of band. To me, the traditional solutions seemed like overkill. And with this, the idea of sucker_punch was born…

Ruby Background Processing Libraries

Until I had heard of Sidekiq, delayed_job and Resque were the standard options for processing background jobs in Ruby. I’ve used both and believe each has a great use case.

In my experience, it’s easiest to start with delayed_job if you need background processing in your application. If and when the DB becomes your bottleneck, it’s time to move on. Redis is a dependency of both Resque and Sidekiq, which increases the complexity and infrastructure maintenance, however, with Heroku’s hosted Redis Add-ons, this dependency is much less of a pain.

My work on Sidekiq

I previously wrote about my open source contributions and how I got started. I had spent the previous several months contributing to Sidekiq and learning more about concurrency patterns in Ruby.

Sidekiq is multi-threaded, which is the reason a single Sidekiq process is more efficient than a library like Resque. Although, nothing in life is free - care must be taken to ensure your jobs are thread-safe. A helpful guide to writing thread-safe code can be found on the Sidekiq wiki.

Use Case

I was working on Defriend Notifier and needed to notify users via email when their Facebook friend list changed. Using Sidekiq or any of the alternatives mentioned above certainly would’ve worked. In fact, the application originally used Sidekiq and functioned perfectly. However, as time went on, I decided the extra cost of the worker wasn’t justified, especially given that the application didn’t produce any revenue outside of advertising (very little…).

At the time, the only background processing library that didn’t require an additional background process was girl_friday, also written by Mike Perham, author of Sidekiq. While this solution worked for awhile, I found the syntax to be slightly non-intuitive and it felt a little dirty to pass around a global variable within the application to manage a single job queue. I also experienced some memory leaks on Heroku as a result of switching to girl_friday, but to be fair, I didn’t spend much time determining the root cause. Disclaimer: It very likely could’ve been due to code that I wrote.

Enter Celluloid

Celluloid describes itself as an actor-based concurrent object framework for Ruby. Celluloid is the guts behind Sidekiq and the reason why the multi-threaded Sidekiq code is so readable. Celluloid abstracts away the details of concurrency so your code doesn’t have to worry about manually managing thread synchronization and object message queuing.

Sucker Punch is born

girl_friday was written several years ago — before Celluloid was created. Realizing the power of Celluloid, I figured there was an opportunity for a new library that behaved like girl_friday, but utilized Celluloid’s more reliable multi-threading capabilities. And because Celluloid handles so much of the complexity, the gem itself would be relatively simple — really a DSL around enqueuing jobs to a Celluloid Pool.

I posed the question to Mike Perham and he confirmed my suspicions.

Fortunately, Celluloid had all of the functionality needed for background queues already built-in. While the functionality was there, the usage syntax was awkward.

I spent the next few days creating a DSL around the Celluloid internals, thus making it feel more specific to background queues.

Here’s the result of a typical job class:

class LogJob
  include SuckerPunch::Job

  def perform(event)
    # do some other stuff to
    # record the event in the background
    puts "#{event} just happened"
  end
end

I deployed my sucker_punch-converted application code to production and gave it a week or so before I declared it a success. Once I realized sucker_punch was going to be a maintainable method for processing background jobs in a single web process, I added additional niceties like logging and queue configuration details that may be valuable for other use cases.

Have you tried sucker_punch yet? If so, I’d love to know how it went and what you used it for…