Chapters:

  1. ActiveJob
  2. Gem good_job
  3. Deploying to production

“Background jobs” or “background workers” are a very important concept in software development. You can create complete architecture based on tasks that will perform independently from the client side request-response cycle.

Background jobs are often used for processing heavy tasks that take too long to perform during a typical web request. Additionally, you can use them when the user is not expecting an output immediately.

Jobs can be scheduled to be processed:

  • As soon as there is resource capacity
  • Once, at a specific time
  • In X time from now
  • Recurringly, once every X period of time (CRON jobs)

Example usecases:

  • Process importing of multiple records
  • Autogenerate & send daily/weekly/monthly reports
  • Schedule sending a reminder email 30 minutes before an event
  • Processing an image or a video after upload

It is also normal for a job to trigger multiple jobs!

Real life scenario: In my app insta2blog.com, I use jobs to:

  1. Refresh Instagram bearer API tokens every 7 days (CRON)
  2. Iterate through all the users instagram posts and copy them into the app (this process can take a few minutes)

The Rails feature for processing jobs is called ActiveJob. It is an inbuilt feature, and you have a /jobs folder in any new Rails app by deafault.

1. Use ActiveJob

Generate new job:

rails g job MyThing

This job will just log something into the console:

# jobs/my_thing_job.rb
class MyThingJob < ApplicationJob
  queue_as :default

  def perform(text)
    puts "Console message: #{text}"
  end
end

Schedule a job for later:

MyThingJob.perform_later
MyThingJob.perform_later(text)
MyThingJob.set(wait: 1.week).perform_later(text)
MyThingJob.set(wait_until: Date.tomorrow.noon).perform_later(text)
MyThingJob.set(queue: :another_queue).perform_later(text)

Job processing adapters

To handle job processing, you will need an external adapter. The most popular one is Redis-based Sidekiq. You will encounter this solution in most enterprise Rails apps, at mostly any Rails job you have in your career.

However Postgresql adapters are enough for most usecases, and this way won’t need Redis as an extra dependency. A really good gem for this is good_job.

2. Use gem good_job

Add the gem:

bundle add good_job

Use it as the default adapter:

# config/application.rb
  class Application < Rails::Application
+    config.active_job.queue_adapter = :good_job
  end

Run the good_job worker in your console:

# Procfile.dev
web: bin/rails server -p 3000
css: bin/rails tailwindcss:watch
+ worker: bundle exec good_job start --max-threads=8

Different job execution modes:

# echo > config/initializers/good_job.rb
# config/initializers/good_job.rb
Rails.application.configure do
  # dev env: rails server, separate thread
  # config.good_job.execution_mode = :async

  # test env
  # config.good_job.execution_mode = :inline
  # config.good_job.inline_execution_respects_schedule = true

  # prod env: separate $ worker
  # config.good_job.execution_mode = :external
end

GoodJob GUI Dashboard

You can monitor all past and scheduled jobs in a web GUI. To do so, add the route:

# config/routes.rb
+  mount GoodJob::Engine, at: "good_job"

Voila, now when you visit http://localhost:3000/good_job/, you will have a dashboard:

gem good_job GUI

To restrict unauthorize users from accessing this dashboard in production, you migth want to require HTTP basic authentication:

# config/routes.rb
+  GoodJob::Engine.middleware.use(Rack::Auth::Basic) do |username, password|
+    ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.dig(:http_auth, :username), username) &&
+      ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.dig(:http_auth, :password), password)
+  end

With that, your credentials.yml file could look something like this:

# config/credentials.yml
http_auth:
  username: superails
  password: 123

Schedule automatic recurring jobs (CRON)

CRON - 5 * * * * * symbols that represent a recurring period. For example:

  • 2 times a month on Wednesday at 13:45
  • weekly on Sunday at 00:00

Cron examples

Schedule cron with good_job:

# config/initializers/good_job.rb

# show in logs
# STDOUT.sync = true
# $stdout.sync = true

Rails.application.configure do
  config.good_job.enable_cron = true
  config.good_job.cron = {
    every_minute: {
      cron: '* * * * *',
      class: "MyThingJob"
    }
    weekly_sunday: {
      cron: '0 0 * * 0',
      class: "MyThingJob"
    }
  }
end

Log errors

# config/initializers/good_job.rb

# exception_notification
+  config.good_job.on_thread_error = -> (exception) { ExceptionNotifier.notify_exception(exception) }
# honeybadger
+  config.good_job.on_thread_error = -> (exception) { Honeybadger.notify(exception) }
# sentry
+  config.good_job.on_thread_error = -> (exception) { Sentry.capture_exception(exception) }

4. Production

If you are using Digital Ocean App platform, inside your app Create Resource From Source Code with the same source repository as your main one, but the type should be Worker.

Run command should be:

bundle exec good_job start --max-threads=8

digitalocean-good-job-worker

Don’t forget to all all the same ENV VARS, like RAILS_MASTER_KEY, DATABASE_URL as you would for your normal Rails app. After deploying, it should start working!

That’s it! 🎉🥳🍾