Speeding up Paperclip Tests by… a LOT

Here’s a quick little trick that I used to speed up my tests involving Paperclip by about 70%.

I posted it over on the Paperclip Google Group, which is a friendly and active place to hang out if you’re a Paperclip user.

Here’s an example using Test::Unit, which is still my favorite way to test🙂

require 'test_helper'
class PhotoTest < ActiveSupport::TestCase
  setup do
    Photo.any_instance.stubs(:save_attached_files).returns(true)
    Photo.any_instance.stubs(:delete_attached_files).returns(true)
    Paperclip::Attachment.any_instance.stubs(:post_process).returns(true)
  end

  # tests...
end

The really important bit is stubbing out the post_process method. That took my unit tests down from 51.77 to 15.14 seconds. That’s a HUGE win, especially if you consider slow tests to be a bug.

I’m not sure what kind of impact this has on test coverage, so you may want to consider not stubbing out the Paperclip internals in every case. I’ve got some separate “remote” tests that I run before deployments that make me feel warm and fuzzy enough. Let me know what you think about it. I’ve had really good results so far!

Published by

Trevor Turk

A chess-playing machine of the late 18th century, promoted as an automaton but later proved a hoax.

8 thoughts on “Speeding up Paperclip Tests by… a LOT”

  1. Thanks for posting that! I don't know if it's my version of paperclip, but I have to use :destroy_attached_files, instead.

    Photo.any_instance.stubs(:delete_attached_files).returns(true)

  2. Trevor, would you be able to include another posts with some basics on how to use Test::Unit to create functional tests for controllers associated with paperclip models?

  3. This technique has worked really well for me for a couple of months now, until tonight. I accumulated enough functional tests with stubbed out Paperclip methods that I hit too many open files and my tests started failing inside Paperclip:

    Errno::EMFILE: Too many open files – /var/folders/45/45kdBsmpGF0A-mb8hYDQxU+++TM/-Tmp-/stream.25027.213

    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tempfile.rb:55:in `initialize'

    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tempfile.rb:55:in `open'

    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tempfile.rb:55:in `initialize'

    vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip/iostream.rb:7:in `new'

    vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip/iostream.rb:7:in `to_tempfile'

    vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip/attachment.rb:79:in `assign'

    vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip.rb:232:in `image='

    The solution seems to be to NOT stub out save_attached_files and delete_attached_files, and it doesn't impact the run time of the tests, either.

  4. Steve, so you're saying it's best to just stub out the "post_process" method? I suppose that would at least save all the time for generating thumbnails…

  5. Based on my debugging, it appears so, but I'm also using very small files for these tests. Any additional overhead in save_attached_files is not impacting the test run time, and it is allowing Paperclip to clean up the temporary files it creates.

    A slow test is a problem, but failing due to a problem outside my code is worse.🙂

  6. The "Too many open files" problem is caused by stubbing out save_attached_files. The storage classes (Paperclip::Storage::S3 and Paperclip::Storage::Filesystem) are responsible for closing the temporary file in the flush_writes method, but that never gets called when you stub save_attached_files.

    Instead, replace flush_writes with something like this:

    def flush_writes

    @queued_for_write.each{|style, file| file.close}

    @queued_for_write = {}

    end

Comments are closed.