Comments on: Speeding up Paperclip Tests by… a LOT

By: Jamie Flournoy

Jamie Flournoy — Tue, 20 Apr 2010 12:21:58 +0000

The "Too many open files" problem is caused by stubbing out save_attached_files. The storage classes (Paperclip::Storage::S3 and Paperclip::Storage::Filesystem) are responsible for closing the temporary file in the flush_writes method, but that never gets called when you stub save_attached_files.

Instead, replace flush_writes with something like this:

def flush_writes

@queued_for_write.each{|style, file| file.close}

@queued_for_write = {}

end

By: Trevor

Trevor — Fri, 06 Nov 2009 15:21:14 +0000

Cool – thanks for the feedback!

By: Steve Madsen

Steve Madsen — Fri, 06 Nov 2009 09:32:11 +0000

Based on my debugging, it appears so, but I'm also using very small files for these tests. Any additional overhead in save_attached_files is not impacting the test run time, and it is allowing Paperclip to clean up the temporary files it creates.

A slow test is a problem, but failing due to a problem outside my code is worse.

By: Trevor

Trevor — Fri, 06 Nov 2009 00:12:19 +0000

Steve, so you're saying it's best to just stub out the "post_process" method? I suppose that would at least save all the time for generating thumbnails…

By: Steve Madsen

Steve Madsen — Thu, 05 Nov 2009 23:37:31 +0000

This technique has worked really well for me for a couple of months now, until tonight. I accumulated enough functional tests with stubbed out Paperclip methods that I hit too many open files and my tests started failing inside Paperclip:

Errno::EMFILE: Too many open files – /var/folders/45/45kdBsmpGF0A-mb8hYDQxU+++TM/-Tmp-/stream.25027.213

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tempfile.rb:55:in `initialize'

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tempfile.rb:55:in `open'

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tempfile.rb:55:in `initialize'

vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip/iostream.rb:7:in `new'

vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip/iostream.rb:7:in `to_tempfile'

vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip/attachment.rb:79:in `assign'

vendor/gems/thoughtbot-paperclip-2.3.1/lib/paperclip.rb:232:in `image='

The solution seems to be to NOT stub out save_attached_files and delete_attached_files, and it doesn't impact the run time of the tests, either.

By: Xavius

Xavius — Mon, 24 Aug 2009 10:37:41 +0000

Trevor, would you be able to include another posts with some basics on how to use Test::Unit to create functional tests for controllers associated with paperclip models?

By: Rodney

Rodney — Thu, 07 May 2009 17:16:25 +0000

Thanks for posting that! I don't know if it's my version of paperclip, but I have to use :destroy_attached_files, instead.

Photo.any_instance.stubs(:delete_attached_files).returns(true)

By: Dev Blog AF83 » Blog Archive » Veille technologique : Navigateurs web, Subversion, Javascript, Frameworks, Tests unitaires, Méthodes, Logiciel Libre

Mon, 16 Mar 2009 17:43:48 +0000

[...] http://almosteffortless.com/2009/03/12/speeding-up-paperclip-tests-by-a-lot/ : une technique pour accélérer les tests unitaires en présence de paperclip [...]