codahale.com٭blog

This is my old blog. My current writing is here: codahale.com

Rails Plugin: rails_rcov

Been paying attention to Mauricio Fernandez’s rcov, but have no idea how to easily integrate it into your Rails application? Well, you had no idea how to do that. In about 15 seconds, you’ll be all over the code-coverage bandwagon. Ready? GO!

First, Install Rcov

I shouldn’t need to say this. Go here, download, untar, and ruby setup.rb && sudo ruby setup.rb install. Go! Go! Go!

Second, Install rails_rcov

Using SVN?

./script/plugin install -x http://svn.codahale.com/rails_rcov

Still partying like it’s 1995?

./script/plugin install http://svn.codahale.com/rails_rcov

Third, Bask In The Joy Of Rake Magic

For each test:blah task you have for your Rails project, rails_rcov adds two more: test:blah:rcov and test:blah:clobber_rcov.

Running rake test:units:rcov, for example, will run your unit tests through rcov and write the code coverage reports to your_rails_app/coverage/units. Running test:units:clobber_rcov will erase the generated report for the unit tests.

Each rcov task can take two optional arguments: RCOV_PARAMS, whose argument is passed along to rcov, and SHOW_ONLY, which limits the files displayed in the report. Here are some examples:

# sort by coverage
rake test:units:rcov RCOV_PARAMS="--sort=coverage"

# show callsites and hide fully covered files
rake test:units:rcov RCOV_PARAMS="--callsites --only-uncovered"

# only show files which begin with "app/models"
rake test:units:rcov SHOW_ONLY="app/models"

Check the rcov documentation for more details.

Please note that rails_rcov has only been tested with a Bash shell, and any other environment could well explode in your face.

Fourth, Marvel At How You Ever Survived Without Mauricio’s Crazy Metaprogramming Robot Watching Your Back

Is that cool or what? All that red in the report is work you haven’t done yet, so get crackin’! Don’t stop shooting until everything’s green! Hup hup hup!

50 Responses to “Rails Plugin: rails_rcov”

  1. Tim Case Says:

    Dude this is amazing, I love it! Green bars are my heroin.

  2. Alex Wayne Says:

    This seems super swank. Trouble is I cant get rcov to install on WinXP SP2. Just says rcov is not a valid command and rake test:units:rcov crashes with an unspecified error.

    Although, rcov 0.5 will be installable via gems so maybe that will help.

  3. Alex Wayne Says:

    rcov 0.5 is not installed via gems and all seems well since I can run rcov directly on a single file with “rcov some_test.rb”.

    however the rake task crashes when I try to use it

    rake test:units:rcov –trace

    yields the following:

    rake aborted!
    You have a nil object when you didn't expect it!
    The error occured while evaluating nil.exitstatus
    (See full trace by running task with --trace)
    rake aborted!
    Command failed with status (1): [c:/ruby/bin/ruby ./vendor/plugins/rails_rc...]
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:722:in `sh'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:729:in `sh'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:812:in `sh'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:807:in `sh'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:747:in `ruby'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:812:in `ruby'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:807:in `ruby'
    ./vendor/plugins/rails_rcov/tasks/rails_rcov.rake:74:in `new_rcov_task'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:387:in `execute'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:387:in `execute'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:357:in `invoke'
    c:/ruby/lib/ruby/1.8/thread.rb:135:in `synchronize'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:350:in `invoke'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:1906:in `run'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake.rb:1906:in `run'
    c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/bin/rake:7
    c:/ruby/bin/rake.bat:25

    Not once in the callstack does it execute any part of my app so I don’t its my code that’s crashing it.

    Any ideas on how to fix this?

  4. Coda Says:

    Alex, this may be one of those Win32-specific problems. The problem seems to be around line 74, in which the Rake task runs the Ruby script which actually executes rcov. That said, the error doesn’t seem to be in my code; it seems to be a shell error of some sort which Rake chokes on, and since it runs like a charm on my Linux boxen I can only assume it’s something in the difference between Linux and Win32.

    I don’t have any Windows machines available to check this on, though, so unless you want to debug it, I’m not sure what I can do.

    One thing to try: on line 54, change it to this:

    run_code = File.expand_path(__FILE__)

    It may be that Rake/Win32 doesn’t handle relative file paths.

  5. Alex Wayne Says:

    Sadly that change made no difference. Although the following does work:

    cd test/unit
    rcov *

    So rcov seems to be working fine.

  6. Coda Says:

    Uninstall the plugin, then install it again.

    I wasn’t escaping paths with spaces in them, which probably caused your problem.

    Hopefully this will work.

  7. Alex Wayne Says:

    No luck. Although my path doesn’t actually contain any spaces at all. Heres the excerpt from my console, from the uninstall to the reinstall to the rake task.

    http://rafb.net/paste/results/dXFOCE57.html

  8. Coda Says:

    Hmm. Try running this and let me know what happens:

    rcov -o "C:/Projects/MagneticWorld/MagneticWorld/coverage/units" -T -x rubygems/*,rcov* --rails  -Ilib;test "c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake/rake_test_loader.rb" "test/unit/address_test.rb" "test/unit/blog_article_test.rb" "test/unit/cart_test.rb" "test/unit/comment_test.rb" "test/unit/content_test.rb" "test/unit/faq_section_test.rb" "test/unit/faq_test.rb" "test/unit/gift_certificate_test.rb" "test/unit/line_item_test.rb" "test/unit/mailer_test.rb" "test/unit/order_test.rb" "test/unit/person_test.rb" "test/unit/product_image_test.rb" "test/unit/product_test.rb" "test/unit/review_test.rb" "test/unit/sale_test.rb" "test/unit/tag_test.rb" "test/unit/ups_api_test.rb" "test/unit/user_test.rb" "test/unit/wishlist_product_test.rb"

    This is the command that Rake is choking on.

  9. Alex Wayne Says:

    Success!

    http://rafb.net/paste/results/Bp1E3k42.html

  10. Brian Says:

    ‘rake test:rcov’ as mentioned above doesn’t exist, and running ‘rake test:test:rcov’ seems to clobber the report for each type of test (unit/functional/integration) with the previous type, instead of putting each in its own directory. Is this an API change or a bug?

  11. Coda Says:

    rake test:rcov was a bit I forgot to excise from the README, Brian. Sorry about that. There’s currently no way to run all three types of tests through the same rcov process, since the units/functionals load a stubbed version of ActionController, and integration tests depend on the real deal. I’ll make those changes.

  12. Brian Says:

    Are you sure it can’t be done? It seems to work for me from the command-line with this:

    rcov -o coverage -T -x rubygems/*,rcov*,db/* –rails –callsites test/**/*.rb

    But I haven’t tested it except on one app, so maybe I just got lucky.

  13. Coda Says:

    If it can be done, I’d love to hear about it. The two apps I tested the original approach with (which looked a lot like what you posted) both had fairly extensive collections of tests, and both ate flaming death when all tests were run at the same time. It may depend on what your integration tests are actually doing, I don’t know.

  14. Nils Franzen Says:

    As Alex above, I couldn’t get rails_rcov to work correctly on WinXP. For me, the problem lies in how the arguments to -x are passed:

    By default, it looks something like this:

    -x rubygems/*,rcov*

    But if you change it by quoting the value, so it looks like this:

    -x “rubygems/*,rcov*”

    … it works without problems for me. Alex, you only need to update line 58 in the file vendor/plugins/rails_rcov/tasks/rails_rcov.rake

  15. Nils Franzen Says:

    … I was too quick in my response above, you’ll also need to change line 111. From:

    status = sh(”rcov #{RcovTestSettings.to_params} #{args}”, {}, &block)

    To:

    status = sh(”rcov.cmd #{RcovTestSettings.to_params} #{args}”, {}, &block)

  16. Alex Wayne Says:

    Awesome, thanks for the tip Nils. This should be rolled into the code ASAP. Although it should probably be wrapped in some sort of “if win32″ block, especially for the second change.

  17. Coda Says:

    Okey doke–the Rubygems exclusion is quoted now, and on win32 it calls rcov.cmd. Keep ‘em coming! ;-)

  18. Alex Wayne Says:

    SHOW_ONLY option does not appear to work. The output is unchanged and still shows all files including helpers and models.

    This command:

    rake test:functionals:rcov SHOW_ONLY=”app/controllers”

    generates rcov command:

    rcov.cmd -o “C:/Projects/MagneticWorld/MagneticWorld/coverage/functionals” -T -x “rubygems/*,rcov*” –rails -x ^\(\?\!\app/controllers\) -Ilib;test “c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake/rake_test_loader.rb” “test/functional/account_controller_test.rb” “test/functional/blog_controller_test.rb” “test/functional/faq_controller_test.rb” “test/functional/home_controller_test.rb” “test/functional/person_controller_test.rb” “test/functional/products_controller_test.rb” “test/functional/store_controller_test.rb” “test/functional/admin/admin_controller_test.rb” “test/functional/admin/blog_controller_test.rb” “test/functional/admin/content_controller_test.rb” “test/functional/admin/homepage_controller_test.rb” “test/functional/admin/home_controller_test.rb” “test/functional/admin/inventory_controller_test.rb” “test/functional/admin/orders_controller_test.rb” “test/functional/admin/products_controller_test.rb” “test/functional/admin/reviews_controller_test.rb” “test/functional/admin/sales_controller_test.rb” “test/functional/admin/tags_controller_test.rb”

  19. Coda Says:

    Alex: it works like a charm in a bash shell. The key part there is -x ^\(\?\!\app/controllers\). If you can figure out how to escape ^(?!app/controllers) for the Windows shell, I’ll add it in. I don’t have access to a Windows machine, so I can’t do it myself.

  20. Alex Wayne Says:

    calling the -x parameter twice doesn’t cause a problem?

    (…snipped..) -x “rubygems/*,rcov*” –rails -x ^\(\?\!\app/controllers\)

  21. Coda Says:

    No, it doesn’t, unless Mauricio is shipping a totally different version for Win32, which I doubt. SHOW_ONLY works fine for me on GNU/Linux/bash, so the problem must be in the way I’m escaping the exclusion regular expression.

  22. Alex Wayne Says:

    line 58 should be:

    params

  23. Eric Nielsen Says:

    I’m trying to use your plug-in in conjunction with ZenTest/Test::Rails. Test::Rails splits functional tests to controller and view tests (among other things). Rake –tasks shows

    rake test:controllers
    rake test:functionals
    rake test:functionals:clobber_rcov
    rake test:functionals:rcov

    So it look like your plug-in is correctly doing its magic on the built in rails tasks (and rake test:units:rcov SHOW_ONLY=”app/model” works beautifully). However it looks like its not detecting the added tasks from Test::Rails.

    The Test::Rails instructions for hooking up its tasks is simply to add
    “require test/rails/rake_index’ to the Rakefile. I suppose that your method for searching for .rake’s doesn’t actually follow links from within the found files. If I copy the rake_tasks.rb from gem to my lib/tasks and rename to a .rake, everything works. So the two add-ons aren’t intrinsically incompatible.

    So from a best practices view point, what’s the preferred way to make sure your plugin finds new tests tasks? (Ie I don’t think my hackish copy/rename is the “right” way, but not sure what’s more correct)

  24. Coda Says:

    Eric, there isn’t a good way to do that, mainly since I don’t think Rake is really built to do what I’m doing. The Test::Rails rake file appears to be loading after rails_rcov, and so rails_rcov is blissfully unaware of its presence. If you can get Test::Rails to load before rails_rcov, you’re good to go.

    rails_rcov is not complicated in the way it looks for new tasks. It doesn’t parse any files, since Rake already does that. Instead it iterates through the tasks Rake has already loaded, and looks for tasks which start out with “test”.

    I’d suggest perhaps making a new rake task in your lib directory, and requiring Test::Rails from there. That would probably work.

  25. Eric Nielsen Says:

    That worked, and is substantially cleaner. Thanks.

    I don’t suppose you’ve put any thought into how one could create a rake task to run each of rake test:foos:rcov SHOW_ONLY=’app/foos’ and extract information for each to generate summary index for each test suites? ;)

  26. Jesper Rønn-Jensen (justaddwater.dk) Says:

    Hey this is great. I just ran this on a commercial project I’m working on. Thanks for making this plugin!

  27. Alex Wayne Says:

    Although your plugin rocks my socks, it recently became totally incompatible with edge rails, at least on linux. It crashes any rake task mysteriously.

    More info here:
    http://www.ruby-forum.com/topic/76723#120940

    Just an FYI.

  28. Coda Says:

    Alex–I’ve gotten reports of that, but it’s nothing I can reproduce at all, and no one’s been able to pin it down for me. I’m open to suggestions, but I’ve got several Edge projects, none of which exhibit this behavior.

  29. JGeiger Says:

    I’m using the plugin with RadRails 0.7 on windows, and it works and creates the coverage directories and the coverage percentages, but when I look at the code itself, it doesn’t highlite the missed coverage. Only the top line (Class definition) is the green for “Code reported as executed by Ruby looks like this…”

    Seems odd that everything but that part works.

  30. JGeiger Says:

    Well, blame eclipse and it’s built-in web browser.

    It seems that it can render some of the html code, but not all of it properly. I opened the file in firefox, and it’s showing the un-tested code.

  31. How much is too much with tests? Says:

    [...] I’ve been having fun with tests lately now that I’ve overcome my initial allergy (more like trepidation actually). So naturally my curiosity leads me to new toys like the rails rcov plugin based on (surprise!) rcov that gives you a rough estimate of your code coverage. The you start thinking about testing your helpers. [...]

  32. Don Park Says:

    oh this is really really great. thank you for the plugin. its just what I was looking for.

  33. Labrat Says:

    Regarding the rcov plugin breaking edge rails, I’ve confirmed and reproduced this.

    The problem is not the rcov_plugin itself but having the plugin without the rcov gem installed (this happens when you don’t have the same gems in your deployment servers or whatever other environment).

  34. Goynang Says:

    Good stuff.

    Question: how can you turn off the filter that stops it reporting on code within vendor? I’m trying to test some plugins within an app (engines to be exact) and can’t get it to report on them. Would be extra cool if I could somehow get it to ignore all of vendor/plugins except for *_engines folders. I’ve tried passing in extra params for –include-file and –exclude-only but to no avail.

    Is this possible?

  35. Casper Says:

    Also want to know if you can configure rcov to test engines and other plugins. I have become dependent on rcov and my current project involves alot of engines and plugins. Please let me know if there is a solution thanks.

  36. Ambethia.com » Blog Archive » More Rails Testing SQLite Woes Says:

    [...] Choose the most recent “ruby” version, in this case I chose “2”. And that’s it you should be good to go. Some more about using RCOV with Rails. [...]

  37. Pelargir - Musings on software and life from Matthew Bass. » Introduction to rcov, code coverage for Ruby Says:

    [...] Introduction to rcov, code coverage for Ruby By Matthew Bass Do you routinely test your code? Are you using rcov to run code coverage reports for your tests? If you aren’t, you should be. rcov is a code coverage tool for Ruby that is available as a Gem or a Rails plugin. It can generate text or HTML reports outlining what percentage of your code base is being exercised by your tests. The HTML reports even allow you to drill down and see exactly which lines in your code are being touched. If you’ve ever used a tool like Cobertura for Java, you’ll know exactly what to expect with rcov. [...]

  38. Jeroen van Doorn » Blog Archive » Rake fails on tasks when gems are missing Says:

    [...] The exact same repository checkout works perfectly fine on my development machine. After much head scratching and hair pulling, I found out by accident what caused the problem: I had the rails-rcov plugin installed (which I highly recommend!), which requires the gem rcov to be installed, but Rake doesn’t error out when it can’t load the gem. It doesn’t even squeek at you to let you know something’s wrong. It simply stops processing tasks and continues on its merry way, trying to execute the task which it didn’t load. [...]

  39. MatthewRudy Says:

    I was just wondering if there’s a way to get coverage data on Views?
    Is that possible?

    It seems like it would be really useful
    (although presumably it would make the report much longer.

    Mj

  40. Ravi Says:

    Hi,

    I installed ruby and gem install rcov also.
    Rcov is working properly . But Now i installed RCOV plugin also.
    How can i use it??? Please describe me. Awaiting Reply

  41. Customize Your Rake Files Says:

    [...] a method, you forgot to test a conditional statement. Because I find this handy, I incorporate the rcov plugin in my rails [...]

  42. Doktah Hahpah Says:

    Thanks for the quick commands, Man — up and running and covering in minutes!!!

    H

  43. James McCarthy Says:

    Damn that’s a time saver… damn I have a lot to do now. :)

  44. Keegan Quinn Says:

    Hello Coda. Lovely little plugin you’ve got here. Powerful minimalism at it’s finest.

    I have a couple of patches to rails_rcov.rake. The first adds a working test:rcov task, which invokes rcov separately for each of the other test:*:rcov tasks. This avoids the ActionController stub issues mentioned previously.

    The second patch is less interesting, focusing on making the task consistent with practices used in Rake and Rails core tasks. Both are now posted on my blog. I’d appreciate if you (or anyone) would review them and maybe consider applying them in the plugin.

    Thanks very much for a fine piece of code!

  45. Frederico Says:

    what an awesome plugin it is.
    Congratulations!

  46. Todd Says:

    hi Coda,

    great plugin! Seeing code coverage and green bars actually makes me want to write tests — I need that kind of motivation.

    However, I’ve noticed a discrepancy in the code coverage results when I run rake test:test:rcov and when I run rake test:units:rcov. The former claims that I’m not covering code that i know I am; the latter reports coverage that I agree with (and can verify).

    I’ve patched my rakefile with Keegan’s patch (comment #44, above), which adds a rake test:rcov task. This task runs each of the units, functionals, etc tasks in sequence, and it generates the correct output.

    This may be what you were referring to with the ActionController issue earlier, but I thought I’d mention it anyway just in case.

    thanks again for a nice piece of work.

  47. Paul Hepworth Says:

    This plugin no longer works with ruby 1.8.6 due to bugs in rexml.

    Also, if you fix the bugs in rexml, rcov decides to output empty html files. :(

  48. jhernandez Says:

    Argg, I can’t access to comment #44 patches. Why are not they integrated??

  49. Denis Says:

    Anybody have the patch mentioned by Keegan in post 44 above? His web site seems to be gone…

  50. frank Says:

    To make the rcov-rake-tasks run in a Windows-environment I had to change ‘rcov.cmd’ to rcov.bat’ in file rails_rcov.rake /line 134

    cmd = (is_windows? ? ‘rcov.bat’ : ‘rcov’) << ” #{RcovTestSettings.to_params} #{args}”

    Obviously rcov has renamed its Windows batch file… I am using gem installed version 0.8.1.2.0 of rcov

    Good luck everyone!