geek!daily

Cucumber, Webrat ... Who Names These Things?

I'm tinkering a bit with Cucumber and Webrat for my day job and am very excited by some of the prospects for our QA group's automation efforts. Along the way I'm finding that I have to explain how all the moving parts relate to each other, so I made this diagram:

Cucumber, based on RSpec, uses Webrat to drive Selenium. Seems to make it all make sense to the folks with the questions.

(and thanks, brynary + all the webrat contributors, aslakhellesoy + all the cucumber contributors, and dchelimsky + all the rspec contributors, and the whole selenium crew ... this is very, very cool stuff).

Google Chrome: The Next Big Thing?

A friend at work recommended I have a look at Google's explanation of Google Chrome as illustrated by Scott McCloud [via Google Blogoscoped where they were kind enough to scan the comic they received] and I couldn't wait. You see, Scott McCloud is the author of two of my favorite books, "Understanding Comics" and "Reinventing Comics" ... both must-reads if you're interested in comic history, the vocabulary of comics, or even just in thinking about how we communicate and interact with things on pages.

The Google Chrome strip [via Google Books] is an engaging 38 pages which lightly explains what they're trying to accomplish and how they've gone about it so far. Amusingly, it seems the comic went out a bit earlier than planned, but Google is taking things in stride, with grace and good humor, showing the rest of us how it should be done.

Some of the more interesting choices:

Webkit-based: Google Chrome is based on the Webkit browser engine, which is also used by Safari, Android, and a number of other mobile device browsers. Google's developers liked it because it's fast, memory-efficient, embeddable, easy to adopt, and generally keeps things simple.

The V8 Javascript Virtual Machine: written by "the V8 team in Denmark" specifically to address the shortcomings of previous JsVMs in a number of ways, including compilation to machine language and real garbage collection. I'm pretty excited about this, particularly that they've made V8 browser-independent.

One process per tab: They've implemented this as a protection from crashy apps as well as a way to clearly reveal what's chewing up resources in your browser. This concept also seems to be foundational to their anonytabs and anti-popup measures; it's definitely the basis of their sandbox security measures.

The New Tab Experience: When you open a new tab in Google Chrome, they assume you want to go somewhere rather than see your default page (or a blank page, which I've used for years to avoid unnecessary page loads). A nice UX touch that shows the depth to which they're thinking about this.

Gears Inside: It's no surprise that the Google Chrome browser builds in Google Gears, which has been aiming to bring the Web 2.0 experience even when offline for over a year now. I couldn't find any estimates of how widely installed Gears has become in the past fourteen months, but I'd guess it's low at present despite MySpace and Windows Mobile having jumped on board in the past six months.

There's lots of other good bits in the strip, like the anti-phishing measures page (which makes an analogy between malware and dog poop which couldn't have been made nearly so excellently in any other medium, imho) and a late appearance by Chris DiBona to sum up the open-sourcedness of it all.

I have some minor concerns about the extra resource consumption and possible need for interprocess communication (IPC) in their model. However, this is probably because my last significant experience with IPC dates back almost a decade to SGIs running IRIX. It was my experience then that IPC is hard to implement cleanly as well as the most frequent source of race conditions. It's likely times have changed since then. I hope.

The biggest disappointment? When the beta is officially released tomorrow, it'll be Windows only for now. The official blog post notes that they're working hard on versions for MacOS and Linux, too (and I suspect they would have liked those to be available on launch day), so when the Google Chrome project page lights up tomorrow, it won't be for the rest of us quite yet.

But I'll be eagerly watching, just the same. You don't see a new browser born every day.

Update: Paul Thurrott's WinSuperSite has screenshots purported to be Google Chrome [via Blogoscoped]

Cuzillion: 'cuz there's a zillion pages to check

Once in a while I get lucky. Most recently, this means that I was asked to write a blog post for LinkedIn about the tech talk Steve Souders gave for us last week: Even Faster Web Sites. Unfortunately, in the interests of brevity, I couldn't dive on something incidental to the talk which fascinated me: Cuzillion.

Cuzillion is one of those headslappers Steve seems to produce on a regular basis; it's a tool to model web pages and experiment with load order and methodology in order to better understand performance bottlenecks and find your way to a solution. He uses it to illustrate examples in his talks of late (as he did in the slides for his talk at LinkedIn), which highlights another feature: the ability to share a precise example among the team seeking a solution. It fascinates me because it's so straightforward and so obviously useful, yet you can twiddle it to great depth, willfully violating all the well-known best practices, and see clearly why that's not a good idea. It's like checkers: you get the basics instantly, and yet there's so much more there.

After playing with it for a while, it occurred to me that I very much want to say, "Cuzillion, go model the way this web page loads so I can play with it." When Steve announced Cuzillion, he mentioned that he intends to set up a Google Code repo for Cuzillion; when he gets around to that, it might be fun to spend a little time adding that feature.

Check it.

Test Helper: assert_not_valid

I was a little surprised that Rails offers assert_valid but not its implied converse, assert_not_valid ... so I wrote it:

  def assert_not_valid(record, message='expected not to be valid, but was valid')
    clean_backtrace do
      assert !record.valid?, message
    end
  end

Test Helper: assert_equal_multiline_text

I got tired of poring over the crappy output when comparing two large text blobs for equality fails; the more subtle the difference, the harder it is to spot it by eye. So I wrote this:

  def assert_equal_multiline_text(a, b, label = nil)
    return true if a == b
    label = label ? label + ', line' : 'Line'
    a_lines = a.split(/\n/)
    b_lines = b.split(/\n/)
    (0..([a_lines.length, b_lines.length].min - 1)).each do |i|
      assert_equal a_lines[i], b_lines[i], "#{label} #{i}:"
    end
  end

... where label is just a nice human-readable name for the things being compares. I call it like this:

assert_equal_multiline_text @expected.encoded, 
  Mailer.create_invoice(payment).encoded, 
  "Invoice mail"

My eyes are much happier.

Setting Constants While Testing

Let's suppose you've got behavior in your app which is dependent on how RAILS_ENV is set and — of course — you want to test that it's behaving properly. To do this, you need to change the value of RAILS_ENV just before you call that function (and change it back just after). This puzzled me for a while, but I finally figured it out:

class FooTest << Test::Unit::TestCase
  def test_doing_the_thing_with_the_stuff_in_production
    foo = foos(:the_one_to_test)
    orig_rails_env = RAILS_ENV
    suspend_warnings {Foo.const_set(:RAILS_ENV, 'production')}
    assert foo.do_the_thing(:with => the_stuff)
    suspend_warnings {Foo.const_set(:RAILS_ENV, orig_rails_env)}
  end
end

(const_set comes from Ruby's Module class, which also provides const_get, const_defined?, and const_missing. Nifty!)

That was all well and good for my unit tests, but what about functionals tests? That turned out to be a slight variation on the theme:

class FooControllerTest << Test::Unit::TestCase
  def test_getting_the_thing_with_the_stuff_in_production
    foo = foos(:the_one_to_test)
    orig_rails_env = RAILS_ENV
    suspend_warnings {FooController.const_set(:RAILS_ENV, 'production')}
    get :the_thing_with_the_stuff
    assert :success
    suspend_warnings {FooController.const_set(:RAILS_ENV, orig_rails_env)}
  end
end

You'd think that was enough, but noooo, I've even got a function in my ApplicationController which not only behaves differently per RAILS_ENV but also honors a constant (call it FOOISH) which may be set differently (or not set) in each environment. Holy crap! Dealing with that's a bit painful since I don't want to hardcode FOOISH's current per-environment settings in my tests. Then I tripped over a neat bit of code in config/boot.rb and mutated it into this test helper:

def get_var_from_rails_environment(varname, env = RAILS_ENV)
  global_env_file = File.join(RAILS_ROOT, 'config', 'environment.rb')
  specific_env_file = File.join(RAILS_ROOT, 'config', 'environments', "#{env}.rb")
  [specific_env_file, global_env_file].each do |env_file|
    IO.readlines(env_file).grep(/^\s*#{varname}\s*=\s*(['"]?)(.*)\1/)
    break if $2
  end
  $2
end

This lets me make it clean and easy:

def test_getting_the_thing_with_the_stuff_fooishly
  orig_fooish = ApplicationController.const_get(:FOOISH)
  orig_rails_env = ApplicationController.const_get(:RAILS_ENV)
  %w(development staging test production).each do |rails_env|
    fooish = get_var_from_rails_environment('FOOISH', rails_env)
    next unless fooish
    silence_warnings {
      ApplicationController.const_set(:FOOISH, fooish)
      ApplicationController.const_set(:RAILS_ENV, rails_env)
    }

    get :the_thing_with_the_stuff
      
    silence_warnings {
      ApplicationController.const_set(:FOOISH, orig_fooish)
      ApplicationController.const_set(:RAILS_ENV, orig_rails_env)
    }
  end
end

That's as elegant as I can make it. Not a bad day's work.

Update: Ryan Bates rightly advises caution in changing RAILS_ENV as you might trigger subtle side effects. Proceed with caution.

0-30% Test Coverage? That's Nothing.

I'm finding with some amusement that most cases where rcov says something is 30% covered or less, this means all the function definitions (e.g. def foo(bar)) are covered ... and nothing else.

Time to put all the excellent knowledge gained at Ruby Hoedown and RailsEdge to good use ...

Thoughts on Software Testing and Rails (a little)

Three good reasons to make sure your code has full, solid test coverage:

1. Tests are a transfer of confidence from you to the next developer down the line. It's a joy to walk into a codebase that's fully covered by solid tests. It means I can safely seek to understand only the part of system I need to alter rather than worry over side effects of my changes. It doesn't guarantee there won't be any side effects, but if there are, they're either (a) found immediately due to the tests, (b) new, or (c) harmless.
2. Tests are distributed ownership. The level of confidence I mentioned above means new developers can hit the ground running and start to take ownership of some portion of the code nearly immediately. In turn, earlier code owners can move on to the next interesting problem rather than taking a new job just to get their hands on a new challenge. Rapid onboarding, continued growth of mature team members -- did you hear that, you managers of technologists? It's in your best interest to demand your codebase is fully covered by solid tests.
3. Time spent testing saves time. Time is your most valuable asset. Nuff sed.

You knew all this, though. You can't have been a software engineer for any length of time without the test-early-test-often meme coming into your consciousness. What you did with it depends on the culture you were in at the time. For me, it was 1999 when I got my first SIGSOFT proceedings talking about this "coverage testing" thing which sounds really cool ... but the studio where I was working didn't much value testing and I was the juniormost engineer in a team of nearly 60 developers. They must know better than me, right? It took me a few years to get over that and start writing tests wherever I could. But I'm not bitter. Really.

One of the most incredibly great things about the Rails community is its passionate pursuit of testing. I don't think I've ever seen a development community which values testing so highly, discusses it so frequently, and even goes so far as to make testing cool.

It's cool to do the right thing? Excellent.

Rails and TDD/BDD

I'm going to limit this post to my notes on the excellent learnings from the day of many testing talks at Rails Edge Chicago and follow up with another post for my addle-pated musings thereupon.

First I'd like to observe that Rails benefits heavily by the fact that Railers care passionately about BDD, coverage testing, seeking complete coverage, and beautiful, elegant code. In the past month at various Rails and non-Rails conferences, I've heard it hammered over and over again in many ways:

• You've got to have 100% coverage; anything less is uncertainty.
• It's better to submit a test with no patch than a patch with no test.
• I pretty much don't accept patches without a test.

...and many more. This shared mindset results in a community of developers who not only bring an ethic of constant improvement but also who seek to make code ownership irrelevant. More on that in the Conclusions.

Watching Jim Weirich do BDD was a religious experience. His talk was titled, "Red, Green, Refactor" which is the concise expression of his process: make a failing test, make the test pass, then do it again. He just sat down and started making tests, then writing just enough code to satisfy the test. It was design on the fly, iteratively building up the model in small increments, testing, and again; part of my joy from this was to watch as an excellent programmer exposed his process, even to the point of incorporating feedback from the room. His rake task for testing tracks the number of times it's been called. If his talk is to be believed, he does it like this every day. And I believe.

Incidentally, Jim writes monster-long test names like "test_portfolio_with_one_stock_has_value_same_as_stock_value" to make his intent instantly readable. His point is that you're only going to type this name once, so don't let a desire for "manageable" method names get in the way of expressive, semantic naming.

And he made me realize how cool flexmock is: flex_mock_obj.should_receive(:quote).times(3).and_return(4,30,100)

Justin Gehtland and Stuart Halloway are awesome presenters and it's easy to see they enjoy working together. They're BDD practitioners and advocates of it and it was a pleasure to hear them talk.

David Chelimsky is the lead developer of RSpec, a very thoughtful person (as in "he thinks often and well", though I'm sure he's also kind =), and a vicious Werewolf waiting to happen. He did a half-hour speed talk about RSpec, which I'd only just heard about three weeks ago when talking to Matt Pelletier and again a couple of weeks ago at Ruby Hoedown. The general idea is you define your spec in a domain-specific language (DSL) which can then be executed to demonstrate that you meet spec. This was already on my list, but David's moved it up quite a lot. He left me with some research:

• RBehave: story level behavior testing
• assert_select: find CSS items in page and assert about them.
• CrossCheck: java tool that emulates popular browsers to test your Javascript "in browser"
• ZenTest/AutoTest: test like mad
• Test Notifications via Growl: combined with AutoTest, you can just watch things start working as you fix tests. Cool!

He also gave me reading: Domain Driven Design, by Eric Evans.

Mike Mangino's talk, "Testing for the Real World", made some excellent points:

1. Fast tests get run.
2. Slow tests don't.
3. The only tests that help are the ones that get run.

Actually, his talk seemed like a barrage of excellent points in retrospect; I ended up with a bunch of concise snippets in my notes which I'll try to reconstitute into full explanations here. Apologies to Mike if I screw it up and libel his abilities; you should assume any errors are in my understanding or ability to explain the concept.

Fixtures don't scale well for large quantities of data, so if you're testing against a lot of data you're best not using them. However, they're alway useful as templates for test objects; for example, if you're going to do a bunch of tests which all use the same data with only slight variations (testing each of many validators, &c) it can be handy to get the original object from a fixture and tweak it to suit your test.

Mocking can reveal the ugly spots when testing a controller. For example, if you find you need to mock a phonebook object to return an entry, an entry object to return a name, and a name object to return a last name ... you've got issues in the code. In fact, it's a good generalization that hard-to-test code very likely needs refactoring.

Meanwhile, it's much easier to generate tests for views and controllers if you do them separately. Particularly, Mike recommends mocking helpers in view tests and testing the helpers separately.

Mike finally said the right words to make the difference between stubbing and mocking stick for me (yeah, I may be dumb but I'm slow). If the return value is important to the bit you're testing, mock it; otherwise, stub it. He also mentioned that filters are hard to stub, but I didn't catch an explanation so I'll have to learn why on my own.

One of the best snippets: Tests should be informative first, DRY second. You want people who read your tests to instantly get it, not have to slog through various helper functions to figure out what's going on.

Oh, and more research: CruiseControl, a continuous build/integration tool based on the work of Martin Fowler and Matthew Foemmel. When I tried to have a look at the Ruby/Rails version is seemed the server was not responding.

Hey, Technorati! Get this: therailsedge railsedge railsedge2007