geek!daily

... it is by will alone i set my mind in motion ...

Advanced YAML: Tricking Out Your database.yml

I've been cultishly carrying this snippet around for years:

login: &login
 adapter: mysql
 host: localhost
 username: myuser
 password: mypass
 encoding: utf8

For a couple of reasons around running DB migrations in differently configured environments, I wanted to factor this into bits common to all environments and bits particular to the config on the local machine. I came up with something like:

# Provide default local block
local: &local
 socket: /var/run/mysqld/mysqld.sock
 
common: &common
 adapter: mysql
 encoding: utf8
 reconnect: false
 pool: 5
 username: myuser
 password: mypass
 <<: *local

development:
 database: myproj_test
 <<: *common

This works swimmingly, but I was tired of ritually mimicking and extending the original snippet without actually understanding it. I decided to dig deeper into what was actually going on and found, unsurprisingly, that I stand on the shoulders of giants like Doug Alcorn, James Duncan Davidson, and Ben Bleything. To render proper credit, Ben's post from June 2006 builds on a missing post from James's blog, both of which were preceded by Doug's observations in March 2006, and all of which draw on tricks originally found in the Typo database config. None of them seems to have detailed what's going on with the &login/*login construct, so I thought I'd dive into that a bit for posterity.

YAML is nothing more than a human-readable way to serialize basic data structures like lists and associative arrays (aka hashes), which makes it ideal to represent the configuration values needed by Rails. It turns out that YAML provides anchors (&), references (*), and associative array merges (<<), all of which allow you to include by reference either by assignment or by merging a referenced array into another array. 

Here's an example of each:

# Generate a reference
mammal: &mammal_ref
 warm_blooded: true
 lays_eggs: false

# Define via reference assignment
beaver: *mammal_ref

# Define including a hash merge
otter:
 cute: true
 <<: *mammal_ref

# Define including a hash merge, overriding a value in the reference
platypus:
 <<: *mammal_ref
 lays_eggs: true

I found that the Wikipedia entry on YAML was a good, quick overview; the official YAML spec has all that and more, but it's not a quick read. Meanwhile, both Doug and Ben observed that bringing ERB into the mix let's you seriously customize things. I took a bit from both of them; here's my database.yml now:

# Provide default local block
local: &local
 username: myuser
 password: mypass
<% if File.exist? "/opt/local/var/run/mysql5/mysqld.sock" %>
 socket: /opt/local/var/run/mysql5/mysqld.sock
<% elsif File.exist? "/var/run/mysqld/mysqld.sock" %>
 socket: /var/run/mysqld/mysqld.sock
<% elsif File.exist? "/tmp/mysql.sock" %>
 socket: /tmp/mysql.sock
<% end %>
 
# Allow for local DB configuration
<%= File.read(File.join(File.dirname(__FILE__), 'dblocal.yml')) if File.exist?(File.join(File.dirname(__FILE__), 'dblocal.yml')) %>

common: &common
 adapter: mysql
 encoding: utf8
 reconnect: false
 pool: 5
 <<: *local

development:
 database: myproj_development
 <<: *common

test:
 database: myproj_test
 <<: *common

production:
 database: myproj_production
 <<: *common

Works a treat. Thanks, Doug, James, and Ben.

2010.08.13 in Ruby/Rails | Permalink | Comments (2) | TrackBack (0)

Got the "can't activate" Gem::Exception Blues? gem cleanup!

What to do when rubygems can't activate a version of a gem because it's already activated a different version of the same gem? My Google magic wasn't good enough to find this one quickly, so I'll happily point out Jesse Hu's nearly one-year-old post about the same problem, descended from a Ruby on Rails thread on ruby-forum.com where, unsurprisingly, it's Jeremy McAnally who knows that `gem cleanup` is the way to go.

Took nearly 45 minutes. Freed untold amounts of disk. Much happiness.

2009.06.14 in Ruby/Rails | Permalink | Comments (1) | TrackBack (0)

Technorati Tags: ruby, rubygems

Cucumber, Webrat ... Who Names These Things?

I'm tinkering a bit with Cucumber and Webrat for my day job and am very excited by some of the prospects for our QA group's automation efforts. Along the way I'm finding that I have to explain how all the moving parts relate to each other, so I made this diagram:

Cucumber Diagram

Cucumber, based on RSpec, uses Webrat to drive Selenium. Seems to make it all make sense to the folks with the questions.

(and thanks, brynary + all the webrat contributors, aslakhellesoy + all the cucumber contributors, and dchelimsky + all the rspec contributors, and the whole selenium crew ... this is very, very cool stuff).

2009.06.10 in Ruby/Rails, Testing | Permalink | Comments (0) | TrackBack (0)

Generating with Ruby OFF Rails

I'm spending a fair amount of my time working on non-Rails Ruby projects lately and I've been missing the fine code generation toys in Rails. So tonight, for the fun of it and because I was hankering to play with cucumber, I hacked up the minimal set of bits that gets you script/generate in your non-Rails project.

This is how we do it:


> mkdir my_non_rails_proj
> cd my_non_rails_proj
> mkdir script config
> touch config/environment.rb
> cat > config/boot.rb
RAILS_ROOT = "#{File.dirname(__FILE__)}/.." unless defined?(RAILS_ROOT)
RAILS_ENV = nil
require 'rubygems'
require 'initializer'

Rails.configuration = Rails::Configuration.new
^D
> cp -p /path/to/gems/rails-2.3.2/bin/generate script/.
> ./script/generate
Usage: ./script/generate generator [options] [args]
Rails Info:
-v, --version Show the Rails version number and quit.
-h, --help Show this help message and quit.
[...]
> ./script/generate cucumber
create features/step_definitions
create features/step_definitions/webrat_steps.rb
create features/support
create features/support/env.rb
create features/support/paths.rb
create lib/tasks
create lib/tasks/cucumber.rake
create script/cucumber
> touch Rakefile
> rake features
[... large lack of FAIL ...]
>

Voila! Not guaranteed to work for all Rails generators, but it bootstrapped rspec and cucumber just fine, which was the point of my exercise. Someday I'll make a generator:bootstrap rake task or gem something if I need to do it again.

2009.06.03 in Hacking, Ruby/Rails | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: rails, ruby

Re-finding my rubyligion ...

Life's been busy. Good, but busy. My gig at LinkedIn, two hours of commuting each day, plus our four person plus dog family certainly keeps me awash in things I should be doing. Thing is, ever since I stepped away from managing LinkedIn's rubyists, I haven't had much to keep me practicing my Ruby skills or tied to the Ruby community. That's made me a bit sad.

So, for the last few months, I've been finding spare-time, non-urgent Ruby projects on github to play with, forking them when I'm ready to make contributions. The one I'm currently twiddling most is a fork of Jon Maddox's IMDB library. He got the very basics in to deal with movies; I'm expanding it to handle TV series and episodes as well. In the process, I've added testing and some heavy refactoring, mostly to arrange the libraries in a more idiomatic fashion but also to swap out Hpricot for Nokogiri (sorry, why, but Nokogiri has better support for complex XPath and XSLT fun that simplifies some of the code dramatically).

I'm also digging into the guts of the Confluence4R and Jira4R libraries (and I am gratefully indebted to Ben Walding for setting up a github mirror of the Jira4R main git repo at Codehaus); for them, I'm mostly adding spec code to demonstrate what I think is true as I'm learning. Jira4R is driving me a bit nuts -- I'm having a love/hate relationship with its wsdl2ruby underpinnings. Slowly I learn.

But, then, that's the point, isn't it?


2009.05.03 in Hacking, Ruby/Rails | Permalink | Comments (0) | TrackBack (0)

Old & Rusty to Shiny & New, Part 1

I'm feeling particularly masochistic today, so I'm going to tackle a project I've been putting off for a while: moving a Rails 1.2.6 app in our subversion repository into github, then updating it to Rails 2.1.

The first part, getting from svn to github, should be relatively painless according to github's "Importing From SVN" guide. It is, as long as you have already created your github repo (I hadn't) and already have git-svn installed (I didn't). So here's the expanded tutorial

1. Create your github repo

2. Be sure you have git-svn installed. I didn't, so here's what I had to do:

sudo port uninstall git-core
sudo port install git-core +svn

I had a bit of trouble with the serf port breaking during build time:

--->  Building serf with target all
Error: Target org.macports.build returned: shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_www_serf/work/serf-0.2.0" && make all " returned error 2
Command output: /opt/local/share/apr-1/build/libtool --silent --mode=compile /usr/bin/gcc-4.0 -O2 -I/opt/local/include -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I. -I/opt/local/include/apr-1 -I/opt/local/include/apr-1  -c -o buckets/aggregate_buckets.lo buckets/aggregate_buckets.c && touch buckets/aggregate_buckets.lo
libtool: compile: unable to infer tagged configuration
libtool: compile: specify a tag with `--tag'
make: *** [buckets/aggregate_buckets.lo] Error 1

Error: The following dependencies failed to build: p5-svn-simple subversion-perlbindings serf p5-term-readkey
Error: Status 1 encountered during processing.

... so I got all medieval on it, building it manually, then finishing the port install:

> cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_www_serf/work/serf-0.2.0"
> head config.status
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by configure, which was
generated by GNU Autoconf 2.61.  Invocation command line was

  $ ./configure --prefix=/opt/local --with-apr=/opt/local --with-apr-util=/opt/local

## --------- ##
## Platform. ##
> sudo ./configure --prefix=/opt/local --with-apr=/opt/local --with-apr-util=/opt/local
[...generally successful configuration noises...]
> sudo make all
> [...generally successful build noises...]
> cd -
> sudo port install serf
[...generally successful install noises...]
> sudo port install git-core +svn
[...more generally successful install noises...]

3. Create an authors file

I made mine as authors.txt; it looks kinda like:

jim = Jim Meyer <jim@...>
joe = Joe DiMaggio<joe@...>

4. Run the commands from the github guide. My pseudo-values included below:

$ git-svn --authors-file authors.txt clone http://events-svn.example.com/ events_tmp
$ cd events_tmp
$ git remote add origin git@github.com:my_github_username/events.git
$ git push origin master

...é voila! It's done! It even (quite cleverly) recognized other authors and properly linked them. Very nice!

2008.08.07 in Hacking, Ruby/Rails | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: git-svn, github

LinkedIn Mobile is Live!

It's been a busy few months since I joined LinkedIn, but some of that work is finally out there for the world to see: LinkedIn Mobile is now in public beta. You can get more details from the LinkedIn Blog.

Today's going to be an exciting day. =]

2008.02.25 in Ruby/Rails, Social Networks | Permalink | Comments (1) | TrackBack (0)

Textile, RedCloth, and MonkeyPatching

Okay, I need to go through ~2700 articles, some large, some very large, and programmatically decide which of them use an interesting amount of Textile formatting. Don't ask why; it's not polite to stare at the unfortunate. Just smile and nod and think about how you'd puzzle this one out.

So I've got a bunch of content, some of which may be in Textile format; how can I tell without looking at every piece and making a judgement call? First, I decided to try comparing the text before and after RedCloth processing, but that didn't work since RedCloth does some whitespace cleanup (among other things) which mean that feeding it a string devoid of Textile markup doesn't guarantee you'll get the same string back. A small part of me is offended by this -- it breaks the principle of least astonishment -- but I'm enough of a situational ethicist to value the good in what it's doing for me and move on.

I started poking around in RedCloth 3.0.4, found the bits which were perpetually going to alter some of my strings (for the better!) and discovered something wonderful -- you can pass an ordered list of rules into RedCloth#to_html, allowing you to tweak it just the way you like it. Do you like using -- gasp! -- long dashes? The :glyphs_textile rule converts them to an HTML entity (&#8212; to be precise) along with a bunch of other conversions. And there are even handy shortcuts to specify all of the textile markup rules (:textile) and all of Markdown (:markdown). Yay! Flexibilty rewards to those who read the code closely!

But why, then, if I do something like:


RedCloth.new('foo -- bar -- baz').to_html(:glyphs_textile, :textile)

Do I get this:

foo - bar - baz

It turns out that the glyph and inline rules are run in the same method, inline, which looks like this:


    def inline( text ) 
        [/^inline_/, /^glyphs_/].each do |meth_re|
            @rules.each do |rule_name|
                method( rule_name ).call( text ) if rule_name.to_s.match( meth_re )
            end
        end
    end

See the problem? Because inline comes first in the ordered list of regular expressions, all inline rules come straight to the head of the line. It should look more like this:


    def inline( text ) 
        @rules.each do |rule_name|
            method( rule_name ).call( text ) if rule_name.to_s.match(/^(inline|glyphs)_/)
        end
    end

... and that's exactly what I monkey-patched into my app.

Meanwhile, judging from the Textile demo, Textile should handle my double-dash example above; this is also a bug in the Ruby implementation -- and all this time I blamed Textile ... sorry! I think I'll send in a patch as penance.

Things I've Learned

  • Making your code flexible is good for other developers.
  • Being good to other developers means they might send you patches when they find problems.
  • There's a hidden feature in Textile which allows you to disable Textile for a chunk of text: <notextile>-not struck-</notextile> -or- ==-also not struck-==
  • Monkey-patching is a nice way to get something fixed in your app quickly.
  • It's handy to be able to silence constant setting warnings when you have to monkey-patch a constant.
  • Even smart people make bugs (so what hope is there for me?)

2007.12.19 in Ruby/Rails | Permalink | Comments (1) | TrackBack (0)

Ruby Benchmark Shootout, Round 2

Back in February, Antonio Cangiano did a fairly impressive run at benchmarking the various Ruby interpreter implementations and comparing the results. He's just done it again and, just nine months later, the improvements are fairly dramatic.

I'd summarize here but frankly I hate trying to reduce the complexity of a benchmark suite to a pithy conclusion. Instead, I'll inline one of the several results tables here; this one presents the benchmark execution times in seconds:

Befor you draw any conclusions, go read the rest for yourself.

I hope he'll provide the raw numbers in some format I can pull into a spreadsheet and play with myself.

2007.12.05 in Ruby/Rails | Permalink | Comments (1) | TrackBack (0)

Me @ LinkedIn.com

Today is my first day as the manager of light engineering development at LinkedIn and I'm incredibly excited about it. Why's that, you might ask? Because LED is the group that's bringing agile development and Ruby on Rails to LinkedIn.

Who says Rails isn't ready for the enterprise? ;p

2007.11.26 in Business, Leadership, Management, Ruby/Rails, Social Networks | Permalink | Comments (1) | TrackBack (0)

Next »
My Photo

About

 Subscribe in a reader

AddThis Social Bookmark Button

Categories

  • Administrivia
  • Blogs
  • Books
  • Business
  • Computing
  • Data Portability
  • Economics
  • Electronics
  • Engineering
  • Environment
  • Facebook
  • Food and Drink
  • Fun!
  • Graphing.Social
  • Hacking
  • History
  • Identity
  • Leadership
  • Linux
  • MacOS X
  • Management
  • Metadata
  • Open Source
  • Organization
  • Parenting
  • People
  • Photography
  • Privacy
  • PublicSquare
  • RailsRumble
  • Reputation
  • Ruby/Rails
  • RubyConf 2007
  • Science
  • Social Networks
  • TagEverything
  • Technology
  • Testing
  • Thinking
  • Trust
  • UI
  • Web 2.0
  • Weblogs
  • Writing

Archives

  • August 2010
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • January 2009
  • September 2008
  • August 2008

Words on a Page

  • Carol Tavris: Mistakes Were Made (But Not by Me): Why We Justify Foolish Beliefs, Bad Decisions, and Hurtful Acts

    Carol Tavris: Mistakes Were Made (But Not by Me): Why We Justify Foolish Beliefs, Bad Decisions, and Hurtful Acts

  • Steven Gary Blank: The Four Steps to the Epiphany

    Steven Gary Blank: The Four Steps to the Epiphany

  • Chip Heath: Made to Stick: Why Some Ideas Survive and Others Die

    Chip Heath: Made to Stick: Why Some Ideas Survive and Others Die

  • Patrick M. Lencioni: Silos, Politics and Turf Wars : A Leadership Fable About Destroying the Barriers That Turn Colleagues Into Competitors

    Patrick M. Lencioni: Silos, Politics and Turf Wars : A Leadership Fable About Destroying the Barriers That Turn Colleagues Into Competitors

  • Marc Ian Barasch: Field Notes on the Compassionate Life : A Search for the Soul of Kindness

    Marc Ian Barasch: Field Notes on the Compassionate Life : A Search for the Soul of Kindness

Pages

  • If
  • The Tagline Graveyard