WE DONE LAUNCHED

Guess who’s live?

Take control of your money, find the best values in your area, and reach your financial goals by joining the Wesabe community.

Pound vs. Pen: Because You Need A Load-Balancing Proxy

Update 11/9/06: I just realized that pen handles servers in the order they’re listed on the command line. This means my code can cause all the Mongrel processes on a single server to be used while the other server is idle. I’ve updated the code to interweave the various hosts together, so that the load is even distributed, even under less than near-maximum loads. So, download again!

(Holy crap! I haven’t posted since August?!? Time to throw down on some technical issues, lest I be consigned to the Dustbin Of History! Lemme get this mad knowledge out of the way, and I’ll let you know what I’ve been up to. Meet you at the bottom!)

A Mongrel cluster walks into a bar and orders a bowl of water…

So. You’ve looking to set up Mongrel cluster, but the idea of compiling Apache 2.2 for each of your machines or moving over to the experimental branch doesn’t sit well with the part of you that doesn’t like hanging out in pickup truck beds on the freeway. Perfectly reasonable. You’ve decided to stick with Apache 2.0 because the nice folks at Debian manage the packages. (Don’t have Apache configured yet? Check this out!)

Now you’ve got your Apache config all lined up (knock on wood), and it’s time to make the decision: what the hell is Apache going to hand requests off to? A single Mongrel process? No. Hah. No. This is a business, not a blog in a bottle. Lighty and FastCGI? Oh hells no. Zed isn’t answering the same questions over and over again just so’s we can punk out and not use his amazingly cool HTTP server. mod_rewrite and some crazy-pantsed text file rotation? Not so much. No, the choice seems to come down to either Pound or Pen. Which to use? Oh, which to use?

Pound: Where stray Mongrels go to die

Pound is nice. Pound is great. Pound smells like fresh popcorn. Pound is not what you should use, and I’ll try to be brief about it. Pound replaces the X_FORWARDED_FOR header on every request, which for its usual use (an HTTP front-end) is great–it means the server cluster can still tell where the requests are coming from. But if it’s behind an Apache configuration, it will change the headers on each request to show that each request is coming from Apache–127.0.0.1. And Rails, even in production mode, will show a debug screen to local requests. This combination means that Rails will treat all requests as local, even if they originated with Evils McHackerson von Hideypants III. Awwwwkward.

Also? This means that your exception notification code will never fire, so not only are you handing out debug information to folks who can cause unhandled exceptions, but unless you’re camping out on your production log, you’ll have no idea that your app is bombing.

Worst of all, there’s no way to disable this functionality short of hacking the source and recompiling. Pound, regardless of what it needs to do, will always change the X_FORWARDED_FOR, which makes it pretty crap at what we need it to do.

Bottom line: Apache + Pound + Mongrel = NO BAD BAD NO NAUGHTY BAD

So what’s left?

Pen: Mongrels don’t have thumbs

Well, there’s Pen. It’s also a load-balancing proxy, but handily it lets us choose whether or not the X_FORWARDED_FOR header gets changed. It’s about as fast as Pound, and about as stable. Its SSL support is still experimental, but that’s OK, since we’re proxying between local services, not serving as an external front-end. (If we were, we’d probably use Pound.)

The only downside is that it doesn’t come handily packaged to run as a service. That’s ok, folks, because we’re programmers. Right?

pen-service: Life’s too short to start services manually

Here you go: pen-service_0.2.tar.gz

Installation

server:~$ wget http://blog.codahale.com/wp-content/pen-service_0.2.tar.gz
server:~$ tar -xzvf pen-service_0.2.tar.gz
server:~$ cd pen-service
server:~$ sudo mkdir /etc/pen
server:~$ sudo mv pen.conf /etc/pen
server:~$ sudo mv pen-service /etc/init.d/pen
server:~$ sudo chmod +x /etc/init.d/pen
server:~$ sudo update-rc.d pen defaults
server:~$ sudo vi /etc/pen/pen.conf

Configuration

A simple three-Mongrel cluster, starting at port 8000 and ending on port 8002. It listens on port 8080, doesn’t keep track of which clients go with which server, and has a PID file in /var/run. Each Mongrel process can take a maximum of one connection at a time, since Rails is not multi-threaded. You can bump this up, but Mongrel will just queue the requests, defeating the purpose of load-balancing.

/etc/pen/pen.conf

hosts:
- 127.0.0.1:
    ports: 8000..8002
    max_connections: 1
port: 8080
disable_tracking: true
pid_file: /var/run/pen.pid

Full explanation:

# This is YAML.
#
# The backend hosts to be proxied to. Ports can be a single port (8000) or a
# range (8000..8002).
hosts:
- 127.0.0.1:
    ports: 8000..8002
    max_connections: 1

# Used to send Pen commands.
# Default: none
control_port: 8081

# Attach X_FORWARDED_FOR headers. (Don't.)
# Default: false
proxy_headers: true

# Use poll() instead of select().
# Default: false
use_poll: tue

# Dump any debug data in ASCII format.
# Default: false
debug_ascii: true

# Put servers that don't respond on a blacklist for 5 seconds.
# Default: 0
blacklist_time: 5

# Maximum number of HTTP clients.
# Default: 2048
max_clients: 1024

# Log debug information.
# Default: false
debug: false

# If all servers are down, direct to this host.
emergency_host: backup.railsapp.com:80

# Hash the IP before assigning it to a server.
# Default: false
use_ip_hash: true

# chroot Pen to this directory.
# Default: none
chroot: /var/chroot/pen/

# Log Pen info to this file.
# Default: none
log_file: /var/log/pen.log

# Use blocking sockets. (Don't.)
# Default: false
disable_asynchronous_sockets: false

# Use simple round-robin assignments instead of sticky sessions.
# Default: false
disable_tracking: true

# Don't load balance. (Don't?)
# Default: false
disable_failover: false

# Connection timeout, in seconds.
# Default: 5
timeout: 10

# Run Pen as this user.
# Default: none (whoever runs pen-service, usually root.)
user: www-data

# Log Pen's pid to this file.
# Default: /var/run/pen.pid
pid_file: /var/run/railsapp/cluster1.pid

# Maximum number of simultaneous connections.
# Default: 256
max_connections: 500

# Log HTML-formatted usage statistics to this file.
# Default: none
stats_file: /var/www/stats/pen-stats.html

Ok, more about me

I’ve been working like crazy with an amazing team on an amazing application: Wesabe. Everything you hate about your bank website? We fix. Everything you need to know about your money, past, present, and future? You get it. We’re going live reeeeeaally soon, so check us out!

Also, I’m going to be a panel member for this month’s Ruby Tuesday, the San Francisco Bay Area’s Ruby user group. The topic is “Putting Rails To Work,” and I’ll get to squabble with folks like Josh Susser, with whom I’ve only squabbled bloggo-a-bloggo. I’m pretty sure we’ll have launched by then, so I’ll probably be all strung out on developer post-partum depression, or covered in sweat and soot from putting out fires. Expect to see folding chairs fly, folks. (I can’t help but feel a little out of place, seeing my name next to Mr. Rails Associations, Rabble, Florian frickin’ Weber, and Chris “Err The Blog” Wanstrath. Who wants to bet that someone’ll just point to me and shout: “Hey! Who the hell is that guy!” If that happens, I’ll spin around and point at Josh and yell “Yeah, who the hell are you?!”) I’m sure it’ll be a blast–Josh and I owe each other reconciliatory beers from way back when, and I really want to talk with Rabble about his social activism (little known fact–my degree was in Peace & Conflict Studies, with an emphasis in Nonviolent Social Movements).

(This month’s Ruby Tuesday is at Odeo, which means it’s all full. So, uh, crash it. And bring some beer–Daddy’s gotta speak in public.)

Go see Half-Nelson

(as seen from the lobby of one of the theaters on Shattuck Ave.)

So. Movies.

First, let me be up front: I am shilling for a friend here. Once I’ve disclosed the full extent of this, I’ll go on to convince you that it doesn’t matter. Ryan Fleck directed this film and co-wrote it with his girlfriend, Anna Boden, who produced it. Ryan’s dad and my dad lived in a Berkeley commune together, and Ryan’s kind of like my fictive cousin. Friend of the family, not terribly close, but still related, but not technically. Communes make things complicated like that.

So when we went to see it at the SF Film Society’s International Film Festival in April, I was totally willing to cheer my head off for whatever the hell he had made, and clap for family’s sake. Luckily, he’s made an insanely good film. Like, easily the best film I’ve seen in two years–and I’m hard to please. It picked up the Fipresci prize there, and from what I can tell just about every critic who’s ever seen it loved it. It got reviewed on Ebert & Roeper, where Roeper and guest critic Kevin Smith couldn’t stop telling the audience to see it. Entertainment Weekly gave it an A, Time Out New York gave it 5/6, and The Onion gave it an A-. Its Rotten Tomato meter is just under 90%.

The short version? This is an amazing film. It opens in a handful of theaters on Friday. If you pass it up, you’re missing out.

Stupid Ruby Tricks: String#to_proc

I was thinking about one of the cool bits of ActiveSupport, Symbol#to_proc, which allows you to do this:

[1, 2, 3].map(&:to_s) #=> ["1", "2", "3"]

That’s useful for sorting, enumerating, mapping, etc. Quite handy, and it saves you the trouble of writing a block to return a single value. I was recently someplace with a fair bit of time to kill and no reading material, and I began to wonder… how does this work?

Magic via one of the few typographic characters I can’t even write

Oh, the ampersand. Curly figure of import. That little pretzely freak is the sin qua non for this entire operation. You can tell, because Array#map acts all angry when you try to just pass it a Symbol:

irb(main):001:0> [1, 2, 3].map(:to_s)
ArgumentError: wrong number of arguments (1 for 0)
        from (irb):1:in `map'
        from (irb):1

The ampersand is how Ruby denotes a block variable.

Let’s try to pass our block in a different way:

my_block = lamdba{ |x| x.to_s }
[1, 2, 3].map(my_block)

Ruby will raise an ArgumentError because it’s trying to pass your block as a regular method parameter, rather than as a block parameter. Let’s try passing it as a block parameter:

[1, 2, 3].map(&my_block) #=> ["1", "2", "3"]

Sweet! It worked!

Now we’ve got a better of idea of what the hell we’re actually doing. When we write map(&:to_s), we’re saying “call the map function and pass it this block.” But a Symbol isn’t a block!

Very true, and what does Ruby try to do when presented with a type mismatch? Coerce! How? Conversion methods! Anything with a to_proc method can be converted to a proc. Our first example can be rewritten as follows:

[1, 2, 3].map(&:to_s.to_proc) #=> ["1", "2", "3"]

This means that anything with a to_proc method can partake of this lovely syntactic sugar.

Pushing the limits for no apparent reason

And now for the inevitable section of any article on programming that I write: taking things too far.

class String
  def to_proc
    eval "Proc.new { |*args| args.first#{self} }"
  end
end

robots = []
robots << { :name => 'Jeevesotron',  :type => 'robo-butler' }
robots << { :name => 'Crushmonster', :type => 'robo-baby-sitter' }
robots << { :name => 'Fluffy',       :type => 'doombot' }

robots.map(&'[:name].upcase') #=> ["JEEVESOTRON", "CRUSHMONSTER", "FLUFFY"]

It may not be useful, but it sure as hell is fun.

CRUD Explained

As it’s used on the Rails community, CRUD is when you limit your controller actions to index, new, edit, show, and destroy.

I eagerly await counter-examples.