What should the server do? Server static files? Ruby? Python? I think with the latter you'll get into trouble quickly (at least Ruby) in the memory-usage area.

But, given you can run your stack on a single RasPi. Use one to run HaProxy in front of it.