While headless tests are faster, they are a pain to debug. I like to see tests running, and it provides the opportunity to see any styling issues throughout the application.
A problem I've encountered with all testing, whether using Selenium/Firefox, Poltergeist or headless Webkit, they've all got their specific annoyances that needs to be catered for when writing your tests. Random failures for no reason is very common, and for reason this I use rspec-retry.
While reading up on Rspec failures, I realized you could actually use Chrome along with Selenium instead of Firefox (which just broke again the last time it updated). To enable Selenium to use Chrome instead of Firefox, add the following to your spec_helper.rb:
Capybara.register_driver :chrome do |app|
Capybara::Selenium::Driver.new(app, :browser => :chrome)