Here’s the call graph for #strip_tags (as profiled in an application I’m working on). As you can see, it tokenizes the entire string, and then iterates the tokens, likely pushing and popping sections onto and off of a stack as tags are opened and closed.

This is a lot more than a quick little regex to strip out tags; it’s actually parsing the full HTML document. Fortunately, there are already tools to do that, and they have their slow parts written as C extensions. Nokogiri is my weapon of choice in this regard – it’s battle-tested and generally rocks at parsing markup, even when it’s poorly-formed.

So, let’s benchmark a “strip all the markup out of a string” use case with #strip_tags and nokogiri.

require 'rubygems'
require 'action_view'
require 'nokogiri'

include ActionView::Helpers::SanitizeHelper

f = open("news").read

LOOPS = 1000
Benchmark.bmbm do |x|
  x.report("strip_tags") { LOOPS.times { strip_tags f }}
  x.report("nokogiri") { LOOPS.times { Nokogiri::HTML(f).text }}
end

The data file in this case is a snapshot of the current page of Hacker News. It’s a 23kb HTML file. Nothing too huge, but certainly not small, either. Let’s run it through the benchmark:

[chris@luna projects]$ ruby strip.rb
Rehearsal ----------------------------------------------
strip_tags  33.070000   0.010000  33.080000 ( 33.092638)
nokogiri     3.220000   0.020000   3.240000 (  3.241090)
------------------------------------ total: 36.320000sec

                 user     system      total        real
strip_tags  33.010000   0.010000  33.020000 ( 33.056551)
nokogiri     3.190000   0.000000   3.190000 (  3.200680)

Yikes. It’s not just slower, it’s ~10x slower.

Don’t use strip_tags. Also, profile your code. But just because it’s convenient doesn’t mean you should use it.

Fortunately, Ruby is deeply entangled with another more portable serialization format: YAML.

Rails manages its session cookies through the MessageVerifier. Easy enough – we can just write our own MessageVerifier that uses YAML rather than Marshal.

module ActiveSupport
  class YamlMessageVerifier < MessageVerifier
    def verify(signed_message)
      raise InvalidSignature if signed_message.blank?

      data, digest = signed_message.split("--")
      if data.present? && digest.present? && secure_compare(digest, generate_digest(data))
        str = ActiveSupport::Base64.decode64(data)
        if str[0..2] == '---'
          YAML::load str
        else # Handle old Marshal.dump'd session
          Marshal.load(str)
        end
      else
        raise InvalidSignature
      end
    end

    def generate(value)
      data = ActiveSupport::Base64.encode64s(YAML::dump value)
      "#{data}--#{generate_digest(data)}"
    end
  end
end

You’ll notice that verify() can accept a Marshaled session as well; this lets you transparently transition existing cookies to the new format without any kind of session breakage. Nice.

Now, to use the verifier, we monkeypatch CookieStore:

module ActionController
  module Session
    class CookieStore
      def verifier_for(secret, digest)
        key = secret.respond_to?(:call) ? secret.call : secret
        ActiveSupport::YamlMessageVerifier.new(key, digest)
      end
    end
  end
end

Now, this will work…at least at first glance, until you try to use the flash. This is a particularly nasty little problem, and it stems from the fact that Ruby’s YAML implementation serializes Hash objects without their instance variables, and FlashHash inherits from Hash, and thus inherits its serialization/deserialization strategy. I worked for a while to monkeypatch those strategies, but I didn’t like the result, and it felt a little hacky. Instead, I just took advantage of the YAML load lifecycle to make sure the FlashHash initializes properly:

module ActionController
  module Flash
    class FlashHash
      def update_with_initializer(h)
        @used ||= {}
        update_without_initializer(h)
      end
      alias_method_chain :update, :initializer
    end
  end
end

The core problem is that YAML::load calls Hash#update, and FlashHash presumes that the @used instance variable is present and initialized to an empty hash. To fix that, I just aliased in an initializer to make sure that variable is set.

Note that if you are storing other Hash subclasses with instance variables that rely on those variables being persisted across sessions, they will break. However, you should only be storing primitive/array/hash data in the session if possible. FlashHash is sort of a nasty violation of this principle.

At this point, your session should be serializing to and from YAML. We’ll want to read it from PHP, naturally. I’m using SPYC in the PHP project, which gets us Close Enough(TM). It doesn’t handle symbol keys, but we’ll handle those in the PHP itself.

Reading from PHP

Reading the data back out is surprisingly simple. We have to verify the authenticity of the data, of course, by checking the hash, but then you just base64 decode the data, load it with spyc, and perform some simple transformation to turn symbols into strings. If you wanted to make it even easier, you could monkeypatch the cookie store to call #stringify_keys! on your session hash before serializing it (and then call #with_indifferent_access on the hash when you deserialize it. Be aware of the speed impact of such a decision before you do it.)

function explode_symbols($arr) {
  $result = array();
  foreach($arr as $key => $val) {
    if(is_numeric($key) && $val[0] == ":") {
      $bits = explode(":", $val, 3);
      $result[trim($bits[1])] = trim($bits[2]);
    } elseif (is_array($val)) {
      $result[$key] = explode_symbols($val);
    } else {
      $result[$key] = $val;
    }
  }
  return $result;
}

function deserialize_session($session_key, $secret) {
  list($session64, $hash) = explode("--", $_COOKIE[$session_key], 2);
  if(hash_hmac("SHA1", $session64, $secret) == $hash) {
    $session = base64_decode($session64);
    return explode_symbols(spyc_load($session));
  } else {
    throw new Exception("Invalid session signature");
  }
}

$rails_session = deserialize_session("your_session_cookie_name", $your_session_cookie_secret);

Caveats

• Be aware that YAML is slower than Marshal
• Be aware that storing Hash subclasses in the session is likely going to Not Work.

And that’s all there is to it. You can now share data between the two apps via the session cookie.

I couldn’t find a quick and easy answer to this with some Googling, but it turns out that the answer is fortunately rather simple.

In your test_helper.rb:

 def setup
    Resque.redis.select 1
  end

  def teardown
    Resque.redis.keys("queue:*").each {|key| Resque.redis.del key }
  end

That’s all there is to it. The setup causes Resque to write to database #1 (#0 is default, and is what your development environment is likely using), and the teardown just deletes all your queues (which are really just lists of jobs to run). Test all you want and you won’t have to worry about tens of thousands of jobs junking up your redis DB.

After continually slimming down the code, I realized that even though it’s tiny, it’s danged useful to be able to just drop this into a Rails app and go. Thus, I’d like to present Tarot, my Rails configuration solution.

Tarot’s current form is heavily inspired by the Rails I18n usage, and is very quick and easy to use in your app. The generator installs a sample yaml file at config/tarot.yml, as well as an initializer to bootstrap your configuration and provide a handy helper method for quick access to those config values.

Assuming you have a config file like so:

---
base: &base
  foo: bar
  nested:
    tree: value
  array:
    - value 1
    - value 2

development: &development
  <<: *base

test: &test
  <<: *base

production: &production
  <<: *base
  foo: baz

You’ll notice that all the environments inherit from your base environment; this gives you an easy way to define common settings once, then override them per environment. Handy!

You could can access values by key, or by dot-delimited path:

config('foo') => 'bar'
config('nested.tree') => value

Default values are similarly easy.

config('foo.missing', 42) => 42

Finally, while Tarot will read your current application environment’s config, if you want to reach into another environment, that’s likewise easy:

config('foo', nil, 'production') => 'baz'

As of 0.1.2, Tarot also supports method_missing invocation:

Config = Tarot::Config.new('settings.yml', Rails.env)
Config.foo.bar.baz => "bin"

It also supports default values:

# Assuming foo has no subkey bar
Config.foo.bar("default") => "default"

But it’ll fail if you try to invoke method_missing on a non-leaf node

# Assuming that there is no `blaze` tree
Config.blaze.blarg => NameError

That’s about all there is to it — config isn’t (or shouldn’t be) a hard problem, so there’s not a whole lot to it, but it should get you up and running with easily-configured Rails apps in seconds.

It’s been broken for some time now, and not really in much of a state to be used by anyone, but I needed it, so I fixed it up. You can get my source on github, but I’ve done one easier and made it a gem. Major changes are:

• Bad assumptions fixed. Actually works now.
• Reorganized the whole thing and repackaged it as a gem. Doesn’t work as a Rails plugin anymore, but that’s okay. Just add it to to your environment or Gemfile and you’re good to go.
• Added tests!
• Fixed documentation, and normalized rake tasks names.

To get it, you can just gem install mongrations and you’re off to the races.

This is primarily going to be useful for MongoMapper people, who are likely still on Rails 2.3.x. If you’re using mongoid, check out mongoid_rails_migrations instead.

• Varnish is easy to use, if your app isn’t setting session cookies until you actually need them. The presence of a session cookie usually means that content shouldn’t be cacheable.
• Hitting any page with a form results in Rails generating a CSRF token and sticking it in the session, generating a session cookie and effectively locking the rest of the session out of being cacheable (even if it should be).
• Just *breathing* the method session in your app initializes your session.
• The Rails cookie session middleware assumes you always want to write a session cookie.

Fortunately, since so much of this is in Rack middleware, we can fix its mistakes with a middleware of our own. In a nutshell, I’m going to:

• Avoid writing CSRF tokens until we actually need them
• Check and see if we have an “empty” session (ie, no interesting data, just the session ID)
• Prevent session cookies from being sent to the browser unless there’s actually useful data in them.

Let’s get started. There’s a lot to this, and it’s a delicate collection of hacks, but it works nicely.

The care and feeding of CSRF tokens

To get started, I disabled CSRF functionality based on user state. In your application_controller.rb:

  protect_from_forgery :if => :user?
  skip_before_filter :verify_authenticity_token, :unless => :user?

protected

  def form_authenticity_token
    if user?
      session[:_csrf_token] ||= ActiveSupport::SecureRandom.base64(32)
    end
  end

In this case, user? is a method from my authentication framework that lets me check if I have an active session. The astute reader will note that this check performs the aforementioned breathing-on (and thereby initializing) the session, so it’s unfortunately not quite as simple as this. However, this will prevent authenticity checks for unauthenticated sessions. There’s no real point to them if you aren’t performing privileged operations anyhow, so we’ll just save the overhead.

Stuffing your cookies back into the jar

Next, we need to deal with the session cookie itself. We have two options when invalidating cookies – either strip them from the already-written headers, or just add another Set-Cookie line to instantly invalidate them. Since we’re dealing with Varnish, we want option 1 – ideally, we won’t be passing the Set-Cookie header, since Varnish (by default) won’t cache any response that attempts to set a cookie.

Session cookie management happens in ActionController::Session::CookieStore, and it’s really high up in the middleware stack. rake middleware will dump your stack – you’ll find it’s usually in position 2 or 3. So, in order to tweak it, we’ll need to inject a new middleware into your stack to mess with your cookie headers after the cookie handler itself blindly writes them out.

Scroll down if you want the code, but the gist of it is this:

• Check for a special “cookie.logout” environment parameter. If this is present, we’re going to just flat-out nuke the session cookie. More on this later.
• Otherwise, check to see if the session has any interesting keys. If it doesn’t, remove it from the Set-Cookie header

The code itself. Drop this in lib/strip_empty_sessions.rb.

class StripEmptySessions
  ENV_SESSION_KEY = "rack.session".freeze
  HTTP_SET_COOKIE = "Set-Cookie".freeze
  BOGUS_KEYS = [:session_id, :_csrf_token]

  def initialize(app, options = {})
    @app = app
    @options = options
  end

  def call(env)
    status, headers, body = @app.call(env)

    session_data = env[ENV_SESSION_KEY]
    sc = headers[HTTP_SET_COOKIE]
    if env["cookie.logout"]
      value = Hash.new
      value[:value] = "x"
      value[:expires] = Time.now - 1.year
      cookie = build_cookie(@options[:key], value.merge(@options))

      if sc.nil?
        headers[HTTP_SET_COOKIE] = cookie if env["cookie.logout"]
      elsif sc.is_a? Array
        sc << cookie if env["cookie.logout"]
      elsif sc.is_a? String
        headers[HTTP_SET_COOKIE] << "\n#{cookie}" if env["cookie.logout"]
      end
    elsif (session_data.keys - BOGUS_KEYS).empty?
      if sc.is_a? Array
        sc.reject! {|c| c.match(/^\n?#{@options[:key]}=/)}
      elsif sc.is_a? String
        headers[HTTP_SET_COOKIE].gsub!( /(^|\n)#{@options[:key]}=.*?(\n|$)/, "" )
      end
    end

    [status, headers, body]
  end

  private

  # Copied from the cookie session middleware.
  def build_cookie(key, value)
    case value
    when Hash
      domain  = "; domain="  + value[:domain] if value[:domain]
      path    = "; path="    + value[:path]   if value[:path]
      # According to RFC 2109, we need dashes here.
      # N.B.: cgi.rb uses spaces...
      expires = "; expires=" + value[:expires].clone.gmtime.
        strftime("%a, %d-%b-%Y %H:%M:%S GMT") if value[:expires]
      secure = "; secure" if value[:secure]
      httponly = "; HttpOnly" if value[:httponly]
      value = value[:value]
    end
    value = [value] unless Array === value
    Rack::Utils.escape(key) + "=" +
      value.map { |v| Rack::Utils.escape(v) }.join("&") +
      "#{domain}#{path}#{expires}#{secure}#{httponly}"
  end
end

Next, you’ll need to add this to your middleware stack. In your environment.rb:

config.middleware.insert_before "ActionController::Session::CookieStore", "StripEmptySessions", :key => "your_session_key", :path => "/", :httponly => true

The :key and :path parameters should match your session cookie settings.

What this will do is let this middleware run on the way back up the stack, right after the session handler gets a crack at things. If there is nothing interesting in the session, it’ll remove that line from the Set-Cookie header, so if you aren’t setting any other cookies, the header should end up being empty and should get thrown away. If you triggered a logout, will invalidate the client cookies (rather than just writing a cookie with no data in it back to them).

To do that, you’ll need to modify your logout method:

def logout
  request.env["cookie.logout"] = true
end

That should be it. You should now:

1. Not be setting session cookies for empty sessions
2. Not be setting CSRF tokens for anonymous sessions
3. Not be leaving “empty” session cookies laying around on client machines after a logout.

The net result is that you should be cookieless for anonymous sessions, resulting in trivial caching with Varnish. This can vastly improve the performance of your site – especially if you’re catching high-traffic pages from web crawlers and the like with Varnish, so they never touch your Rails stack.

Cookie image (C) scubadive67, used under Creative Commons license

It’s not too hard to set up fixtures to be inserted once per test suite run –

In your test_helper.rb, above the class ActiveSupport::TestCase definition, add the following:

Fixtures.reset_cache
fixtures_folder = File.join(RAILS_ROOT, 'test', 'fixtures')
fixtures = Dir[File.join(fixtures_folder, '*.yml')].map {|f| File.basename(f, '.yml') }
Fixtures.create_fixtures(fixtures_folder, fixtures)
Fixtures.reset_cache

Next, turn off transactional fixtures and comment out the fixtures macro:

self.use_transactional_fixtures = false
# fixtures :all

That’s all there is to it. Your fixtures will be inserted into your test database once when test_helper is included for the first time, and then not again for the rest of the test suite run. This should speed your tests up substantially.

I recently discovered some very cool functionality it includes – the WillPaginate::Collection class can be used as a custom paginator for effectively any enumerable collection. It’s very simple, too. I recently used it to build pages of the most popular tags on posts in my database. My data store is MongoDB, and I’m fetching an array consisting of two-element arrays, [tag, tag_count]. To use will_paginate’s functionality with this, I just use the following:

tags = Post.tag_counts(nil, {:sort => ["value", "descending"]}) # Return an array of tag/count pairs. Custom function, so it can't leverage the finder on Post.
@topics = WillPaginate::Collection.create(current_page, 20, tags.length) do |pager|
	pager.replace(tags.slice(pager.offset, pager.offset + pager.per_page))
end

current_page is a helper that derives the current page from the request parameters. The rest of it is self-explanitory. I can now use @topics in my page just as I’d use a paginated result set from the database.

- @topics.each do |topic|
    # ...
=will_paginate @topics

Bam. Doesn’t get much easier than that. You can get exceptionally creative with it, too. Effectively, all you need to know is:

• WillPaginate::Collection#new takes 3 parameters: the current page, the per-page count, and optionally, the total number of entries.
• The pager block variable exposes offset and per_page properties, prime for passing into a DB query or slicing an enumerable with
• Call pager.replace(sub-array) with the current page’s set of elements.

That’s literally all there is to it. Now you can have easy pagination on just about any collection you can conceive of. Let WillPaginate handle all the heavy lifting and such. If you’ve done enough pagination by hand, you’ll probably appreciate the easy beauty of this particular method.

In short, I didn’t want to hack around with the MongoMapper associations code, so I just implemented my own little ride-along version.

module SecretProject
  module CounterCache
    module ClassMethods
      def counter_cache(field)
        class_eval <<-EOF
          after_create "increment_counter_for_#{field}"
          after_destroy "decrement_counter_for_#{field}"
        EOF
      end
    end

    module InstanceMethods
      def method_missing(method, *args)
        if matches = method.to_s.match(/^(in|de)crement_counter_for_(.*)$/) then
          dir = matches[1] == "in" ? 1 : -1
          parent_association = matches[2]
          if parent = self.send(parent_association) then
            name = "#{self.class.to_s.tableize}_count"
            if parent.respond_to?(name)
              parent.collection.update({:_id => parent._id}, {"$inc" => {name => dir}})
            end
          end
        else
          super
        end
      end
    end

    def self.included(receiver)
      receiver.extend         ClassMethods
      receiver.send :include, InstanceMethods
    end
  end
end

Throw that into your lib directory, load it with an initializer, and then you can use it something like so:

class Foo
  include MongoMapper::Document
  include SecretProject::CounterCache

  belongs_to :user
  counter_cache :user  # Will cause a foos_count field on the owning user to be maintained when a Foo is created or deleted.
end

This’ll only increment a counter if you’ve defined one on your parent object, via key :foos_count, Integer or similar, just so that it doesn’t go around updating every model you might associate it with.

Yay.

# Controller
caches_action :show, :unless => :user?, :expires_in => 24.hours

# Sweeper
expire_action :controller => "nodes", :action => "show", :id => record.to_param

This all works dandy, but I generate pretty URLs, which means sometimes there are characters in the URL that Memcached doesn’t like. A few minutes after deploying my patch, I started getting IMs from my logger bot telling me things were unhappy.

blippr. com: [#1265856785] ArgumentError: illegal character in key "views/m.blippr.com/apps/346562-PicFo g.mobile"
blippr. com: [#1265857710] ArgumentError: illegal character in key "views/www.blippr.com/apps/336714-µTorrent  "
blippr. com: [#1265857897] ArgumentError: illegal character in key "views/www.blippr.com/apps/337076-ustre am"
blippr. com: [#1265857924] ArgumentError: illegal character in key "views/www.blippr.com/apps/336714-µTorrent  "

That’s memcached complaining about the hash keys we’re giving to it. This just won’t do. We could just regex out “bad” characters, but that means potential collisions, and potentially leaves edge cases. Why not just hash it instead?

A quick monkey patch later:

class ActionController::Caching::Actions::ActionCachePath
	def path
		@cached_path ||= Digest::SHA1.hexdigest(@path)
	end
end

And we’re all dandy. Now, rather than caching by path, the path is hashed, and the hash is used as the path key. Since hashes will always be hexadecimal characters, we know that it’ll never make memcached unhappy.

Path is blippr.com/movies/6696-The-Silence-of-the-Lambs...
Cached fragment hit: views/9111cdefca4a52cb0e3a5ebac4f618127a30efd0 (1.1ms)

There is an argument for not using this technique if you’re using file-based caching, since it means your cached bits won’t be segregated into directories, but memcached doesn’t support expiry by regex anyhow, so there’s no good reason to not use it in this case.

Enjoy!