Displaying articles with tag

Filtering, sorting, paging... oh, my!

Posted by andy, Thu Aug 13 15:00:00 UTC 2009

The Problem

What do you do when your end user needs to filter? In some cases the answer is easy: provide a custom method for the sort criteria. For example, when you know that your end user wants to know "my jobs" as distinct from "all jobs" a custom 'mine' method might work just fine. Don't worry that the custom routes are not RESTful; they are, they are just not the default Rails notion of REST. That's okay; Rails is not necessarily right.

The filtering that I run into is rarely that simple. Right or wrong end users tend to think more options is better. In the context of filtering that means they often want to filter by a combination of ad-hoc attribute values. Clearly a single custom method won't work there because you can't out guess the combination and permutations that a given user might want to use.

Sorting and paging further complicate the issue. Even when you can pick the 'right' values the user may want them back in the 'right' order. Paging is just a small wrinkle on top of that but yet another piece to layer in.

The Context

The most frequent context for the type of filtering/sorting/paging that I'm talking about is when the user is presented with a list of things that needs to be pared down and organized before the user can make any sense out of it. Maybe they're looking at a list of members in a club or a set of subscriptions or a bunch of invoices. Whatever the case, you've given them a set of things to review in the order you thought best but it doesn't make sense to them. Rather than haggle over who's right (you... of course!), you decide to give them a filtering pane and some click-to-sort headers in the grid. Great. The UI is done. But where does it all go? Where do you send the values?

More often than not, this context leads me right back to where it started. You began presenting the user a list that naturally flowed through the #index action of your controller. Why should the filtered or sorted or paged list be any different? It's still just a list, isn't it?

The Solution

In my mind the answer to that question is yes. A filtered list is still just a list so if the original list came from the index action of my controller then the filtered version of it should come that way, too.

The way that I've settled on implementing most of these scenarios lately is through a combination of named scopes, anonymous scopes, and some recommendations from Uncle Bob's Clean Code.

Scopes

Named scopes and anonymous scopes are hardly new territory. Named scopes came into Rails at version 2.1 courtesy of a very popular plugin. The idea behind named scopes, if you're not familiar with them, is that you create a method that 'scopes' a query against your ActiveRecord-based model. This is really handy if you have a scope like 'current' or 'active' that you can use over and over in your application; you just declare it once on the class and use it wherever you need it.

class Order < ActiveRecord::Base
  named_scope :recent, {:limit=>10, :order=>'id DESC'}
  named_scope :since, lambda{|date| {:conditions=>['created_at > ?', date]}}
end

Order.recent
#=> [...up to 10 orders]

Order.since(1.month.ago)
#=> [... all orders created in the last month ...]

Just as a quick refresher in case you're not using named scopes (as you should be!), you create a named scope on a class by calling the named_scope method in the class definition. The first parameter to the method is the name of the 'scope' and the second argument is either a hash of find options or a lambda that returns a hash of find options.

Anonymous scopes are pretty much the same thing. Really, the only difference is that named scopes are not saved against the model. You create them only where you need them and let the Garbage Collector have them when you are done with them.

Order.scoped( {:limit=>10, :order=>'id DESC'} )
#=> same results as Order.recent

What makes named and anonymous scopes so useful for the type of ad hoc filtering and sorting that we're considering is that you can chain them. By chaining named and anonymous scope calls you build up the query conditions. Thus you do not have to worry too much about the what combination of filering, sorting, and paging the user might want, you simply accumulate them. The query is not actually run against the database until you first try to use the data.

Order.recent.since(1.week.ago)
#=> Up to ten orders from the last week

Clean code

So what does clean code have to do with this? In the third chapter of his 2009 book by the same name, Bob Martin makes an excellent case for code that's organized into a series of very tiny, top-down methods. This, again, is not really new territory for most developers. Martin just pushes it to an end that I'd not really considered before and the discussion of the top-down organization somehow clicked with me in a way that it had not before.

The clean code approach to the filtering/sorting/paging problem would look at the demonstration in Ryan Bates' Railscast* and say it's too big. The filtering should be refactored into it's own method, so that the index can focus solely on getting the right type of view to render out the values. That filtering method should be broken down into a series of smaller methods, each of which performs one specific type of filtering/sorting/paging.

class ProductsController < ApplicationController
  def index
    filter_products
    sort_products

    respond_to do |format|
      format.html # render index.html.erb
      format.xml { render :xml=>@products.to_xml }
    end
  end

  private
  def filter_products
    initialize_scope
    filter_by_name 
    filter_by_category
    filter_by_price
  end

  def initialize_scope
    @products = Product.scoped {}
  end

  def filter_by_name
    return if params[:name].blank?
    @products = @products.scoped :conditions=>[ 'name LIKE ?', "#{params[:name]}%" ]
  end

  def filter_by_price
    filter_by_minimum_price unless params[:minimum_price].blank?
    filter_by_maximum_price unless params[:maximum_price].blank?
  end

  def filter_by_minimum_price
    return if params[:minimum_price].blank?
    @products = @products.scoped :conditions=>['price < ?', params[:minimum_price]]
  end

  def filter_by_maximum_price
    return if params[:maximum_price].blank?
    @products = @products.scoped :conditions=>['price < ?', params[:maxiumum_price]]
  end

  def filter_by_category
    return if params[:category_id].blank?
    @products = @products.scoped, :conditions=>{:category_id=>params[:category_id]}
  end
end

Conclusion

Okay, so nothing too earth-shattering if you've been deep into Rails for a while but hopefully a help if you're a relative novice. Have we gained anything? Sure, we've gained a few things. The use of named and anonymous scopes has greatly simplified our ability to respond to the user who desires to do ad hoc filtering against our data. They also make sorting and paging relatively simple -- just add an appropriate scope that users :order, :limit, and :offset appropriately and you can sort or page any of the target result sets the user could dream up. Just as importantly the top-down organization of very tiny (generally private) methods makes the top-most levels of the program (the index and initial filter method) read somewhat like a newspaper article: the headline (index) describes the main event, the first paragraph (#filter) provides the most important details, and the subsequent 'paragraphs' (lower methods in the chain) progressively reveal the implementation details. We have a simple solution to a problem with many combinations that should be easy to read and maintain in the future.

A side benefit

This approach saved me a lot of work earlier today. I had a rather inelegant solution to a particular problem in which I needed to filter and sort a list of members. The twist was that I needed the exact same sorting and filtering for two completely different contexts with very different views representing the results. The initial approach was to send the filtering request to the #index method of the MembersController; it seemed like a natural place to organize the member-specific logic. The method chose the view for rendering the results based on the 'commit' value (the name of the submit button). Unfortunately, a certain "non-modern browser" was not supplying the commit value. Ugh!

How did we keep it DRY? Simple. Since the code was organized with this top-down approach I simply extracted the methods from the controller class into a module and included the module in the two controllers that needed the search logic. The big benefit was that I only needed to move the lower-level functions; the higher functions (e.g., #index) remained in place. Actually the #index method was simplified since it could now focus on rendering the list of Members in only one way; the other controller picked up the single call to filter and rendered the results as it needed to. Simple code, simple refactoring, DRY solution.

Anyone have a glass of water?


* No offense intended to Ryan or his Railscast, which was focused on the mechanics of using anonymous scopes and immensely helpful in helping me get a handle on how to use them.

0 comments | Filed Under: Rails Ruby | Tags:

5 Tips for ActiveResource

Posted by andy, Thu Apr 24 16:30:00 UTC 2008

The first couple of tips have an indrect impact on ActiveResource. Still, they are worth keeping in mind because they simplify the data with which ActiveResource deals.

Tip 1: Use delegate and :method for encapsulation

If your crash course in Ruby involved reading the Agile book, then the delegate method may be new to you. Delegate is a class-level command that allows you to pass certain method calls on to an associated model. For example, if you have a highly-factored address book you might have a pair of models like this:

class Address < ActiveRecord::Base
  belongs_to :zip_code
end

class ZipCode < ActiveRecord::Base
  has_many :addresses
end

That's a model with some theoretical purity... but in practice it's cumbersome. You really want to deal with an address that has all the information you'd like to render (street, city, state, zip) in on model. Atleast it should feel that way. That's precisely where the delegate command comes into play.

class Address < ActiveRecord::Base
  belongs_to :zip_code
  delegate :city, :state, :zip, :to=>:zip_code
  delegate 'city=', 'state=', 'zip=', :to=>:zip_code
end

Modeled as shown above you can ask an address for it's city and the address will pass the request on to the zip_code object to which it belongs, retrieve the answer, and return it to you. (It's taking advantage of the fact that Rails is doing some method_missing magic to provide getters and setters for your attributes). That level of encapsulation will become increasingly important when you begin to use ActiveResource heavily. In many cases you may want to return only a few fields from an associated model and, as in the example above, you do not want or need to reveal how you've organized your data to the outside world.

The final piece to the puzzle with respect to ActiveResource will be making sure you use the :method parameter when you serialize the delegating object to xml.

addresses_controler.rb

...
def show
  @address = Address.find(params[:id], :include=>:zip_code)
  respond_to do |format|
    format.html # show.html.erb
    format.xml  { render :xml => @address.to_xml(:methods=>[:city, :state, :zip])
  end
end
...

As shown, the call to @address.to_xml tries to include the results of calling the city, state, and zip getter methods on address. The delegate command causes the Address object to pass that request on to the association ZipCode object and the results are returned and placed into the xml envelope as if they were attributes of the address (which they are, indirectly). The application that's consuming all this through ActiveResource remains blissfully unaware of your modeling nirvana. It simply receives some nicely formatted xml along the lines of this:

<home-address>
  <id type="integer">1</id>
  <street>123 Main St.</street>
  <city>Anytown</city>
  <state>XX</state>
  <zip>12345</zip>
</home-address>

Tip 2: Clean up the delgation you just learned to keep the code clean and clear

If you start maximizing your use of delegate your code can get untidy especially since delegate introduces some duplication when you're dealing with attribute accessors. If we keep in mind that class declarations are still Ruby scripts then we can clean the attribute accessor delegation pretty easily while making the intent very clear.

class Address < ActiveRecord::Base
  belongs_to :zip_code
  [:city, :state, :zip].each do |delegated_accessor|
    delegate "#{delegated_accessor}", "#{delegated_accessor}=", :to=>:zip_code
  end
end

On to some tips with more direct bearing on ActiveResource itself.

Tip 3: Use AppConfig to get your site information out of the class file!

The Core did a great job modeling ActiceResource along the lines of ActiveRecord so that using ActiveResource feels very natural to any Rails programmer. But it's also left me stumped as to why there is no equivalent to /config/databases.yml. I suppose that in some cases you will be using a well-known, established, public REST interface but I'm finding ActiveResource to be a very natural way to develop 'sub-applications' that can be shared to create a larger application. Because of that I need to be able to have different site information for development, test, and production. Clearly some configuration is needed.

Even though I shudder at the thoughts that a name like 'AppConfig' brings to mind, it's a great part of the solution to this problem. If you're not familiar with it, AppConfig allows you to provide a yaml config files for global (/config/app_config.yml) and environment-specific (e.g., /config/environments/development.yml) configuration. The plugin reads these config files, merges inforamtion as necessary, and provides all the options as class-level attributes of the AppConfig class.

sites:
  addressbook: http://localhost:3001
  financials: 
    url: http://localhost:3002
    username: money
    password: talks

The yaml above shows two different types of configuration that would be useful for ActiveResource, organized together under a 'sites' attribute. The first one (addressbook) is the way I started before I ran into an application that needed http basic authentication. The site info consists only of the url. The second one (financials) came out of the latter need. A quick extension of ActiveResource causes these to spring into action.

class ActiveResource::Base
  protected
  def self.establish_site_connection(site_id)
    raise(ArgumentError, "#{site_id} is not defined for #{RAILS_ENV}") unless AppConfig.sites.respond_to?(site_id)
    site_info = AppConfig.sites.send(site_id)
    return site_info.respond_to?(:url) ? site_with_basic_auth_info(site_info) : site_info
  end
  
  def self.site_with_basic_auth_info(site_info)
    site = URI.parse(site_info.url)
    site.userinfo = "#{site_info.username}:#{site_info.password}"
    return site.to_s
  end
end

I've been dropping the code above into /lib/core_ext/active_resource_extension.rb. The first method (establish_site_connection) is meant to emulate ActiveRecord::Base#establish_database_connection. It accepts a site id in the form of a symbol or string and retrieves the site configuration matching that id. If that site info is already a simple string, that string is returned unmodified. If the site_info is further broken down into the url, user name and password for http basic authentication then that is handed off to the site_with_basic_auth_info method to build up a simple string.

It's true that the http basic authentication credentials could be written into the url. In fact, that's exactly what the site_with_basic_auth_info does. If that's the case, then why add the username and password to the config file?

Tip 4: Share your site AppConfig settings between your applications

When you have the fortunate advantage of controlling both your ActiveResource-based application and your ActiveRecord-based application you can share the configuration information between the applications. Specifically, you can share the username and password information used for http basic authentication so that both sides can be externally configured... and reconfigured. By sharing the configuration files and including the use of AppConfig in the source application for the ActiveResource your http basic authentication will be as simple as

def basically_authenticated(user, password)
  user==AppConfig.sites.financials.username && password==AppConfig.sites.financials.password
end

What makes this even more compelling is that AppConfig (as anything leaning on yaml) allows you to use ERb in your configuration files. Why is that significant?

Tip 5: Use Embedded Ruby in your configuration files to automatically change your user/password

Clearly with http basic authentication you will want to go the extra step of passing through a secure connection, but if you're too tired to add an 's' to your http, then you'll want to change your clear-text password. Often. Embedding Ruby might be just the trick because you could share a single algorithm between your applications that would change the password for you.

sites:
  addressbook: http://localhost:3001
  financials: 
    url: http://localhost:3002
    username: <%= %w{money cash penny moulah dineiros pennywise poundfoolish}[Date.today.wday] %>
    password: <%= Digest::SHA1.hexdigest("#{Date.today.to_s}---financials") %>

There is a potential pitfall here. With this type of approach -- shifting the user/password each day -- the application servers will have to be kept in step. A reboot on one machine will require a reboot or restart on the other to make sure the applications share the same username/password since the AppConfig object will be re-loaded when the webserver starts. Pick the scheme that works best for you.

0 comments | Filed Under: Rails | Tags:

Why assign site in ActiveResource?

Posted by andy, Tue Apr 22 16:30:00 UTC 2008

ActiveResource is a great tool for helping your business keep not only its business logic DRY, but even keep its business applications dry. If you're not familiar with ActiveResource, think of ActiveRecord using an internet-based datastore. It's a bit more complicated than that but you can do all the basic CRUD methods, custom methods, etc

The advantage that ActiveResource brings, though, is that you only need to create the object once. Ever. Used effectively, you don't need to create an object in one project that you import or somehow reuse in another. You create a small, targetted application and share the application with other applications. For example, you could create an accounting engine that deals with ledgers and accounts and journals and expose the RESTful HTTP interface to higher level apps that simply consume the Journals and Ledgers and Accounts using ActiveResource. Within a single company it might be the ultimate in DRY.

For Rails developers, ActiveResource is very clearly modeled on ActiveRecord. If you've gotten used to one set of methods you should almost seamlessly be used to the other. With one painful exception: setting the site in the class. I honestly cannot understand why there is no configuration yaml equivalent to database.yml for ActiveResource. Maybe it was unnecessary since the creators already had some RESTful applications with which to work. Whatever the case, it's a real pain in the neck.

In an attempt to keep the ActiveRecord-like API going, I've come up with the following code that I've been dropping in /lib/core_ext/active_resource.rb

require 'yaml'
class ActiveResource::Base
  protected
  def self.establish_site_connection(site_id)
    site_yaml = File.new(File.join(RAILS_ROOT, 'config', 'sites.yml'))
    environment_configurations = YAML.load site_yaml
    site_configurations = environment_configurations[RAILS_ENV]
    return site_configurations[site_id.to_s]
  end
end

The code is supposed to emulate ActiveRecord.establish_database_connection. As implemented above it will add an establish_site_connection method to your ActiveResource class that will read a sites.yml file in your /config folder. sites.yml is structured similarly to database.yml -- you have entries for each environment (development, test, production, etc) along with site names and urls for each site.

development:
  activity_center: http://localhost:3002/
  church_member: http://localhost:3001/

test:
  activity_center: http://testy:3002/
  church_member: http://testy:3001/

With such a configuration file, of course, you have a few luxuries. First, you can use different sites while running in different environments. This might make it easier, for example, to create mocks for testing ActiveResource objects. Second, you can more quickly adapt to external changes (e.g., remote resource down or relocated) since it's just a yaml change and not a source code change.

I've typically gone one step further with the ActiveResource hack. As alluded to above, I have sites split into separate sub-applications each responsible for part of the end solution. As a result I have a whole family of ActiveResources that use one source application. For this reason I have emulated the multiple database solution for Rails with the following for ActiveResource.

require File.join(RAILS_ROOT, 'lib', 'core_ext', 'active_resource_extension')
class ActivityCenterResource < ActiveResource::Base
  # see /lib/core_ext/active_resource_extensison.rb
  self.site = self.establish_site_connection(:activity_center)
end

class ActivityCenter < ActivityCenterResource
  ...
end

0 comments | Filed Under: Rails | Tags:

STI Factory

Posted by andy, Mon Mar 17 17:30:00 UTC 2008

Single Table Inheritance

One of the abstractions that I really like in Rails is its implementation of Single Table Inheritance (STI). If you're not familiar with STI, it is a simple design pattern in which you model an inheritance hierarchy in a single database table (Martin Fowler does it more justice here). Since ActiveRecord, Rails' primary domain modeling base class, is also db-centric the marriage of the two is fairly straight forward: include a column called 'type' in your database table and you're done. Simple.

But type is a real headache

In practice it turns out that it's not always so simple. In a number of applications that I've worked on we like to put the user in the driver's seat by allowing them to select the subtype they are going to create. For example, assume that we start with a domain modeling different types of vehicles. Without getting into all the attributes that might distinguish the vehicles, the class model might look something like this:

class Vehicle < ActiveRecord::Base
  def self.inheritance_column
    'vehicle_type' # we'll see why in a bit...
  end
end

class Car < Vehicle
end

class Truck < Vehicle
end

What I'd really like to do is give the user a select and let them pick either 'Car' or 'Truck'. That in itself is should not be too difficult. There is one little gotcha: type is a reserved word in Ruby. If you try to use a select or select_tag helper Rails (Ruby) will complain with an error that will probably leave you scratching your head for a while. The simple way to avoid this problem is shown above. You override the class-level inheritance column method and return the name of the column that you will use to discriminate among classes in the inheritance hierarchy. In this case we're using the column 'vehicle_type' to hold the name of the subclass.

Things get trickier when you get back to the controller. It turns out that Rails musters up some righteous indignation about any attempt to change the class. To see what I mean, let's simulate what you might see back in the VehiclesController if you let the user request a Porche Cayenne...

params = HashWithIndifferentAccess.new(:vehicle=>HashWithIndifferentAccess.new(:vehicle_type=>'Car', :make=>'Porche', :model=>'Cayenne'))
=> {"vehicle"=>{"vehicle_type"=>"Car", "make"=>"Porche", "model"=>"Cayenne"}}
vehicle = Vehicle.new params[:vehicle]
=> #<Vehicle id: nil, name: nil, vehicle_type: nil, created_at: nil, updated_at: nil, make: "Porche", model: "Cayenne">

Already we can see there is a problem. The user sent back a request to build a Car, but what is being assembled is a generic Vehicle. The reasoning is pretty straightforward: you asked for a new Vehicle, not a new Car, so you got a new Vehicle. Perhaps too graciously, Rails assumed that you knew what you were asking for. Unforunately, you don't -- the user knows what is being created but you are clueless. You could try to build a big case statement, but that's very messy and you have to update it each time you add or remove a class from the hierarchy. It also suffers from the fact that it's too concrete; you can't transport your knowledge to any other STI implementation.

An STI Factory Method

This sounds like a classic case for the Factory Method design pattern. One of my favorite design pattern books (Head First Design Patterns) says that this pattern "defines an interface for creating an object, but lets subclasses decide which class to instantiate." If we translate that to Rubyisms and consider our problem, it sounds like we need a module (interface) that will mix into a class that will help the class pick from among its subclasses when it's asked for something new.

I've taken a stab at this a few times and never liked the results. Most of the time it felt like I was injecting too much code. I also got somewhat inconsistent results from the class-level array I was trying to build to maintain the list of subclasses. Recently I was working on a different problem and stumbled on some information that I'd forgotten from my first dance with Rails. I know that David Black told me that ActiveRecord maintained a protected list of subclasses just for STI, but it was washed away in the grey matter (probably because i read it at the beach... I'm a geek.)

Having been reintroduced to ARec#subclasses again, I've worked out an abstract STI factory. I built it as a plugin and the essence of it is in the code that gets mixed into the ActiveRecord base_class in the inheritance hierarchy.

def new(*args)
        target_class_name = requested_class_name(args)
        return self.base_class.factory(target_class_name, *args) unless self.name === target_class_name
        super
      end
      
      def factory(requested_class_name, *args)
        requested_class_name = base_class unless has_subclass_named?(requested_class_name)
        requested_class = requested_class_name.constantize
        requested_class.new(*args)
      end
      
      def type_options_for_select
        subclasses.collect{|subclass| [subclass.name.humanize, subclass.name]}
      end
      
      protected
      # Returns true if the STI tree includes a subclass with the specified name
      def has_subclass_named?(subclass_name)
        subclasses.detect{|subclass| subclass.name==subclass_name}
      end

      def requested_class_name(args)
        class_name = self.name
        if args.last.is_a?(Hash)
          requested_class = args.last.delete(self.inheritance_column.to_sym)
          class_name = requested_class unless requested_class.blank? or !has_subclass_named?(requested_class)
        end
        return class_name
      end

We'll read the code from the bottom up, mostly so that the helper methods make sense when we see them in context.

  • requested_class_name
    This helper method attempts to determine the name of the subclass that is being requested. Like Rails, it begins with the assumption that you knew what you were asking for (class_name=self.name) and then it searches the parameters it was passed to see if the :inheritance_column was passed. If so, it tries to return the value that was requested. There are two conditions on this: if the class name was blank it assumes that you wanted the class from which you requested something new. If you supplied a value but that value is not a subclass it assumes it was a typo (kinder than assuming you were a fool :-) For both cases it falls back to the class_name of the orignal class; otherwise it overrides with a subclass name. An important thing to note is that the subclass name is deleted from the options passed to the method. This is done to prevent an infinite loop but it means that you've got to keep a copy of the returned value.
  • has_subclass_named?
    This helper method checks the list of subclasses for the (base) class and makes sure that the requested class actually exists as a subclass.
  • type_options_for_select
    This is a convenience method for the select/select_tag. It builds an array that can be used as the options source with a human readable name for the class as the text and the class name as the value.
  • factory
    This method requires the name of the subclass. It takes advantage of the fact that Ruby classes are global constants (hence the call to constantize) to get a handle on the requested class and then invokes 'new' on it, passing in the parameters it received.
  • new
    This is the part that I like most about the plugin. The first thing that it does is check to see if the class you requested differs from the class that is trying to fill the request. If so, it automatically to the factory method and if not you proceed with the default 'new' behavior. The advantage to this is that you never have to know what you're trying to create and you don't have to remember to use the facotry method. You can just use new (or create) on this class like every other class... and it will figure out what you meant to do.

With that in place things look a bit different for the VehiclesController.

class Vehicle < ActiveRecord::Base
  has_sti_factory
  
  def self.inheritance_column
    'vehicle_type'
  end
end
...

params = HashWithIndifferentAccess.new(:vehicle=>HashWithIndifferentAccess.new(:vehicle_type=>'Car', :make=>'Porche', :model=>'Cayenne'))
=> {"vehicle"=>{"vehicle_type"=>"Car", "make"=>"Porche", "model"=>"Cayenne"}}

vehicle = Vehicle.new params[:vehicle]
=> #<Car id: nil, name: nil, vehicle_type: "Car", created_at: nil, updated_at: nil, make: "Porche", model: "Cayenne">

Now all I have to do is figure out how to make the VehiclesController fulfill that "create Porche Cayenne" request. :-)

0 comments | Filed Under: Rails | Tags: