Laissez les bons temps rouler!

Starting on the day before Mardi Gras, I will be joining EdgeCase working in their Cincinnati office. To me, a much bigger party than they'll be having in the French Quarter!

This is a phenomenal opportunity, working alongside the genius of Jim Weirich and Scott Barron every day, and collocated with the Gaslight Software crew. (Let me tell you, this is a really smart room.)

Many thanks to Joe, Ken, Adam and Leon for taking the time to meet with me and make sure I'd fit there, and to everyone who's supporting me in this transition. To be sure, I'll miss the folks I've been working with, but I'm ready to get cracking on new stuff!

A little Hash goodness

Some quick refactoring this weekend had me throwing together some tidbits.

module Enumerable

  # return Hash of enumeration to yielded values
  def collect_hash hash={}
    inject(hash) {|h,e| h[e] = block_given? ? yield(e) : nil; h }
  end

  # return Hash of enumeration to non-nil yielded values
  def select_hash hash={}, &block
    collect_hash(hash,&block).compact
  end

end


class Hash

  # return Hash with nil values removed
  def compact
    delete_if {|k,v| !v }
  end
  
  # array-style push of key-values
  def <<(hash={})
    merge! hash
  end

end

Yes, I know they've been done before, but they're quick one-liners and let me eliminate a lot of code. And as we all know, the easy code to maintain is the code you delete.

The Missing Link in RJS Chaining

One of the things that make Ruby and Rails my respective current language and web framework of choice is the collective ingenuity that has made building things easy. These folks have thought of everything, gosh darn it, and I get to use it. A good day is when I get something working, and with Ruby and Rails I have a lot of those days.

Imagine my dismay however, when on a recent project I had the need for something that wasn't already baked into Rails! The problem occurred when chaining partials together - one partial calling another, calling another, and so on down the line. I find myself doing it a lot these days, updating many parts on a page based upon seemingly simple model changes. Chaining partials together is easy, you just set up some context and render, and repeat.

Using Ajax and RJS is now a way of life; I got my baptism in the Javascript waters a long while back, a must for doing esoteric, dynamic rendering. RJS certainly isn't what makes such rendering possible, but it sure makes it easier. Some may think that there are better mechanisms - and maybe in special situations there are - but the simplicity and versatility of generating pieces of web pages using RJS is unparalleled once you understand the subtleties of building Javascript that will be executed in the context of a page.

Context is the issue. Let us say my foo partial renders my bar partial. The context is established as a hash of variables to values that is built by foo and passed into bar as the value of the :locals key in the render call. When bar is rendered, the hash is unraveled behind the scenes and the variables become available.

All except one, that is: the meta-variable that is the locals hash itself!

Unless I'm missing something, when you want foo to pass whatever locals it had received to bar, Rails comes up short! There's no way that foo can know all the variables that got passed in because this information is lost. The hash has been absorbed and is no longer part of the context! The variables in the locals hash that were set up by foo have been unpacked and established as the context of bar, but the containment itself is no longer available.

An argument might be made that making the locals hash available would be wrong; if the code knew it had the locals context, partial-coupling might be considered to be too tight. Perhaps so, but I'd say this really isn't true - assuming context in the form of injected variables is really at the same level of coupling. After all, what Rails is doing for me is just saving me some coding by allowing me to access a variable directly instead of as locals[:variable] if the incoming locals hash were exposed. The way I see it is that permitting a partial to access the incoming locals hash is actually promotes less coupling. A partial could just pass, augment and pass, or create and pass a new hash to the partials it calls - not necessarily knowing to what use the information in the hash might be put, or which partial down the line might be using it.

Chaining is about delegating responsibility - entities at intermediate levels really shouldn't have to be concerned with the details of what's happening above and below them. They should be able to pass information along with the understanding that if they don't need it, that doesn't mean that something further along the chain won't. If there's any doubt about this perspective, consider that partials are simply about rendering a page - the page is the real context and many complex renderings of application state changes may be made upon receipt of a seemingly simple event. That's what RJS is really good for - making multiple coordinated changes to a page. Allowing information to be passed down from on high so chained partials can use it according to need in the context of rendering different parts of a page is exactly why RJS is so sweet. Forcing intermediate partials to know what their subordinates need makes them difficult to write, hard to test, and downright painful to change.

Of course, just because this isn't automatic doesn't mean I can't do it myself. Passing foo a reference to the :locals hash as the value of the :local_assigns variable in the :locals hash it receives itself, for instance, allows me to pass it as the :locals hash to bar. Not a great solution, and it smells a little. Yes, it does mean I have to do some extra work, something Rails has helped me to otherwise avoid, but I can put up with this. Sigh. I sure wish the locals hash was already exposed - it is the missing link for chaining.

Bottom line: not having access to the incoming locals hash in a partial is a bummer, but it isn't the end of the world.

Why Can't We All Just Get Along?

While building a browser-based graphical editor in Rails for a client, I ran across an cross-technology compatibility problem. The reason was not immediately apparent and caused me quite a bit of head-scratching. After sleeping on it for a night, I finally figured it out and found a workaround.

In the graphical editing app that I'm building, one of the things a user does is create selections of items. When selecting, shift-clicking on an item toggles it in or out of a selection. I use SVG to represent the items being edited in the browser and manipulate them using Javascript.

Building composite graphical elements is made easier by using the SVG use element since this can encapsulate each part, setting up display and event handling for each in a compartmentalized way, and the code for representing the parts can be shared across items. An item may reference many use elements in its representation. These elements are then added to the document and displayed automagically in the browser. Javascript functions handle the events targeted at the items.

Everything's good so far. Simple SVG elements can be manipulated easily by the Javascript, and AJAX calls are sent to the controller to update the model. Shift-clicks toggle items in and out of the selection as expected. But shift-clicking an element that includes a use element in its definition (in my case, a path element is being referenced by another element) causes a new graphical editing window to be opened in addition to the toggling.

What the heck?! Debugging the Javascript in Firebug shows that all of the expected things are happening correctly and the code is running cleanly. Yet as soon as the code finishes, up pops a new window. I'm certainly not firing a request to do this. What's going on?

It turns out that Firefox implements shift-click-on-a-link functionality that will open a linked reference in a new tab or window. This is extremely useful - I do it all the time when I'm browsing. But it didn't occur to me that this behaviour transcends normal event handling! I need different behavior in my editor!

It turns out that the way that SVG support is built in, shift-clicking on an SVG element that's drawn with the use element is misinterpretted by Firefox as a request by the user to follow the link embedded in the use element. But this reference is meant to be invisible - SVG uses these reference to display graphics, not to expose clickable links.

This wouldn't be so bad if there were a way to tell the browser that the event has been handled and not to do the default shift-click-on-a-link functionality. Normally such things are done by halting the propagation of events. Alas, for the shift-click, this has no effect. It appears that it is not possible to stop this behavior through Javascript. There may be a way, but I couldn't find it.

It's not a bug in the browser. It's not a bug in SVG. It's a bug that emerges when the two technologies come together. If the browser didn't hide access to its shift-click behavior, all would be well. Or if SVG didn't use the html linkage mechanism to do element referencing, all would also be well. But shift-click is inescapable in the browser and the element referencing is as it is in SVG. The bug is in the space where they overlap.

The workaround is not to use the use element in SVG if such elements can be shift-clicked. This causes the size of scene representations to balloon when the items are complex, but at least they can respond appropriately to events. The fix is to make Firefox (and whatever other browsers exhibit this problem) understand the difference between internal and external referencing via linkage in SVG.

Emergent bugs like these are insidious. It is the clash between multiple context-free systems that occupy the same technological space that lead to such problems. I predict they will appear more frequently as we move forward and continue building embeddable software. The best we can do for now is to fix or workaround the problems as they are found as I have here.

The bottom line is that we must recognize that in the areas where context-free systems that are used together overlap there will be ambiguity. The composite behavior of these systems must be considered and dealt with appropriately, either through code or policy. It's not clear that we'll ever be able to find all of these overlaps prior to encountering the problems they cause; I just hope the scenarios in which the problems occur aren't too dangerous.

Users, Roles, Rights and Sights

Chad Fowler's Rails Recipes book lays out authorization as the interrogation of the many-to-many connections between users and roles, and between roles and rights, a right being a named controller-action pair. The many-to-many relationships are established using roles_users and rights_roles tables in the database.

This indirection makes bulk assignment of rights easy, simply by assigning roles to a user, and authorizing using nested detection:

class ApplicationController
  before_filter :check_authorization

def check_authorization
    unless @operator.roles.detect{|role|
      role.rights.detect{|right|
        right.action == action_name &&
        right.controller == self.class.controller_path } }
      flash[:notice] =
        "You are not authorized to view the requested page"
      request.env["HTTP_REFERER"] ?
        (redirect_to :back) : (redirect_to home_url)
    end
  end
end

Here, @operator is the user that is operating the application.

This works fine, but the interrogation of rights seems a little too removed from the user. I prefer asking if a user has a particular right directly:

class User
  def has_right?(controller,action)
    rights =
      User.find_by_sql [
        "select * FROM rights ri, rights_roles rr, roles_users ru where"+
        " ru.user_id = ? and ri.controller = ? and ri.action = ?"+
        " and rr.role_id = ru.role_id and ri.id = rr.right_id",
        id, controller, action ]
    rights.size > 0
  end
end

I also took the liberty of dropping the in-Ruby detection since a single sql query is faster than the multiple smaller queries that detection requires (look at the log files - one query for each role.)

So this is a bit faster and is a drop-in replacement for the detection in the original code:

class ApplicationController
  def check_authorization
    unless @operator.has_right?(self.class.controller_path,action_name)
      flash[:notice] =
        "You are not authorized to view the requested page"
      request.env["HTTP_REFERER"] ?
        (redirect_to :back) : (redirect_to home_url)
    end
  end
end

Besides simplifying the authorization, since has_right? is a method of a User, any user's rights can be simply interrogated, which useful in setting up rights administration for an application.

Formulating the right-checking in this way also leads to the notion of sight-checking, that is checking if a user has the right to see something. For instance, when building out a page if there is a question as to whether or not a user is allowed to see something, a sight can be established that allows it to be seen. The absence of a requested sight in the rights table for that user's roles implies that the user should not see the sight.

Sights don't necessarily depend on controller-actions; typically they're just checking to see that the user has a right with a particular name. The code to check sights is a simple as that of checking rights:

class User
  def has_sight?(name)
    sights =
      User.find_by_sql [
        "select * FROM rights ri, rights_roles rr, roles_users ru where"+
        " ru.user_id = ? and ri.name = ?"+
        " and rr.role_id = ru.role_id and ri.id = rr.right_id",
        id, name ]
    sights.size > 0
  end
end

Checking for sights looks for a Right's name, while checking for rights looks for a Right's controller and action. Moving the interrogation of rights to the user and adding the notion of sights allows pages to be constructed more simply based on what a particular user is allowed to do.

Enhancements to Railroad

A picture is worth a thousand words, and is certainly easier to talk about than reading a bunch of code. At least for me.

On a recent Rails project two dozen models and sixty associations were needed to drive the application. A medium sized effort. However, it's tough to talk users through the complexity when the need arises, even when you're just dealing with a small chunk. After drawing circles and arrows on whiteboards too many times, I decided to mechanize.

I'd used Railroad (a great rubygem written by Javier Smaldone) a few times before to document smaller projects. It loads models and controllers from a rails project and renders everything in 'dot' format that can be processed by graphviz, an open source graph rendering framework. I railroaded the app and quickly had diagrams in hand. Happiness all around.

But there were a few things I noticed as I talked around the pictures. I needed colors. And labels. And fewer circles and arrows. I decided to take a dive into the railroad code and add some features.

What I eventually ended up with was a considerable set of changes.

Changes to Allow Subgraphing

Often I didn't want to see the whole graph at once. What I wanted was the ability to include only the models I listed, or exclude a set of models from the whole. After implementing this, I decided I also wanted the ability to focus on a set of nodes, including them and any other directly-connected nodes.

	I added support to only include model classes in a diagram by name specifying -I class1[,classN] on the command line.
	I added support to exclude model classes from a diagram by name using -E class1[,classN].
	I added the capability to focus on a set of model classes. By specifying -F class1[,classN] on the command line, only these nodes, the nodes they connect to, and the associations between them are displayed. However, this is also subject to any additional model class exclusions (-E) or inclusions (-I) that are specified.

By using -F, -E and -I in combination, a logical subset of the model can be displayed fairly easily.

Model Node Content Display Changes

I found that I sometimes needed more or less information in a displayed node. Brief mode was already present (for just displaying the class name in a node) but I wanted a little more content control.

	I changed the display of magic fields in model nodes to be off by default. Before, you'd hide the magic fields by specifying --hide-magic on the command line. I added a --show-magic option to turn them on instead.
	I changed the display of association fields in the node to be off by default, shown only by specifying --show-assoc-fields on the command line, and label them with the name of the associated model.
	I added a -B option to display all nodes as brief except focused nodes (those specified with -F).

Using -B with -F turned out to be a great way to present information. I could see what was being focused upon in context, without extraneous detail.

Changes to Association Display

I found I needed to be able to show different associations differently. Conventional Rails associations were fine, left black and unlabeled. But polymorphic associations, many-to-many through relationships, and unconventionally named associations needed different labels and colors. By default, I decided to always display these and provide methods to hide them.

	Unconventional associations and their labels are blue; navy blue if only one side of the relationship is unconventional. The labels are hidden by specifying --hide-uaslab on the command line.
	Polymorphic associations and labels are red. The labels are hidden with --hide-paslab.
	Through associations and labels are dark green. The labels are hidden with --hide-taslab.
	For convenience, all association labels may be hidden with --hide-aslab.

Multiple Diagrams at Once

Once the rest was done and I'd used it all for a while, I decided that automatically generating a focused diagram for each model would save me a lot of time.

I added support to create a focused diagram for each model class in its own dot file using -O FILE. The output file name for each model is created as FILE.dot, and FILE may include directory separators.

A simple rake task completed the automation:

@railroad_command = Config::CONFIG["target_vendor"] == 'pc' ?
  'railroad.bat' : 'railroad'

task :graphs do
  FileUtils.mkdir_p 'graphs'
  `#{@railroad_command} -M -b -o graphs/_overview.dot`
  `#{@railroad_command} -M -B -O graphs/`
  FileUtils.cd 'graphs' do
    FileList['*.dot'].each do |f|
      `dot -Tpng #{f} -o #{f.gsub(/dot$/,"png")}`
    end
  end
end

Finally, I refactored the completed code, DRYing out the iterative changes I'd made.

What I now have is a nice, simple way to produce diagrams of the database and associations between tables for discussion and documentation. I submitted the patch to Javier and hopefully it will be integrated into the railroad trunk fairly soon. You can grab railroad-0.5.0 and the patch and play with it if you want by downloading it from rubyforge.

Running Fast and the Demise of Dynamic Scaffolding in Rails

Sometimes things just get away from you. You get distracted, turn your head, and then when you look back everything's changed.

"In this place it takes all the running you can do, to keep in the same place." - Red Queen to Alice in Lewis Carroll's Through the Looking Glass

Admittedly I haven't been on the Rails edge for a while. My day job keeps me in Java, and my own work has been more focused on interfacing with FXRuby than through a browser. When I did use Rails, I used my older version and got along just fine.

But over the last few weeks I decided to pull in the latest and greatest. A few minutes of Rails 2.1 Gem downloading, another minute to create an app framework, lay in a db design as a migration, create a controller and add the requisite scaffold call into the code, and give it a try...

Dynamic scaffolding is no longer part of Rails. It was removed because it was considered detrimental - since scaffolding is intended to be temporary, it should be able to be easily removed. If the guts of scaffolding are dynamic, only being instantiated at runtime, you can't replace the scaffolding incrementally. It's all or nothing, which is hardly agile.

Of course, in the upcoming third edition of Agile Development with Rails this all explained. In the old days (sic) of dynamic scaffolding, a change to the database table underlying an interface changed the interface. Though this is no longer the case, the book includes examples of how to change the code files manually. You add a few lines of code to a few different files and everything just works.

This is great when you're changing polished code, but a pain when you're prototyping. When you're building the initial cut of an app and data model, you want to play before you do any polishing. Work on the big picture first! Adding code to a couple of different files is a burden at this point. Especially since every character you have to type has a relatively large potential to be wrong and slow you down.

While I certainly don't expect to get dynamic scaffolding back (although I could use some alternatives - Streamlined or ActiveScaffold, for instance,) internally Rails does know how to produce the scaffolding code. It makes sense to be able to use Rails to generate the textual pieces of any new scaffolding so that they can be quickly cut and pasted into place.

I don't really care how it's done - output to a file, or the screen, or grafted to the end of the migration as a comment, or whatever - but it should be done by Rails. I really don't want any specialized capability for producing missing scaffolding in an editor macro or template. Editors should be general purpose, requiring little to no maintenance. Putting the intelligence into macros and templates would force them to have to track the next changes in Rails. Cut and paste seems an optimal strategy in the face of losing dynamic scaffolding.

Don't get me wrong. Removing dynamic scaffolding is clearly a good thing for all the reasons it was done. But in the spirit of writing less code, Rails should facilitate the development process by producing intended scaffolding on demand to be copied into place by the developer. This would make it quicker to "get things right" and keep any new scaffolding style-synchronized with the initially generated Rails scaffolding.

Leastways, it couldn't hurt.

A Little Easier EasyLogging

In need of easier access to thresholding, I added some more methods:

one to pull the current threshold level,

five to interrogate the current threshold with respect to a given level, and

five to set the current threshold level

as they relate to the default logger. These are available as mixed-in methods on the EasyLog Module and class methods of EasyLogger class.

Additionally, I added special support for writing empty log messages to aid in grouping related log entries.

I extended the earlier paper with a description of their use, and have added them to in the eymiha_util-1.0.2 rubygem in my Chunks of Ruby Infrastructure project on RubyForge.

Creating Alternate Ruby Objects

One of the things I've found slightly frustrating about Ruby is the new method. Nearly twenty years ago I was writing code in Objective-C on NeXT, which had explicit allocation and initialization:

   Object foo = [[Foo alloc] init]

Using this invocation, foo could be any sort of Object, and init could return any sort of Object. Ruby (and most other OO languages, for that matter) combine the alloc-init into a 'new' method:

   foo = Foo.new

The problem is that the object returned is a Foo, at least unless a class supplied its own specialized new.

In a short paper I've written, Creating Alternate Ruby Objects, I discuss a generalized mechanism that allows object remapping in new. This enables a class' new method to return a different object than the one allocated, according to decisions made in the initialize method.

The mechanism has been added to the release 1.0.4 of my eymiha rubygem in my Chunks of Ruby Infrastructure Project on RubyForge.

The Joy of Testing with EasyLog

In a previous entry I talked about my EasyLogging infrastructure. I'm even happier about it now. I ended up writing some tests and code over the weekend and found myself wanting to track values during my development iterations. With my log_debug statements in the code I simply needed to establish the logging context in the test case to see what my value flows looked like.

At the beginning of the test file, test class, or even interspersed in the test method itself I do a

  start_logging STDERR

when I want logging to begin. At that point any messages that are logged will be written while the test proceeds. When I'm no longer interested, I can do a

  finish_logging

to stop the logging output.

I started to use it consistently in my test cases, putting finish_logging in the teardown method, and then at the top of the test case logging to STDERR if I wanted output or to nil if I didn't. Coupled with the logging in the classes I'm writing (which are subjected to thresholding and directed to files in normal operation) this makes quick work of determining where things have gone awry when tests fail in subtle ways.

Fail in subtle ways? Isn't that an indicator that more tests need to be written? Yes, of course! But in the real world, not all the tests that should get written actually do get written, despite our good intentions. The logging is especially useful in these situations to help identify untested conditions. If I turn on logging output, and then look at the code and a few lines of logged values, I can figure out what's going on and whip together a test a lot more quickly than just wondering why the heck my code isn't doing what I though it should.

Another Ruby Inconsistency

Maybe it's just me, but I keep finding interpreter issues underlying my code. The latest has to do with accessing the file system. In the context of unit testing,

require 'test/unit'
require 'fileutils'

class TC_File_Access < Test::Unit::TestCase

  def setup
    rm_r "temp" if File.exist? "temp"
  end

  def test_non_exisitent_intermediate
    mkdir "temp"
    touch "temp/foo"
    assert_raise(Errno::ENOTDIR) { File.new "temp/foo/bar", "w" }
    rm_r "temp"
  end

end

runs fine on my Mac, bar cannot be created under temp/foo since it's not a directory. Running it under Windows also errors, but the raised error is Errno::ENOENT instead. Of course this plays havoc with tests until the context of the environment is added

class TC_File_Access

  def test_non_exisitent_intermediate
    mkdir "temp"
    touch "temp/foo"
    if Config::CONFIG["target_vendor"] == 'pc'
      assert_raise(Errno::ENOENT)  { File.new "temp/foo/bar", "w" }
    else
      assert_raise(Errno::ENOTDIR) { File.new "temp/foo/bar", "w" }
    end
    rm_r "temp"
  end

end

I hate having to add this sort of code, but the underlying OS is pushing these errors to Ruby and its doing the 'right' thing, pushing up what it gets. I guess my complaint is that I wish Ruby could hide these sorts of details and remap the error to a consistent (most correct) value. It'd be a little more code, but it's all about making life easier for the lowly developer, right?

Imperfect worlds are such a pain.

Easy Logging For Ruby

In the spirit of invisible infrastructure, I built some framework around the Ruby Logger class and made it more succinct, more ubiquitous, and easier to use. The results are discussed in Easy Logging For Ruby, a paper describing the motivation and code that allows a developer to add logging to Ruby code with virtually no effort.

The code is available Chunks of Ruby Infrastructure project as part of the eymiha_util-1.0.1 (or later) rubygem.

How Accurate Is Your Clock?

I bumped my head against another one. While Ruby is not equal in all environments, I at least want to control the difference wherever possible.

When you do a Time.now, you'll get something like

  Tue May 20 17:08:21 -400 2008

Lets say that you're formating that time for a log message though...

  Time.now.strftime "%m/%d/%y %H:%M:%S"

which gives

  05/20/08 17:08:21

Great, except that you'd like subsecond precision.

  (time = Time.now).strftime "%m/%d/%y %H:%M:%S." <<
     ("%06d" % time.usec)

Well, my iMac gives me six digits of microseconds,

  05/20/08 17:08:21.943286

but alas, my PC only three,

  05/20/08 17:08:21.943000

While you may say, "So what?" I must reply, "Yuck." I just don't want those empty zeros hanging out there. The purist in me wants to lop them off. What I want is

  (time = Time.now).strftime "%m/%d/%y %H:%M:%S." <<
     ("%06d" % time.usec)[0,@usecs]

where @usecs is the subsecond precision of the clock.

Now, I could just use the Config to get the platform I'm on and assign the correct value, but in this instance I'd like to be more proactive. I can figure out the right value empirically. I start by getting an Array of sample microseconds that are slightly spread out in time.

  t = (1..5).collect { sleep 0.001001; "#{Time.now.usec}" }

Then I figure out what digit contains the last non-zero digit

  nz = t.collect { |s| s.length - (/[1-9]/ =~ s.reverse) }

And finally, I just take the max

  @usecs = nz.max

Why the multiple samples? Because there's still a one in ten chance that a zero will occur naturally in the real non-zero digit position. Or one in one hundred for two zeros, or one in a thousand for three. By running multiple samples and taking the max, we won't be likely fooled, statistically speaking. Why five samples? I just figure that those odds are pretty darned good.

Of course, we can collapse this all nicely,

  class Time

    @@subsecond_precision = nil

    def self.subsecond_precision
      @@subsecond_precision ||=
        (1..5).collect {
          sleep 0.001001
          s = "#{Time.now.usec}"
          s.length - (/[1-9]/ =~ s.reverse)
        }.max
    end

  end

Now I can just use Time.subsecond_precision in place of @usecs above. I don't have to worry about using system-dependent assignments. I can just do it once when I need it and move forward.

In the Spirit of Brevity, class_attr Methods

Ruby is the language for coding brevity with elegance. The more you "get" Ruby, the less code you end up writing. One of the ways you do this is to just metacode the things you find yourself doing a lot and become more productive. And that's a beautiful thing!

Recently, I've found myself writing and rewriting class attribute accessors, like

class Foo

  def self.bar
    @@bar
  end

  def self.bar=(bar)
    @@bar = bar
  end

end

Being from the old school, while I'm not adverse to just using the @@ directly inside the class, I prefer some encapsulation. And when you want to expose the class' innards, you need to write these methods anyway. This if fine, but it gets a bit too verbose for me.

Instance attributes are exposed easily with the attr methods

class Mumble

  attr_accessor :barfle

end

which creates the barfle and barfle= instance methods. What I want is an analogous

class Foo

  class_attr_accessor :bar

end

to create my self.bar and self.bar= methods. Metaprogramming to the rescue!

class Module

  def class_attr_reader(*kattrs)
    kattrs.each { |kattr|
      ka = kattr.to_s
      reader = <<EOS
def #{ka}
  @@ka
end
EOS
      self.module_eval reader
    }
  end

  def class_attr_writer(*kattrs)
    kattrs.each { |kattr|
      ka = kattr.to_s
      writer = <<EOS
def self.#{ka}=(#{ka})
  @@#{ka} = #{ka}
end
EOS
      self.module_eval writer
    }
  end

  def class_attr_accessor(*kattrs)
    class_attr_reader(*kattrs)
    class_attr_writer(*kattrs)
  end

end

Sweetness. I just love Ruby!

Life in the Windows-Bizarro Universe

You know what my biggest coding-rant about Windows is? Drive designators. They just suck.

Back in the pre-PC days when Unix was the new kid on the block, the notion of path specifications became popular. It was a big change from Dataset declarations in JCL; directories as files was a concept that made sense. It caught on quickly and everyone wanted to do it the Unix way. (One of my first projects at Bell Labs in the early 80s was finishing a port of Unix's ed command in the mainframe environment - the directory-structure logic was as close to Unix as I could make it and made the mainframers quite happy.)

Then PCs came along. Rooted in CP/M, with many influences affecting its evolution, the paths became much like Unix's paths with three main differences: the filenames were limited to 8.3 format, the separators within a path were backslashes, and the a drive designator could be specified.

While modern windows has eliminated the 8.3 limitations and underlying filesystem libraries treat both the slash and backslash with ease, the drive designator lives on as a pain in the ass. I understand the pervasiveness of the feature; the allure of a simple top-level shortcut is strong. But it really has crufted up portability of software from one platform to another for the last thirty years.

Take, for example, the home directory. This is the directory that a user first "lands in" after logging into a system. From inside Ruby code, if you need to get that home directory, you can just

  ENV['HOME']

But on a PC, you have to

  "#{ENV['HOMEDRIVE']}#{ENV['HOMEPATH']}"

This may not seem too painful perhaps, but you need to jump through this hoop whenever you access the filesystem. I've been gradually building up a set of covers for this sort of behavior, such as

  class File
    def self.home_directory
      if Config::CONFIG['target_vendor'] == 'pc'
        "#{ENV['HOMEDRIVE'}#{ENV['HOMEPATH']}"
      else
        ENV['HOME']
      end
    end
  end

for getting the home directory, but there's still no good way to hide the problem once and for all. You just need to be aware of drive designators when you're building paths. It's a fact of life.

But it still sucks.

Opening a URL in a Viewer from Ruby

While web apps are all the rage, what with the browser being ubiquitous and everything, there are still some features that don't scale - usually stuff that requires a lot of high-bandwidth data getting manipulated and displayed. Sorry, but interactive data-heavy apps just don't yet run well enough on the web to make the jump. For other things, yes; but not for these.

Unfortunately, applications based on these sorts of features are the ones I've spent a lot of time writing. They want to be web-enabled, not web-based. It's one thing to use the web as a communications medium, but another to use it as an interaction medium.

So, I got Lyle Johnson's book (FXRuby: Create Lean and Mean GUIs with Ruby) I've been messing with FXRuby lately. Quite a nice little package. I'm moving some big-application stuff into it and should have more to say about it in the upcoming weeks and months. One of the parts I just moved in deals with opening html documents and URLs. A long time ago I decided to write all of my help documents in html and use the browser for displaying them. When the user clicks a help button in an app (or something like that) I need to bring up the requested page in a browser.

Opening a URL in a browser seems like it should be almost trivial, but from a Ruby app there is a little art to it... Different vendors have different mechanisms to do it, and you have to make it work right so the users of your apps don't get flustered. So I hide the vendor-specific logic under a general method:

def open url
  send "open_#{Config::CONFIG['target_vendor']}".to_sym, url
end

This dispatcher will call the open for the specific vendor. I make the separation here, so I can handle each vendor cleanly no matter what sort of crazy stuff they may be doing.

Some vendors make it easy. Take Apple, for instance:

def open_apple url
  system "open #{url}"
end

Apple's made opening a file in your viewer of choice a basic function of the OS. By hiding the details, it's less work and higher productivity for me.

In Windows it takes more effort. Because the viewer of choice is buried in the registry, I have to dig it out:

require 'win32/registry'

def windows_browser
  Win32::Registry::HKEY_CLASSES_ROOT.
       open('htmlfile\shell\open\command') { |reg|
            reg_type, reg_value = reg.read('')
              return reg_value
  }
end

Now I can do the open:

def open_pc url
  system "#{windows_browser} #{url}"
end

On other systems (say Linux, for instance) there's no direct solution. Because the viewer-of-choice is buried in desktop preferences that may be different for different window managers, there's no definitive way to know what it is and how to pull it out. So I opt for flexibility - I depend on an outsider to inject the name of the URL viewer into the mechanism prior to opening the url.

attr_accessor :url_viewer

def method_missing(method,*args)
  if method.to_s =~ /^open_/
    if @url_viewer
      system "#{@url_viewer} #{*args}"
    else
      raise DocumentException,
            "no URL Viewer was designated to open the URL."
    end
  end
end

This isn't enough, however. The Kernel's system method will block until the application opening the URL returns. I don't want the application to stop and wait! In order for the app to stay in control, the open has to run in it's own thread.

def open_document url
  Thread.new {
    send "open_#{Config::CONFIG['target_vendor']}".to_sym, url
  }
end

But there's a problem. I'd like to catch the raised exceptions from the dispatched opens if something unexpected happens - but since the exceptions come from a new thread that I'm not waiting for, they'll just go into the aether. I need to do this using another mechanism. So instead of raising, I'll hypothecate an exception handling mechanism that the thread can use to notify the app that there was a problem during the open.

attr_accessor :exception_handler

def open_document url
  Thread.new {
    begin
      send "open_#{Config::CONFIG['target_vendor']}".to_sym, url
    rescue Exception => exception
      @exception_handler.handle exception
    end
  }
end

and we'll let the caller pre-designate the exception handler. Though this isn't quite as nice as rescuing in the caller, I'm not as concerned since the limited set of things that can fail when opening a url really come down to configuration issues or the absence of the URL's target.

While this will open a URL in a viewer from a Ruby app, there's a little more work needed. I want to ensure the URL is properly-formed enough not to choke the viewer. I'll do this by normalizing before I do the open.

def open_document url
  Thread.new {
    begin
      send "open_#{Config::CONFIG['target_vendor']}".to_sym,
           normalize(url)
    rescue Exception => exception
      @exception_handler.handle exception
    end
  }
end

The normalizing is just a bit funky, but trivial in concept - just return a String containing the normalized URL. My top-level logic is: if it's a file resource, then normalize it as a file; otherwise, validate it as a URI. Since Ruby already comes with a URI class, I just use it.

@@using_pc_filesystem =
     Config::CONFIG['target_vendor'] == "pc"

def normalize(url)
  (file_url? url) ? nomalize_file(url) : URI.parse(url).to_s
end

def file_url?
  (url =~ /^file:/) or
  (url =~ /^\//) or
  ((url =~ /^[A-Za-z]:/) and @@using_pc_filesystem) or
  !(url =~ /:/)
end

It's a URL file resource if it starts with file:, a slash or a drive designator (on a pc) or it doesn't have a colon in it.

Normalizing a file amounts to giving back the normalized file name with file:// prepended to it.

def normalize_file file_url
  path = normalize_file_path(
       (file_url =~ /file:\/\//) ? $' : file_url)
  "file://#{path}"
end

def normalize_file_path file_url
  if absolute_file_path? file_url
    file_url
  elsif @relative_base != nil
    "#{relative_base}#{file_url}"
  else
    raise UrlException, "no relative file base was configured"
  end
end

A file path is absolute if it starts with a drive designator and it's on a PC, or a slash (with no drive designator) if it isn't.

def absolute_file_path? file_url
  @@using_pc_filesystem ? (file_url =~ /^[A-Za-z]:\//) :
      (!(file_url =~ /^[A-Za-z]:/) and (file_url =~ /^\//))
end

Finaly, a relative base is prepended to relative file paths. It fits between a drive designator and the relative path or a PC, or otherwise just sits at the front of the path. When I assign it, I make sure it looks like it'll work.

def relative_base=(relative_base)
  if valid_relative_base? relative_base
    @relative_base = relative_base
  else
    raise UrlException, "Invalid relative base '#{relative_base}'
  end
end

def valid_relative_base? relative_base
  ((@@using_pc_filesystem and (relative_base =~ /^[A-Za-z]:\//)) or
   (!@@using_pc_filesystem and (relative_base =~ /^\//))) and
  (relative_base =~ /\/$/)
end

This wraps everything up nicely. Nice enough that I wrapped it up into a gem I can pull into any of my apps. I added it to my cori project (Chunks Of Ruby Infrastructure) on rubyforge in the eymiha_url rubygem.

Having done this once, I can now get on with the meat of writing my interactive-but-data-heavy Ruby applications, waiting for enough bandwidth on the Internet to someday move them to the web.

Exploratory Surgery and Unit Testing

Test Driven Development is one of those milestones in your development career that once reached, seldom causes you to look back. When writing unit tests before you write the code to pass them becomes second nature, and re-running the unit tests on each change becomes part of the development process, quality simply mushrooms.

But the Church of TDD has several orthodoxies. One sect insists that only the public parts of a unit should be tested. Another says that everything needs to be tested, public or private. I'm a proud member of a third group, whose credo is expounded by Cedric Beust: When it comes down to testing, I follow a simple rule: "if it can break, test it." This pragmatic position seems perfect for the agile developer. Still, as much as possible, I try to keep my private methods lightweight and stick to testing the visible public methods, exercising them extensively enough to make sure the underlying private ones work.

However, I was recently putting together some code that served as a dispatcher: a public method would dispatch to one of a number of private methods depending on some set of conditions. Normally I'd just set up each condition and make sure the returned value was what was expected. But in this case, the dispatching was done inside a new Thread, and no reasonable surface value was returned to check. I needed to get under its skin and test the private methods.

Ruby is kind to testers. I'm a big fan of extending a class within the unit test code itself that will let me do a more complete job. Exposing an attribute or mocking a return value in the testing code is all part of the game, as long as I'm not changing the code I'm releasing. If I have to slip something into the code under test, then it's too dirty. It shouldn't have any inkling being tested, otherwise the system breaks down, governments collapse and fire starts raining from the sky. No thank you.

Instead, one needs to do exporatory operations with surgical instruments. Probe the body, touch a nerve, see what twitches. And for god's sake, don't change anything! In this sense, the private method is the nerve you're touching - what is needed is the probe that lets you get at it.

As everyone knows, testing a public method, say bar, is easy:

   def test_bar
     Foo foo = Foo.new
     assert_equal foo.bar, expected_bar,
                  "all is not well in Foo's bar."
   end

but if we have a private method, frapp, you need to get at it surgically. I opt for a variation in a mechanism proposed by Jay Fields that offers less exposure:

class Class
  def publicize_private_instance_method(method)
    needs_publicity =
         self.private_instance_methods.include? method.to_s
    public method if needs_publicity
    yield
    private method if needs_publicity
  end
end

This allows the simple test:

   def test_frapp
     Foo foo = Foo.new
     Foo.publicize_private_instance_method :frapp do
       assert_equal foo.frapp, expected_frapp,
                    "all is not well in Foo's frapp."
   end

By only exposing the one private method during the course of the test, I'm minimizing the possibility of any unintentional effects of its publicity, and thus am bit more confident than changing the character of the entire class and all of the classes from which it inherits.

I've found this to be a quite a nice surgical probe in my quest for better unit testing.

An Optional Require for Ruby

In the fine balance between code organization and keeping the system quiet, I like to err on the side of silence. I just don't like warnings and noise coming from the compiler and runtime, and will sometimes code around the squeaky parts of a system so I don't have to listen to the racket. Despite the messiness of my physical reality, I like to keep my virtual reality well-organized and clean. That goes for my code too - I don't like writing Goldbergian code unless it's for play. I like to put the right code in the right place.

I write most of my Java code in Eclipse these days and there are little noisy warnings that seem to pop up everywhere. While they can be supressed or filtered, I kind of hate it when I have to refactor correct code and move things around because of the judgement of a coding tool. Happily, I don't have to in Ruby. Ruby doesn't have a "compile time" and unit testing removes the stray vibrations from the code, so it's almost always nice and quiet. Plus, the concept of "where things go" is never tainted by file boundaries - code for any class can go into any file and Ruby sorts it all out.

Until yesterday.

I was writing some code to open html documents, keeping everything organized together so the functionality was all in place. The code was intended to open the html in the user's selected browser on any platform so I could call one high-level method and have it just work. To do this on Windows I needed the win32/registry feature. So I added it with a require and everything was fine. I was happy. Later I pulled the code onto my iMac and ran it. LoadError.

I was aghast. My Ruby was befouled. Of course, Apple has no need for Windows Registry access, and so was missing the feature I had required. Fine. I could install the gem anyway and move forward. But then I thought, this is code I want to use in production - customers would be using this and there is no need for them to load that gem if they were on Apple or Linux.

So I was faced with two evils. Pull in the gem or code around it. I decided to code around it. I then had to decide whether to break up the module and only include the windows part if needed, or keep everything together so the logical functionality was colocated. I decided to keep everything together. This now meant that I needed some conditions around my require.

require 'win32/registry' if
  Config::CONFIG["target_vendor"] == "pc"

That's fine - everything was back on track.

I did get to thinking that I really was working too hard, and that I'd end up repeating myself again somewhere down the road. I wanted something more general, that would give me a pass if a LoadError occurred, and handle the missing piece later somehow. I decided to create an optional require.

class Object
  def optional_require(feature)
    begin
      require feature
    rescue LoadError
    end
  end
end

Now if I used it and got a LoadError, everything would just move on.

optional_require 'win32/registry'

This removed the guts of the decision from the actual require, and expected the require to fail on the platforms that didn't need it. It also left the door open for writing a 'check_configuration' method that would make sure that the user's feature set was complete when an application starts, which to me is just good practice. I was happy enough with this solution to place it into the eymiha-0.1.3 rubygem, available from in my cori project (Chunks Of Ruby Infrastructure) on rubyforge.

So, less noise and more organization. Just what I was looking for.

The Units Pipe Dream...

Much of what's driven my off-off-hours development over the last year is the architecture, design and coding of a Units Framework for Ruby. While there are a few out there, this one is different - predicated on a NumericWithUnits, units are added to Numerics transparently. They act exactly like Numerics without units, but assert their types when appropriate and handle conversions automatically, such as:

puts 5.inches + 3.feet	->	41 inches
puts 55.miles_per_hour.feet_second	->	80.666666667 ft / s
puts seconds_per_week	->	604800

To use them you just include the framework and everything just works. Go grab them for free at my CORI (Chunks of Ruby Infrastructure) rubyforge page: http://rubyforge.org/projects/cori/.

I wrote about the development as I went along, and humble submit it for your reading pleasure at http://www.geocities.com/eymiha/papers/TheUnitsPipeDreamV1.html.

This is just the first part of the development, however - only scalar units. I am going to add formulaic conversions (like temperature) and data-driven conversions (like currency) in the next go-round. Coming soon.

Arrg...

Jump onto windows. Go into the irb.

irb(main):001:0> puts 0.00001
1.0e-005

Now, get on a mac. Go into the irb.

irb(main):001:0> puts 0.00001
1.0e-05

Three digit versus two digit defaults for exponents. I hate sounding like a hobgoblin, but I really want the same results on different platforms without jumping through extra hoops.

Arrg.