Laissez les bons temps rouler!

Starting on the day before Mardi Gras, I will be joining EdgeCase working in their Cincinnati office. To me, a much bigger party than they'll be having in the French Quarter!

This is a phenomenal opportunity, working alongside the genius of Jim Weirich and Scott Barron every day, and collocated with the Gaslight Software crew. (Let me tell you, this is a really smart room.)

Many thanks to Joe, Ken, Adam and Leon for taking the time to meet with me and make sure I'd fit there, and to everyone who's supporting me in this transition. To be sure, I'll miss the folks I've been working with, but I'm ready to get cracking on new stuff!

Prototype and Witch's Brew

When things work differently in one part of a package than they do in another - for no apparently good reason - programming indigestion may occur. When you beat your head against the code for too long trying to figure out why things are screwing up, big headaches are sure to come. The cause of my current indigestion and headache? Element.insert in prototype.

I have some javascript code that renders a string containing some HTML to be added to the bottom of the contents of a some target element. I do the necessary incantation,

Element.insert('target',htmlString)

Which quite promptly does nothing. WTF?

It turns out that the htmlString I'm sending either needs a javascript toElement method defined that will convert it into a DOM object or it must be a DOM object already. The first alternative is unpalatable - I just want to create html, not a tree of DOM cruft. The second is just as bad, unless I can get the work done for me. Fortunately, I can.

In the body of my document, I declare a special, invisible div,

<div id="_cauldron_" style="display:none;"/>

and I use it to magically transform my html string into an object that I can insert into the target,

Element.update("_cauldron_",htmlString)
Element.insert(target,$("_cauldron_").firstChild)

The cauldron is where the html must brew to make the magic happen.

You may be asking, "Why does the conversion happen differently in the update? Why doesn't the insert work the same way?" Good question. I have no good answer though. To my way of thinking, The conversion should happen exactly the same way - otherwise kludges like this are forced. Suffice to say that it's all a moving target and everything's always changing. I'm sure this one will get fixed in a future Prototype release (or maybe it is already, my version is not the latest) but it does get frustrating.

Though the indigestion lingers, at least the headache is a little better now.

Another Tuesday Morning Zen Moment

Wrangling an older dog and a new puppy has brought a certain form of enlightenment to me. I can express it best as a koan.

Student: Master, what is the sound of one dog barking?
Master: Nothing like the sound of two dogs barking.

The Joy of Testing with EasyLog

In a previous entry I talked about my EasyLogging infrastructure. I'm even happier about it now. I ended up writing some tests and code over the weekend and found myself wanting to track values during my development iterations. With my log_debug statements in the code I simply needed to establish the logging context in the test case to see what my value flows looked like.

At the beginning of the test file, test class, or even interspersed in the test method itself I do a

  start_logging STDERR

when I want logging to begin. At that point any messages that are logged will be written while the test proceeds. When I'm no longer interested, I can do a

  finish_logging

to stop the logging output.

I started to use it consistently in my test cases, putting finish_logging in the teardown method, and then at the top of the test case logging to STDERR if I wanted output or to nil if I didn't. Coupled with the logging in the classes I'm writing (which are subjected to thresholding and directed to files in normal operation) this makes quick work of determining where things have gone awry when tests fail in subtle ways.

Fail in subtle ways? Isn't that an indicator that more tests need to be written? Yes, of course! But in the real world, not all the tests that should get written actually do get written, despite our good intentions. The logging is especially useful in these situations to help identify untested conditions. If I turn on logging output, and then look at the code and a few lines of logged values, I can figure out what's going on and whip together a test a lot more quickly than just wondering why the heck my code isn't doing what I though it should.

Another Ruby Inconsistency

Maybe it's just me, but I keep finding interpreter issues underlying my code. The latest has to do with accessing the file system. In the context of unit testing,

require 'test/unit'
require 'fileutils'

class TC_File_Access < Test::Unit::TestCase

  def setup
    rm_r "temp" if File.exist? "temp"
  end

  def test_non_exisitent_intermediate
    mkdir "temp"
    touch "temp/foo"
    assert_raise(Errno::ENOTDIR) { File.new "temp/foo/bar", "w" }
    rm_r "temp"
  end

end

runs fine on my Mac, bar cannot be created under temp/foo since it's not a directory. Running it under Windows also errors, but the raised error is Errno::ENOENT instead. Of course this plays havoc with tests until the context of the environment is added

class TC_File_Access

  def test_non_exisitent_intermediate
    mkdir "temp"
    touch "temp/foo"
    if Config::CONFIG["target_vendor"] == 'pc'
      assert_raise(Errno::ENOENT)  { File.new "temp/foo/bar", "w" }
    else
      assert_raise(Errno::ENOTDIR) { File.new "temp/foo/bar", "w" }
    end
    rm_r "temp"
  end

end

I hate having to add this sort of code, but the underlying OS is pushing these errors to Ruby and its doing the 'right' thing, pushing up what it gets. I guess my complaint is that I wish Ruby could hide these sorts of details and remap the error to a consistent (most correct) value. It'd be a little more code, but it's all about making life easier for the lowly developer, right?

Imperfect worlds are such a pain.

How Accurate Is Your Clock?

I bumped my head against another one. While Ruby is not equal in all environments, I at least want to control the difference wherever possible.

When you do a Time.now, you'll get something like

  Tue May 20 17:08:21 -400 2008

Lets say that you're formating that time for a log message though...

  Time.now.strftime "%m/%d/%y %H:%M:%S"

which gives

  05/20/08 17:08:21

Great, except that you'd like subsecond precision.

  (time = Time.now).strftime "%m/%d/%y %H:%M:%S." <<
     ("%06d" % time.usec)

Well, my iMac gives me six digits of microseconds,

  05/20/08 17:08:21.943286

but alas, my PC only three,

  05/20/08 17:08:21.943000

While you may say, "So what?" I must reply, "Yuck." I just don't want those empty zeros hanging out there. The purist in me wants to lop them off. What I want is

  (time = Time.now).strftime "%m/%d/%y %H:%M:%S." <<
     ("%06d" % time.usec)[0,@usecs]

where @usecs is the subsecond precision of the clock.

Now, I could just use the Config to get the platform I'm on and assign the correct value, but in this instance I'd like to be more proactive. I can figure out the right value empirically. I start by getting an Array of sample microseconds that are slightly spread out in time.

  t = (1..5).collect { sleep 0.001001; "#{Time.now.usec}" }

Then I figure out what digit contains the last non-zero digit

  nz = t.collect { |s| s.length - (/[1-9]/ =~ s.reverse) }

And finally, I just take the max

  @usecs = nz.max

Why the multiple samples? Because there's still a one in ten chance that a zero will occur naturally in the real non-zero digit position. Or one in one hundred for two zeros, or one in a thousand for three. By running multiple samples and taking the max, we won't be likely fooled, statistically speaking. Why five samples? I just figure that those odds are pretty darned good.

Of course, we can collapse this all nicely,

  class Time

    @@subsecond_precision = nil

    def self.subsecond_precision
      @@subsecond_precision ||=
        (1..5).collect {
          sleep 0.001001
          s = "#{Time.now.usec}"
          s.length - (/[1-9]/ =~ s.reverse)
        }.max
    end

  end

Now I can just use Time.subsecond_precision in place of @usecs above. I don't have to worry about using system-dependent assignments. I can just do it once when I need it and move forward.

Life in the Windows-Bizarro Universe

You know what my biggest coding-rant about Windows is? Drive designators. They just suck.

Back in the pre-PC days when Unix was the new kid on the block, the notion of path specifications became popular. It was a big change from Dataset declarations in JCL; directories as files was a concept that made sense. It caught on quickly and everyone wanted to do it the Unix way. (One of my first projects at Bell Labs in the early 80s was finishing a port of Unix's ed command in the mainframe environment - the directory-structure logic was as close to Unix as I could make it and made the mainframers quite happy.)

Then PCs came along. Rooted in CP/M, with many influences affecting its evolution, the paths became much like Unix's paths with three main differences: the filenames were limited to 8.3 format, the separators within a path were backslashes, and the a drive designator could be specified.

While modern windows has eliminated the 8.3 limitations and underlying filesystem libraries treat both the slash and backslash with ease, the drive designator lives on as a pain in the ass. I understand the pervasiveness of the feature; the allure of a simple top-level shortcut is strong. But it really has crufted up portability of software from one platform to another for the last thirty years.

Take, for example, the home directory. This is the directory that a user first "lands in" after logging into a system. From inside Ruby code, if you need to get that home directory, you can just

  ENV['HOME']

But on a PC, you have to

  "#{ENV['HOMEDRIVE']}#{ENV['HOMEPATH']}"

This may not seem too painful perhaps, but you need to jump through this hoop whenever you access the filesystem. I've been gradually building up a set of covers for this sort of behavior, such as

  class File
    def self.home_directory
      if Config::CONFIG['target_vendor'] == 'pc'
        "#{ENV['HOMEDRIVE'}#{ENV['HOMEPATH']}"
      else
        ENV['HOME']
      end
    end
  end

for getting the home directory, but there's still no good way to hide the problem once and for all. You just need to be aware of drive designators when you're building paths. It's a fact of life.

But it still sucks.

Mulling and Pondering...

It's happening again.

I'm turning another corner in my software development life only to find myself revisiting some of the same issues I've resolved before. Whenever this happens I learn again that though my instincts were good, the solutions I came up with need to be cleansed of my habits from that time. Interestingly, what comes out of this process is something a little simpler, a little more robust, and a little more consistent. Crisper and cleaner. Which is good, I suppose.

I guess I'm just older and crankier now, and as I look over the things I've built I feel the need to streamline the whole mess and make it hook together better. I guess I just want a more holistic software world when things seem to be getting more disconnected in real life.

Why do things feel more disconnected? Tax day? The price of gas? Stiffness in the joints? Perhaps it's just because I'm older and crankier, and the little pleasures have paled a bit. Or maybe I need a warm sandy beach somewhere for a week or two. I just don't know. But I'm tired of disconnectedness.

So I find myself thinking about simplifying again - hiding details through progressive disclosure and creating seemingly invisible interfaces. But this time I'm thinking harder than I have before. I feel like I need to get this right once and for all. I need that layer of infrastructure that I can live with and not second guess again in seven years.

Damn.

Such is my life, as always.

Screaming...

About this time last year I got a Mac and have been absolutely loving it. I cut my teeth on Unix back in the 70s and to me, Apple got it completely right.

This morning I logged onto a Windows machine I hadn't touched in nearly a year.

While I use Windows to do development at work, I'm usually in Eclipse - well-insulated from the OS. However, this brief taste of what I left behind, actually getting into Windows itself and having to use it...

Clearly, a century ago Edvard Munch had anticipated the feeling of a Mac user having to occasionally return to a PC.

Arrg...

Jump onto windows. Go into the irb.

irb(main):001:0> puts 0.00001
1.0e-005

Now, get on a mac. Go into the irb.

irb(main):001:0> puts 0.00001
1.0e-05

Three digit versus two digit defaults for exponents. I hate sounding like a hobgoblin, but I really want the same results on different platforms without jumping through extra hoops.

Arrg.

Transcience and Tragedy

It saddens me deeply to be reminded of the transcience of life; now in the form imminent demise of my friend and business partner of seven years, Wendel R. Wendel, from Creutzfeldt-Jakob Disease.

The disease is a human version of mad-cow, killing brain cells with an infectious protein that catalyzes healthy proteins into copies of itself at an exponential rate. Because it's not water-soluble like the healthy form, it builds up and ends up disrupting cell functions. It's called a spongiform disease, since the brain ends up looking like a sponge in the microscope - it's as if millions of tiny pockets have been eaten away.

The tragedy is that a once-vital person is gone, replaced with a man suffering from dementia who will soon die. Wendel was an entrepreneur and businessman who pushed the ordinary aside and since the 1960's created geodesic and spaceframe structures renowned for their simplicity and elegance. He was someone who tried to make the world better by pushing the envelope of architectural design. Now he waits in an unknowing limbo while his friends and family grieve for a life about to be lost.

He lived his life to the fullest. I am sorry for us all to see him pass.

TechnoSmiles

Suddenly, from the depths of your mind, up bubbles a childhood memory... a snippet of a song lyric - just a few words and a fragment of a tune. An online search, a glance at few entries to get the name of the song, a buzz over to a music repository, and a double click. Thirty-five years melt away as you listen.

We live in the best of times: we still have the mysteries of the past, but now we have the means to solve them at our fingertips.

Life is good.

Reverie of an Aging Programmer

As I find myself writing on a Saturday afternoon with my 48th-an-a-half birthday approaching, having just started working at a new client this week, and with my wife off on a weekend sabbatical, I seem to be in a reflective mood.

I've spent a lot of time doing software development.

I've been doing it so long that it's like music to me now. I've written several software symphonies, many programming concertos, and countless sketches of codal counterpoints. Sometimes it has swept me up, the harmonies lifting me high above the methods and objects I'm creating, letting me see the beauty of the whole systems, how everything works together. Other times I've been dragged down into the depths, frustrated with the limitations of the classes I've designed, struggling to overcome the dissonance that threatens to tear a program apart. I've always wanted so badly to find the lost chord and the music of the spheres, and hear within them all the things yet to come.

A few years back, Dave Thomas gave a lecture at a conference I attended in which he equated software development with the work of other artisans, such as painters, sculptors and writers - as opposed to the more methodical approach taken by engineers that those not involved in programming often associate with what we do. When I look back I see the truth in this perspective. But it's more than that. No artist truly creates total beauty. No artist ever creates the end-all, be-all composition. There is always the next painting, sculpture or novel.

I've grown older. The ambitious dreams of my youth have been subjected to the realities of life with all the foibles and foolishness that have taken up so much of my time from then to now. The joys of family and friends have tempered my desire to singlemindedly do impossible things. The dreams are still there, but I know that I will not be able to grasp the stars that as a child I thought were only just beyond my reach. But even though the sorrow of this realization is profound, I am less troubled by it than I thought I would be.

Nothing is singular. All things stand in relation to others. It is only through the totality of each others' accomplishments that efforts can be appreciated. We all achieve and add to the richness of each others' efforts. When I was young, my software dreams were for me. Now I write software for more than just myself.

No person can do everything they want to do. There is simply not enough time. Sometimes it is difficult enough to just keep up with all that goes on. Even when things come easily, there is just so much more to know, so much that there is no time to experience. There is always much more left to do - much more than anyone could ever hope to achieve despite their youthful dreams. And there is no finality to it. In the ocean of human achievement, we each only get to play in the waves for a short time. I've written a lot of software, but it is only a drop in that vastness.

I want to do more than I have time left to do it in. This is my continuing frustration. I wish it were different. But the dreams that once dominated my thoughts are now just hopes. Hopes that perhaps others will someday achieve some of what I had envisioned but that I will not be around to see.

My best hope now is that what I've done and what I will still get to do before I'm gone will help others more than hinder them.

Teeny Weeny Little Pains

While writing on some Ruby code, I hit a snag. I'm developing some infrastructure for engineering programming and was curious about values not coming out exactly right.

Having a few minutes, I got into irb and entered

(1..100).select{ |i| 10**-i - 10.0**-i != 0.0 }.join " "

Imagine my surprise when I got back

21 23 24 25 26 28 29 30 32 34 39 42 45 49 50 54 56 60 63 66 72 75 81 82 84 86 88 91 97

that's 29 numbers in the first 100 that have roundoff errors when taking the difference between integers and floats raised to high negative powers. Or perhaps this isn't curious - perhaps what is curious is that 71 numbers in the first 100 have no roundoff error.

Is this a Ruby bug? Well, maybe. The numbers are very small and probably won't cause anyone to lose too much sleep.

I'm certainly not waving hands or pointing fingers. I was just surprised. It just reinforces the skeptic in me that says nothing is ever finished and things may not always do what you expect.

Program with abandon, but check your work.

Geodesic Dome Sighting

Daniel Ellingsen of Plug In ICA worked with his organization and produced "Wildflowers of Manitoba" using TekCAD, my 3D CAD system. The exhibit features a partially-covered 2-frequency icosahedral geodesic built with steel pipe and carriage bolts. The art piece is currently on a short tour in Canada at the Montreal Biennale this May and June at the Parisian Laundry Gallery.

"Impressive creation presented at the 5th Biennial one of Montreal: a geodesic dome, created by Noam Gonick and Luis Jacob, returning to Expo 67 and the music of Harmonium." - La Presse   5/10/2007

Closer to Yottabytes!

In "Digital Data will Increase Sixfold by 2010" (http://www.channelregister.co.uk/2007/03/08/digital_data_explosion), O'Brien claims that it is suggested 988 Billion Megabytes - almost one Zettabyte - will be in play by the end of the decade. Of course, much of this data won't be unique, but that's still an awful lot of ones and zeros.

I'm wondering how long it'll be before we have a million times the 2010 amount - a thousand Yottabytes. Yotta is currently as far as the prefix numbering system officially goes...

Jet Shoes, Flying Cars, and Meals-in-a-Pill Dept.

By now why aren't we all living in weatherproofed domes so it never happens that a neighbor's huge pine tree gets coated with ice in a big winter storm and a branch rips off and lands on your electric and cable wires that extend across the street and knocks out your power for three days or more and you have no light or heat or connectivity and have to find somewhere else to sleep so you don't wake up looking like a member of the blue man group in the morning, hmm?

More People Everyday...

Wow. When I was back in grade school, the population statistics I remember were that the US population was 250 million, and the world population was 4 billion. Big numbers, and I thought, if I could just get everyone to send me a dollar...

Ah, the dreams of youth.

A quick trip to the population clocks at http://www.census.gov/main/www/popclock.html took me by surprise. The US population has just recently climbed over 300 million and the world population is up to over 6.5 billion. Simply amazing. 20% and 60% growth respectively.

I suppose that it's good to keep track of this stuff, if just for the sake of scale. There really are quite a lot of people out there.

Growing Pains and Browsers

Why can't the browser writers make html and the DOM behave according to a standard? By now you'd expect standardization, but no, they apparently still don't get it. Just this morning I've butted my head against more three differences between IE and Firefox!

1 - Table Cells. IE formats table cells painfully. After an input text field, a break pushes a large vertical space while Firefox doesn't. To try to solve this, if you instead put the broken text in the following row, in IE the row height isn't based on size of the text in the cells while it is in Firefox.

2 - Page caching, Ajax and Radio buttons. If you update a text field on a page using Ajax in Firefox and reload it, and if the page contains radio buttons, their state is switched - you can get it right by doing a shift-reload. IE doesn't mangle the radio button state.

3 - Input text fields and focus events. In Firefox, entering a return key will cause a focus change, while in IE it doesn't.

Some of this can be solved through extra coding, while some can't. Perhaps I ought to be less critical, but it just seems like things should be a lot cleaner and just work consistently, independent of the browser choice. It just seems to me that it shouldn't be that hard to get right.

It begs the question of whether such a behavior standard even exists. Maybe it's time for the web development industry to form a browser certification group to create and push such a standard, and evaluate different browsers. Even if it had no real clout, it'd be nice to have a list of inconsistencies for each browser, and possibly a recognized set of workarounds to get the rest of the way.

I hate whining, but I can't be the only one feeling this pain.

Domain Orientation and Rails

Over the last two years I've switched my development target to web apps. Though it started earlier than this, while I was still building TekCAD I was purposefully writing Java and Swing to earn my rent. The transition to full-time J2EE work that I made was good, but when I started doing Ruby and Rails my application-building days got much better.

Ruby and Rails are just vehicles, however. While they make it easy to put together new web apps, the opinionated style of Rails does not always jibe with what I've learned through experience about what end users expect. Users are domain experts. Given their expertise, They certainly don't look at problems the same way as application developers do. They want their complex understanding of a domain distilled into clever interfaces that save them as much effort as possible while still maintaining enough safety to keep them from making horrific mistakes. It's often quite tricky to develop these sorts of domain-expert applications, but that's what these users want. The raw-Rails philosophy enforces a certain perspective, focussing on exposing objects and enforcing rules and validation - however without considerable domain-bias worked into the application, the result can feel relatively flat.

This should be expected. Rails is about building applications, not understanding domains. The two problems are orthogonal. Web applications that are domain-oriented can be realized in any web framework. Rails just makes building them easier for developers.

As I've worked forward through these last two years, learning Rails with an emphasis on developing good domain-oriented applications, I find myself writing smarter and smarter controllers and depending more heavily on state stored in the session. This is somewhat contrary to the stateless mode preferred on the web, but I find it is typical for domain sensitivity. And surprisingly, I'm discovering some good new stuff.

What's good is this: the switch to Rails has given me the opportunity to re-examine domain orientation. In the past I've had to work so hard just to get everything finished for everyone - what with slashed budgets, creeping features and accelerated schedules - that my consideration of domains as a thing unto itself has been myopic at best. It's hard to be philosophic when you don't have enough time for it. Only rarely have I had the chance to step back and look at the meta-level of domains with respect to application development. But Rails has given me more time. Now I am refactoring my thinking as I refactor my code. I'm beginning to see shapes coming into focus: fundamental principles are being revealed about what it means to build software enriched with domain specificity.

Geekhood showing, I am salivating at the thought of capturing this knowledge. More is certainly to come...