Prototype and Witch's Brew

When things work differently in one part of a package than they do in another - for no apparently good reason - programming indigestion may occur. When you beat your head against the code for too long trying to figure out why things are screwing up, big headaches are sure to come. The cause of my current indigestion and headache? Element.insert in prototype.

I have some javascript code that renders a string containing some HTML to be added to the bottom of the contents of a some target element. I do the necessary incantation,

Element.insert('target',htmlString)

Which quite promptly does nothing. WTF?

It turns out that the htmlString I'm sending either needs a javascript toElement method defined that will convert it into a DOM object or it must be a DOM object already. The first alternative is unpalatable - I just want to create html, not a tree of DOM cruft. The second is just as bad, unless I can get the work done for me. Fortunately, I can.

In the body of my document, I declare a special, invisible div,

<div id="_cauldron_" style="display:none;"/>

and I use it to magically transform my html string into an object that I can insert into the target,

Element.update("_cauldron_",htmlString)
Element.insert(target,$("_cauldron_").firstChild)

The cauldron is where the html must brew to make the magic happen.

You may be asking, "Why does the conversion happen differently in the update? Why doesn't the insert work the same way?" Good question. I have no good answer though. To my way of thinking, The conversion should happen exactly the same way - otherwise kludges like this are forced. Suffice to say that it's all a moving target and everything's always changing. I'm sure this one will get fixed in a future Prototype release (or maybe it is already, my version is not the latest) but it does get frustrating.

Though the indigestion lingers, at least the headache is a little better now.

Why Can't We All Just Get Along?

While building a browser-based graphical editor in Rails for a client, I ran across an cross-technology compatibility problem. The reason was not immediately apparent and caused me quite a bit of head-scratching. After sleeping on it for a night, I finally figured it out and found a workaround.

In the graphical editing app that I'm building, one of the things a user does is create selections of items. When selecting, shift-clicking on an item toggles it in or out of a selection. I use SVG to represent the items being edited in the browser and manipulate them using Javascript.

Building composite graphical elements is made easier by using the SVG use element since this can encapsulate each part, setting up display and event handling for each in a compartmentalized way, and the code for representing the parts can be shared across items. An item may reference many use elements in its representation. These elements are then added to the document and displayed automagically in the browser. Javascript functions handle the events targeted at the items.

Everything's good so far. Simple SVG elements can be manipulated easily by the Javascript, and AJAX calls are sent to the controller to update the model. Shift-clicks toggle items in and out of the selection as expected. But shift-clicking an element that includes a use element in its definition (in my case, a path element is being referenced by another element) causes a new graphical editing window to be opened in addition to the toggling.

What the heck?! Debugging the Javascript in Firebug shows that all of the expected things are happening correctly and the code is running cleanly. Yet as soon as the code finishes, up pops a new window. I'm certainly not firing a request to do this. What's going on?

It turns out that Firefox implements shift-click-on-a-link functionality that will open a linked reference in a new tab or window. This is extremely useful - I do it all the time when I'm browsing. But it didn't occur to me that this behaviour transcends normal event handling! I need different behavior in my editor!

It turns out that the way that SVG support is built in, shift-clicking on an SVG element that's drawn with the use element is misinterpretted by Firefox as a request by the user to follow the link embedded in the use element. But this reference is meant to be invisible - SVG uses these reference to display graphics, not to expose clickable links.

This wouldn't be so bad if there were a way to tell the browser that the event has been handled and not to do the default shift-click-on-a-link functionality. Normally such things are done by halting the propagation of events. Alas, for the shift-click, this has no effect. It appears that it is not possible to stop this behavior through Javascript. There may be a way, but I couldn't find it.

It's not a bug in the browser. It's not a bug in SVG. It's a bug that emerges when the two technologies come together. If the browser didn't hide access to its shift-click behavior, all would be well. Or if SVG didn't use the html linkage mechanism to do element referencing, all would also be well. But shift-click is inescapable in the browser and the element referencing is as it is in SVG. The bug is in the space where they overlap.

The workaround is not to use the use element in SVG if such elements can be shift-clicked. This causes the size of scene representations to balloon when the items are complex, but at least they can respond appropriately to events. The fix is to make Firefox (and whatever other browsers exhibit this problem) understand the difference between internal and external referencing via linkage in SVG.

Emergent bugs like these are insidious. It is the clash between multiple context-free systems that occupy the same technological space that lead to such problems. I predict they will appear more frequently as we move forward and continue building embeddable software. The best we can do for now is to fix or workaround the problems as they are found as I have here.

The bottom line is that we must recognize that in the areas where context-free systems that are used together overlap there will be ambiguity. The composite behavior of these systems must be considered and dealt with appropriately, either through code or policy. It's not clear that we'll ever be able to find all of these overlaps prior to encountering the problems they cause; I just hope the scenarios in which the problems occur aren't too dangerous.

Another Ruby Inconsistency

Maybe it's just me, but I keep finding interpreter issues underlying my code. The latest has to do with accessing the file system. In the context of unit testing,

require 'test/unit'
require 'fileutils'

class TC_File_Access < Test::Unit::TestCase

  def setup
    rm_r "temp" if File.exist? "temp"
  end

  def test_non_exisitent_intermediate
    mkdir "temp"
    touch "temp/foo"
    assert_raise(Errno::ENOTDIR) { File.new "temp/foo/bar", "w" }
    rm_r "temp"
  end

end

runs fine on my Mac, bar cannot be created under temp/foo since it's not a directory. Running it under Windows also errors, but the raised error is Errno::ENOENT instead. Of course this plays havoc with tests until the context of the environment is added

class TC_File_Access

  def test_non_exisitent_intermediate
    mkdir "temp"
    touch "temp/foo"
    if Config::CONFIG["target_vendor"] == 'pc'
      assert_raise(Errno::ENOENT)  { File.new "temp/foo/bar", "w" }
    else
      assert_raise(Errno::ENOTDIR) { File.new "temp/foo/bar", "w" }
    end
    rm_r "temp"
  end

end

I hate having to add this sort of code, but the underlying OS is pushing these errors to Ruby and its doing the 'right' thing, pushing up what it gets. I guess my complaint is that I wish Ruby could hide these sorts of details and remap the error to a consistent (most correct) value. It'd be a little more code, but it's all about making life easier for the lowly developer, right?

Imperfect worlds are such a pain.

How Accurate Is Your Clock?

I bumped my head against another one. While Ruby is not equal in all environments, I at least want to control the difference wherever possible.

When you do a Time.now, you'll get something like

  Tue May 20 17:08:21 -400 2008

Lets say that you're formating that time for a log message though...

  Time.now.strftime "%m/%d/%y %H:%M:%S"

which gives

  05/20/08 17:08:21

Great, except that you'd like subsecond precision.

  (time = Time.now).strftime "%m/%d/%y %H:%M:%S." <<
     ("%06d" % time.usec)

Well, my iMac gives me six digits of microseconds,

  05/20/08 17:08:21.943286

but alas, my PC only three,

  05/20/08 17:08:21.943000

While you may say, "So what?" I must reply, "Yuck." I just don't want those empty zeros hanging out there. The purist in me wants to lop them off. What I want is

  (time = Time.now).strftime "%m/%d/%y %H:%M:%S." <<
     ("%06d" % time.usec)[0,@usecs]

where @usecs is the subsecond precision of the clock.

Now, I could just use the Config to get the platform I'm on and assign the correct value, but in this instance I'd like to be more proactive. I can figure out the right value empirically. I start by getting an Array of sample microseconds that are slightly spread out in time.

  t = (1..5).collect { sleep 0.001001; "#{Time.now.usec}" }

Then I figure out what digit contains the last non-zero digit

  nz = t.collect { |s| s.length - (/[1-9]/ =~ s.reverse) }

And finally, I just take the max

  @usecs = nz.max

Why the multiple samples? Because there's still a one in ten chance that a zero will occur naturally in the real non-zero digit position. Or one in one hundred for two zeros, or one in a thousand for three. By running multiple samples and taking the max, we won't be likely fooled, statistically speaking. Why five samples? I just figure that those odds are pretty darned good.

Of course, we can collapse this all nicely,

  class Time

    @@subsecond_precision = nil

    def self.subsecond_precision
      @@subsecond_precision ||=
        (1..5).collect {
          sleep 0.001001
          s = "#{Time.now.usec}"
          s.length - (/[1-9]/ =~ s.reverse)
        }.max
    end

  end

Now I can just use Time.subsecond_precision in place of @usecs above. I don't have to worry about using system-dependent assignments. I can just do it once when I need it and move forward.

Life in the Windows-Bizarro Universe

You know what my biggest coding-rant about Windows is? Drive designators. They just suck.

Back in the pre-PC days when Unix was the new kid on the block, the notion of path specifications became popular. It was a big change from Dataset declarations in JCL; directories as files was a concept that made sense. It caught on quickly and everyone wanted to do it the Unix way. (One of my first projects at Bell Labs in the early 80s was finishing a port of Unix's ed command in the mainframe environment - the directory-structure logic was as close to Unix as I could make it and made the mainframers quite happy.)

Then PCs came along. Rooted in CP/M, with many influences affecting its evolution, the paths became much like Unix's paths with three main differences: the filenames were limited to 8.3 format, the separators within a path were backslashes, and the a drive designator could be specified.

While modern windows has eliminated the 8.3 limitations and underlying filesystem libraries treat both the slash and backslash with ease, the drive designator lives on as a pain in the ass. I understand the pervasiveness of the feature; the allure of a simple top-level shortcut is strong. But it really has crufted up portability of software from one platform to another for the last thirty years.

Take, for example, the home directory. This is the directory that a user first "lands in" after logging into a system. From inside Ruby code, if you need to get that home directory, you can just

  ENV['HOME']

But on a PC, you have to

  "#{ENV['HOMEDRIVE']}#{ENV['HOMEPATH']}"

This may not seem too painful perhaps, but you need to jump through this hoop whenever you access the filesystem. I've been gradually building up a set of covers for this sort of behavior, such as

  class File
    def self.home_directory
      if Config::CONFIG['target_vendor'] == 'pc'
        "#{ENV['HOMEDRIVE'}#{ENV['HOMEPATH']}"
      else
        ENV['HOME']
      end
    end
  end

for getting the home directory, but there's still no good way to hide the problem once and for all. You just need to be aware of drive designators when you're building paths. It's a fact of life.

But it still sucks.

Screaming...

About this time last year I got a Mac and have been absolutely loving it. I cut my teeth on Unix back in the 70s and to me, Apple got it completely right.

This morning I logged onto a Windows machine I hadn't touched in nearly a year.

While I use Windows to do development at work, I'm usually in Eclipse - well-insulated from the OS. However, this brief taste of what I left behind, actually getting into Windows itself and having to use it...

Clearly, a century ago Edvard Munch had anticipated the feeling of a Mac user having to occasionally return to a PC.

Arrg...

Jump onto windows. Go into the irb.

irb(main):001:0> puts 0.00001
1.0e-005

Now, get on a mac. Go into the irb.

irb(main):001:0> puts 0.00001
1.0e-05

Three digit versus two digit defaults for exponents. I hate sounding like a hobgoblin, but I really want the same results on different platforms without jumping through extra hoops.

Arrg.

Teeny Weeny Little Pains

While writing on some Ruby code, I hit a snag. I'm developing some infrastructure for engineering programming and was curious about values not coming out exactly right.

Having a few minutes, I got into irb and entered

(1..100).select{ |i| 10**-i - 10.0**-i != 0.0 }.join " "

Imagine my surprise when I got back

21 23 24 25 26 28 29 30 32 34 39 42 45 49 50 54 56 60 63 66 72 75 81 82 84 86 88 91 97

that's 29 numbers in the first 100 that have roundoff errors when taking the difference between integers and floats raised to high negative powers. Or perhaps this isn't curious - perhaps what is curious is that 71 numbers in the first 100 have no roundoff error.

Is this a Ruby bug? Well, maybe. The numbers are very small and probably won't cause anyone to lose too much sleep.

I'm certainly not waving hands or pointing fingers. I was just surprised. It just reinforces the skeptic in me that says nothing is ever finished and things may not always do what you expect.

Program with abandon, but check your work.

Jet Shoes, Flying Cars, and Meals-in-a-Pill Dept.

By now why aren't we all living in weatherproofed domes so it never happens that a neighbor's huge pine tree gets coated with ice in a big winter storm and a branch rips off and lands on your electric and cable wires that extend across the street and knocks out your power for three days or more and you have no light or heat or connectivity and have to find somewhere else to sleep so you don't wake up looking like a member of the blue man group in the morning, hmm?

Living through Worry

This is my twentieth consecutive day on the bench. While there, I'm writing a kick-ass time-tracking system on rails and creating proposals. However, I worry about my employment future.

My wife is finishing her second week of work in the current year as an elementary school librarian. But she is not able to be a librarian because she has to teach 95% of her time, nearly 1000 kids per week. I worry about the effects of the stress that's building on her.

My mom, sister and her kids are the victims of a Justice system through which they can no longer be protected. Security has been removed that was holding up against a a dangerous situation. I worry about their safety.

And of course, these are just what's in my head this morning. There're all the other worrisome things going on in the world I'm not even trying to consider...

Step back. Relax. Mental and physical deep breaths. The universe stretches infinitely in space and time. Enjoy being in it and think in the moment. Life may be full of things that will make you worry, but you have gotten this far. Don't worry so much. You'll get through it. Strive forward.

And just keep in mind that it's harder for the bastards to aim, pull the trigger and hit you if you keep moving.

Bad Magic

One of my companies uses an ISP a few time zones away. I created all of the pages for our site, while the business guys set up the e-commerce portion of the site, all done about three or four years ago. It was a good value at the time. But this weekend the ISP had a massive crash that supposedly corrupted several of their servers. Of course, it hit our site and took it offline.

Besides being down for a few days now, our e-commerce data was wiped. You'd think that the ISP would have a failover and backup and would be working hard to restore everything. Not so. I spent a few hours uploading the pages and putting things back into place, but the e-commerce area was not something I had worked with... the ISP didn't provide access to it. Hell, it took me a day to get them to even put a placeholder up so I could start repairing things.

So now the business guys are scambling, trying to recover the rest of the site. I'm away for the weekend to go see my son graduate from high school, so I can't help. They're on their own. The fact that the business guys see this all as magic scares the crap out of me. They just hadn't ever really considered what was going on behind the curtain. Nobody's fault... the magic just went bad.

It just galls me that in an age of technology, disaster recovery has been so ill-conceived. Lesson learned: don't let those that are mystified by technology make important technology choices. I'd laugh, but it's all just too sad.

Fixing a Communication-impaired Windows Box

Yahhh. No Rails spoken here. WEBrick runs, but the browser can't get a response. And it just hangs there. No logs. And you can't kill it. And you look everywhere on the system for some record of what's happenning. And you scour the net, searching for a tidbit of information that could lead you to the truth, a fix and salvation...

What would keep WEBrick and a browser from talking? All the host stuff is ok, but packets aren't happening. The browser has no trouble similar touble getting to WSAD. Telnet can sense something, because going to the localhost and port redirects to telnet's own port 23. It has to be something burried down in the layers of the machine's communication mechanism. Hmmmm...

Aha! windowsxp.mvps.org/winsock.htm has the scoop. This machine I'm on is old, has likely been exposed to a nightmarish collection of users and uses, and some fairly icky viruses along the way. But since I'm running XP-SP1, I can't reset the winsock setting directly. But happily, www.snapfiles.com/get/winsockxpfix.html had a little executable to fix the problem. Download. Run. Reboot.

Rails again. Happiness and sighs.

Many thanks to Scott Tabar, for finiding the light at the end of this dark tunnel.