So for a recent project I whipped together the following JavaScript snippet (for jQuery, of course).  The snippet does the following on page load

• All text fields and text areas assume a value equal to their field names
• All select boxes take the value of their first non-empty options
• All radio buttons get the last option checked
• All check boxes are checked

Additionally, radio buttons, select fields, and check boxes each have appended a span tag bearing their field name.

;(function($) {
  $(function() {
    // text field values = text field name
    $.each($("input[type='text']").add("textarea"), function(i, input) {
      var field = $(input);
      field.val(field.attr('name'));
    });
 
    // select field value = first option with non-empty value
    $.each($('select'), function(i, select) {
      var field = $(select);
      $.each(select.options, function(i, option) {
        if (!select.selectedIndex && option.value != '')
          select.selectedIndex = 1;
      });
      field.after("<span>"+field.attr('name')+"</span>");
    });
 
    // checkboxes = checked
    $.each($("input[type='checkbox']"), function(i, checkbox) {
      var field = $(checkbox);
      checkbox.checked = true;
      field.after("<span>"+field.attr('name')+"</span>");
    });
 
    // radio buttons = last checked
    $.each($("input[type='radio']"), function(i, radio) {
      var field = $(radio);
      radio.checked = true;
      field.after("<span>"+field.attr('name')+"</span>");
    });
  });
})(jQuery);

If you use this snippet in your project, there’s no need to credit me with it, but do leave a comment on the blog to let me know what you think.

There don’t seem to be many hosts out there that support Java Web application hosting. Besides, I really like MediaTemple – their pricing is dead-on, their support is top-notch, and their attention to aesthetics puts them head-and-shoulders above every other Web host I’ve ever used.

But they don’t support Java. That is to say, their “(dv)” service line does not have Java installed by default. The Tomcat button in their Plesk control panel is nothing but a tease!

Fortunately, with full root access, we can add Tomcat to any old (dv).

Step 1: Enable root access

Start this process by enabling root access through your Account Center control panel – this is just easier than using sudo.  (If you don’t know what sudo is, this article is probably not for you.)

Step 2: Download Java

First, download the latest Java JDK RPM bin file to your local computer.  You can find this download on the Java developers Web site.  All we need here is the JDK – no Netbeans, no Java EE.  Make sure to download the file ending in .bin, otherwise you’ll have to use a package manager to install it (and I don’t know how to do that).

Since I’ll be using this Tomcat instance exclusively for one of my sites, I install Tomcat there.  You can install Tomcat anywhere you want, so long as you remember where you parked the Web server.

I like to keep my applications and their dependencies in close quarters, so I’m going to put my Java installation at /var/www/vhosts/collegeman.net/subdomains/myapplication/dependencies.  Upload your .bin file here with any old FTP program.

Step 3: Run the Java installation

Use an SSH client (I like Putty on Windows, and the native SSH client on the Mac) to jump onto your (dv). Switch to the dependencies path you created, and the change the permissions on the .bin file with the following command

chmod a+x *.bin

Now you can run the Java installation by typing the following command

./your-installation-file.bin

You’ll have to hold down the enter key for a while to scroll through the Java license agreement.  At the end you’ll be asked to agree to the terms: type ‘yes’ and press enter.  Assuming the installation completes successfully, you can move onto the next step.  (If it doesn’t complete successfully, contact @kennethreitz.)

Step 4: Download Tomcat

You can find the latest Tomcat release on the Tomcat Web site.  Download the zipped tar-ball (.tgz or .tar.gz), and make sure you’re downloading the Core, not the Deployer or the Source. (If you happen to need the Deployer, I have no experience with this version of Tomcat – sorry.)

This file you should be able to download directly to your (dv) with the following command

wget http://apache.inetbridge.net/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.tar.gz

Make sure that when you run this command, you’re in the folder into which you’d like the download saved.

Step 5: Unpack Tomcat

Once the file is on your (dv), unpack the zip file with the following two commands

gunzip apache-tomcat-6.0.20.tar.gz
tar -xf apache-tomcat-6.0.20.tar

This should create an folder named apache-tomcat-6.0.20. I prefer to have Tomcat live in a folder simply named tomcat, and I prefer to have this folder be a sibling to the httpdocs folder in my application root.  So I move the tomcat folder there with the following command

mv apache-tomcat-6.0.20.tar /var/www/vhosts/collegeman.net/subdomains/myapplication/tomcat

Step 6: Test Tomcat

Using my path for the example, run the following command

./var/www/vhosts/collegeman.net/subdomains/myapplication/tomcat/bin/startup.sh

Tomcat will report four pieces of information, boiling down to the location in which Tomcat is installed and the location in which Java is installed.  You will then be returned to the command prompt.  If you don’t see any other output from this command, you can assume Tomcat is running.  Test access to Tomcat with the following URL

http://yourdomain.com:8080

If what you see is the Tomcat Management interface, then Tomcat has been installed successfully.

Step 7: Create a Tomcat service

Using the command-line editor vim, create and open a new service file with the following command

vim /etc/init.d/tomcat

Use the following content to define the service file. Please note where you must configure this file to run on your personal (dv). Once in the editor, press the i key which will put the editor into insert mode.  Then paste the configuration below into the editor

# This is the init script for starting up the
#  Jakarta Tomcat server
#
# chkconfig: 345 91 10
# description: Starts and stops the Tomcat daemon.
#

# Source function library.
. /etc/rc.d/init.d/functions
 
# Get config.
. /etc/sysconfig/network
 
# Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0
 
tomcat=/your/path/to/tomcat
startup=$tomcat/bin/startup.sh
shutdown=$tomcat/bin/shutdown.sh
export JRE_HOME=/usr
 
start(){
 echo -n $"Starting Tomcat service: "
 #daemon -c
 $startup
 RETVAL=$?
 echo
}
 
stop(){
 action $"Stopping Tomcat service: " $shutdown
 RETVAL=$?
 echo
}
 
restart(){
 stop
 start
}
 
# See how we were called.
case "$1" in
start)
 start
 ;;
stop)
 stop
 ;;
status)
 # This doesn't work ;)
 status tomcat
 ;;
restart)
 restart
 ;;
*)
 echo $"Usage: $0 {start|stop|status|restart}"
 exit 1
esac
 
exit 0

Save the file by pressing the esc key, entering wq, and the pressing enter.

Next change the permissions on your script to make the file executable

chmod a+x tomcat

Test the script by executing the following commands

./tomcat stop
./tomcat start

You may get an error when you run the stop command – this is normal.  As long as you can get one full succession (starting then stopping) without errors, then you’re fine.

Step 8 (Optional): Front Tomcat with Apache

There are many things that you do not want Tomcat to be serving, namely static binary (like images) and text files (like CSS and scripts).  Honestly, I think this wisdom harks back to older, more inefficient times Tomcat land, still it is the best practice.  Doing so does have the additional benefit of being able to use more than one language on a single domain, e.g., PHP + Java.  So it’s not all bad.

As I’m pretty sure this task requires some source code compilation, I’m going to document it in a later post.

So do me a favor and let me know if this was useful to you.  Was it written well? Did I skip any steps? And most important, were you able to get it to work?

In the course of using Grails to develop a handful of government Web applications, I’ve discovered a small gap between my applications and my data.

Grails includes an incredible piece of software engineering they call GORM, which is short for Grails Object Relational Mapping and is essentially Groovy syntax for creating HibernateCriteriaBuilder instances. This is the best practice for writing database queries in Grails, and it goes something like this

def courses = Course.withCriteria {
  // only approved courses, please
  eq('approved', true)
  // in a category matching the keyword "database"
  categories {
    ilike('name', '%database%')
  }
  // with offerings occurring in the future
  offerings {
    gt('starts', '2009/09/29 21:12')
  }
}

This Groovy code is roughly equivalent to the following SQL statement

SELECT * FROM course c
INNER JOIN course_categories cc ON (cc.course_id = c.course_id)
INNER JOIN category cat ON (cat.id = cc.category_id)
INNER JOIN offering o ON (o.course_id = c.id)
WHERE
  c.approved = 1
  AND LCASE(cat.name) LIKE '%database%'
  AND o.starts > '2009-09-29 21:12'

Now if you don’t appreciate the difference between these two chunks of code, it’s probably because you either have no use for ORM tools (PHP developers, I’m talking to you), have never tried to use Hibernate, or have never had to write low-level data layer code with JDBC.  The rest of you are probably thinking, “Man, that’s pretty frickin’ cool.” Because it is.

As I’ve experienced it, there’s only one problem with the GORM way, and it’s actually a problem with the ORM way in general.

In our example above we reference three domain objects: Course, Category, and Offering. For the sake of our example, we will assume that there are one-to-many relationships between Courses and Categories, as well as between Courses and Offerings.  That means for every one Course in the system, we’ll have zero, one, or more than one Offerings related to it.

Relationships like these are exactly what RDBMSs and ORM are all about: taking flat relationships and turning them into hierarchies of objects.  The problem is that when we rely on these relationships for the purpose of querying data, we almost always introduce duplication into our result set.

Imagine a set of data in which there are two courses, each with two related offerings.  Imagine that all of these records satisfy our query above.  The result set for our SQL query would look something like this

COURSE.ID NAME             OFFERING.ID STARTS     ENDS
1         Tuning Your SQL  1           2009/10/26 2009/10/26
1         Tuning Your SQL  2           2010/01/01 2010/01/04
2         Hiring Good Help 3           2009/11/04 2009/11/05
2         Hiring Good Help 4           2009/12/06 2009/12/07

Take note of the duplication: two instances of a course record for each instance of the corresponding offering record. The problem has the potential to get worse when we introduce categories.

COURSE.ID NAME             CATEGORY            OFFERING.ID STARTS     ENDS
1         Tuning Your SQL  Database Management 1           2009/10/26 2009/10/26
1         Tuning Your SQL  Database Management 2           2010/01/01 2010/01/04
1         Tuning Your SQL  Databases, General  1           2009/10/26 2009/10/26
1         Tuning Your SQL  Databases, General  2           2010/01/01 2010/01/04
2         Hiring Good Help Database Personnel  3           2009/11/04 2009/11/05
2         Hiring Good Help Database Personnel  4           2009/12/06 2009/12/07

Where once there were two copies of Course #1 now there are four!

Now, in most circumstances, this really isn’t an issue. That’s right. I said isn’t an issue.

GORM and Hibernate both provide the means for eliminating this duplication at object instantiation, and it’s called distinct. All we have to do to implement this feature is change one line in our original GORM query (the first one):

def courses = Course.createCriteria().listDistinct {

Now if we were to loop over the contents of courses, we would find two objects instead of six: one for each unique Course is our result set.

One very common use case for Web applications is large result set pagination. While our own example result set doesn’t warrant pagination, one containing a thousand relationships between hundreds of courses, offerings, and categories, certainly would.

GORM includes two query features for pagination: firstResult, which should be a positive integer indicating the first row to return, and maxResults, which should be a positive integer indicating the page size.  When woven into our original GORM query, the code now looks like the following

def courses = Course.createCriteria().listDistinct {
  // only approved courses, please
  eq('approved', true)
  // in a category matching the keyword "database"
  categories {
    ilike('name', '%database%')
  }
  // with offerings occurring in the future
  offerings {
    gt('starts', '2009/09/29 21:12')
  }
  firstResult(x)
  maxResults(y)
}

So imagine a version of our result set that when raw contains 100 records.  That’s 100 instances of relationships between Courses and Categories, and Courses and Offerings.  Irrespective of the total number of distinct Courses, we decide that we’d like to paginate our result set 10 records at a time.  And this is where somewhere in the beautiful stack that is Grails, something goes ker-splat!

Check back soon for Part 2!

Until tonight, Spotlight had been allowed to continue running quietly in the background.  For some reason unknown to me, tonight Spotlight decided to start indexing the terabyte-or-so of backup data I have on my Drobo.  Stupid Spotlight: the CPU is reserved for programs that matter.

So I tried all of the GUI tricks for making Spotlight stop. There’s really only one: adding the Drobo to the Privacy list in Spotlight Preferences.  Needless to say, if that had worked, I wouldn’t be blathering about it here.

It didn’t work.  Fifteen minutes later, I discovered that this is another one of those OSX features that have to be disabled through a command line procedure:

sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

And just like that: Spotlight was no more.

So I’m using the Blog layout for a Category of Joomla content. (Yes, I realize Wordpress is a better tool for blogging.  Stay with me here.)

By default, the dates displayed with each article title look like this:

Sunday, 29 March, 2009 17:44

I mean, that’s great: if you’re British.

The first thing that baffled me was the composition of the templates in com_content.  Looks like this (in com_content/views/category/blog.php)

for ($z = 0; $z < $this->params->get('num_columns'); $z ++) :
	if ($z > 0) : $divider = " column_separator"; endif; ?>
 
	params->get('num_intro_articles') / $this->params->get('num_columns')); $y ++) :
	if ($i < $this->total && $i < ($numIntroArticles)) :
		$this->item =& $this->getItem($i, $this->params);
		echo $this->loadTemplate('item');
		$i ++;
	endif;
endfor; 

For the life of me, I couldn’t figure out what $this->loadTemplate(’item’) did (mostly because the patTemplate API is buried beneath the Joomla API, and because PHP toolsets just aren’t sophisticated enough yet. But I digress).

Google revealed that when you call $this->loadTemplate(String) from within a Joomla template file, the presentation framework looks for a file named “<template>_<name>.php” where <template> is the name of the current template file (in this case, “blog”) and <name> is the value passed to the loadTemplate method.

So then, in com_content/views/category/blog_item.php, I found this bit of obscurity

echo JHTML::_('date', $this->item->created, JText::_('DATE_FORMAT_LC2'));

Fortunately, this one was easier to follow, but in the end, I still had to use Google to discover that DATE_FORMAT_LC2 is a reference from Joomla’s implementation of internationalization. Internationalization is configured with the files stored in the language folder. At this point, the correct way to fix my problem would have been to create a new language zone (e.g., en-US), but instead I decided just to amend the existing source.

In the file language/en-GB/en-GB.ini, you’ll find a series of definitions looking something like this

DATE_FORMAT_LC=%A, %d %B %Y
DATE_FORMAT_LC1=%A, %d %B %Y
DATE_FORMAT_LC2=%A, %d %B %Y %H:%M
DATE_FORMAT_LC2=%B %d, %Y
DATE_FORMAT_LC3=%d %B %Y
DATE_FORMAT_LC4=%d.%m.%y
DATE_FORMAT_JS1=y-m-d

Theoretically, all of the core components of the Joomla framework use these definitions for their date formats, and the formats correspond to the PHP date function.  Change these settings to whatever you’d like, and enjoy the convenience of seeing your change applied globally.

That project you’ve been working on for months and months or maybe even years is almost into the test system.

You’re running down your checklist of features, painstakingly assembled from a series of torn legal pads, post-it notes, and napkins.   So far, everything is perfect.

Then, at 7:21 PM, you arrive back in the same old tired reality.

“I found another bug.”  Doh!

There are a number of problems with this scenario.

First, why have you been working on your project for months without putting something in your clients’ hands?

There’s a great book written by 37 Signals titled Getting Real.  Getting real means having the right team: small and organized, with a developer, a designer, and a generalist to pull it all together.  Getting real means staying focused: keeping the feature list small, and having a technical non-developer on-hand to establish completion.

But most importantly, getting real means getting real: drop the wireframes; forget about state diagrams; put fingers to keyboard and turn the idea into a product that can be used.

If you’ve never heard of or read the book yourself, check it out.

Second, if your checklist of features requires anything larger than a single piece of legal paper, then you have too many features planned, especially if you’re only on your first release.

At Clutch I’m working on a piece of software that has exactly fifty business rules.  Fifty rules is a lot of rules, but I’ve been doing this Web development stuff for a while.  I’m seasoned.  Plus, I’ve got people working with me: someone to do the straight-forward, fingers to the keyboard kind of work; and a crackshot designer to make it look good.

Feature creep begins innocently enough. “Instead of storing the shopping cart in the session, I’ll start storing it in the database.”

Before you know it, you’re worrying about things like scalability, modularity or code reuse, and efficiency; and you’re no longer getting real.  Instead, you’re tackling all those potential problems before honing any of those fifty business rules; and believe you me: those fifty business rules aren’t going to go away: fifty business rules is a lot of rules.

Keep it lean, dude.  Make it better later.

Third, why are you testing your own software?

Newbie mistake numero uno: you can’t test your own business logic.

It made sense in your head.  It made sense when you coded it.  It makes sense everytime you push that little button, the application turns green, and you get a record in the database.

But when your client pushes that other button, your application turns red, and the database disappears.  Suddenly, things stop making sense.

The only test you should do yourself is to push the play button on your unit testing code.

Unit testing code?  As the name implies, unit testing code is code that tests other code in units or modules.

Say you have an application that allows its users to shop for widgets.  The phrase “shop for widgets” implies all sorts of micro processes: loading records from the database, routing user requests, validating input, storing new records, sending e-mail, handling errors – the list goes on and on.

Each one of those micro processes is or is composed of units of work; and if you’re coding the sane way, you’re testing each one of those units independently of the whole.

It’s much easier to detect a faulty feature in a data access object when the test doesn’t involve the Web application interface – trust me.

These days, you can find a unit testing framework for any platform: JUnit for Java, SimpleTest for PHP, and NUnit on the .NET enterprise.  I can’t vouch for NUnit, but JUnit and SimpleTest are absolutely indispensible tools in my toolkit.

Once you’re unit testing, you might as well start doing test-driven development.  Test-driven development is development that is driven by tests.

First, you write some unit testing code that relies on a valid business process, e.g.,

    public void testMaximum() {
        assertEquals("The maximum of 4 and 2 is 4.", Math.max(2, 4), 4);
    }

Then, you write the code that enacts the business process.  You go back and run the test, and if the test passes, then you’re free to move onto the business rule.

This formula for composing code puts the application’s story ahead of the pages on which it’s written.  If you focus first on what the application is supposed to do, getting it there will feel more like winning the game than fighting an endless battle with yourself.

Fourth, why are you working at 7:21 PM?

Well, perhaps it’s for the same reason that I’m writing this blog article at 10:49 PM: sometimes the creative juices just flow faster in the evening.

But if you’re anything like me, you’ve also probably been at it since 8 AM; and even though you feel like the code is flowing easily, the reality is much more stark.

You’re creating tomorrow’s messes, today.  Hooray for your stamina!

Always remember the rule of thirds: one-third of your life will be spent asleep (if you’re lucky); one-third of your life will be spent at work; and the other third should be spent eating, doing things purely for fun (like sex), creating, or resting (sleeping some more, maybe?).

As developers who love to develop, we find it easy to be lulled into a sense of being creative through our work – that our work itself is part of that creating time belonging to the third-third of our lives.

Get real: work is work.  And though it ain’t like diggin’ ditches, programming is hard on the body.  Checked your posture lately?  How do your eyes feel right now?  Can you pinch an inch of fat around your midsection?  That’s what I thought.   Believe me, I’m right there with you.

So the next time you look down and realize that you should have left work thirty minutes ago, try to remember the fifth thing that’s wrong with our shared scenario.

Your code will never be perfect.

It is for this reason that Rands invented the Larry Test.   Because Larry would never have been done, unless his test told him so.

As Rands explains, when left to his own devices, a developer’s job is never done.  We refine, recode, refactor, reevaluate, and recode some more, until we think we’ve achieved perfection.

But perfection never comes.

Perfection doesn’t exist in our world any more than it does in the world outside the CPU.  At a high level like JavaScript, everything is ones and zeros.  With enough effort, you can probably herd all of those ones and zeros into a straight line, write a program to interpret them, and prevent cross-site scripting attacks along the way.  Perfection seems achievable.

But were you aware that when you depress a button on an electronic device, there’s a program somewhere, patiently and carefully waiting for the electrical switch to stop shaking wildly back and forth?  Waiting patiently to take the final reading, and establish a one or a zero?

What happens if that one or zero never comes?  It is, after all, nothing more than electricity.  A stray magnet, and EMP of one variety or another, a lighting bolt.  Any one of these will drive the voltage right of the charts, leaving that little program with nothing to measure and no way to recover from the failure.

Every single piece of code we write rests atop a stack of hardware: an inescapable electrical reality, confining our perfect programs to a dimension of physical randomness.

At the end of the day, fifty lines of code are probably just as efficient as five.  It’s only when your user base increases by an order of magnitude that the game becomes something different.

Consider how long its been since Twitter rolled out a new feature.   Surely they have some good ideas on hand.   But the developers at Twitter are busy doing one thing and one thing only: scaling their core feature set to 100,000 new users a month.  Their idea of perfection is a Twitter than can stand the test of the 2008 elections, or the next Super Bowl, or a natural disaster of Katrina proportions.

—

So, the next time you embark on a new development project, remember these basics:

• Get real as soon as possible: make it your mission to create a physical manifestation of your ideas
• Keep the feature list small and allow your project to evolve iteratively
• Do not test your own software: start with code that expresses your business processes and work backward
• Go home at the end of the day: exercise, eat well, and invest your free time in another human being
• And for the love of peat, stop chasing perfection because it doesn’t exist; and if you keep pursuing it, neither will you.

The problem with this has been that the stuff from Chicago, when unread, shows up in all three places.  What I wanted to do was separate unread, from unread Chicago e-mail, from everything else.  Each inbox in multiple inbox mode is driven by a filter, a feature of Gmail that is really broad and useful.

In filters, the search field acts more like a command line than a simple keyword matching tool.  For instance, if you prefix an e-mail address or name with the keyword from:, you’ll see only messages from that person.  This is different from just searching for the name or e-mail address by itself, whereby you’ll get address, subject, and body keyword matches. There are other prefixes and special keywords too, like to: and subject:, and also is:unread, is:read, and has:attachment.

But what I wanted to do was filter out everything unread and from Chicago.  To do this, I needed a not, as in show me everything that is unread, but not from Chicago. Gmail supports and documents the use of the word NOT in keyword searches, e.g., foo AND bar NOT foobar, but this doesn’t work with the prefixes listed above.  It took a bit of guess and check to figure it out, but eventually I discovered this solution: is:unread from:!email@domain.com.  (Take note of the exclamation point.)

This solution works perfectly, proving once again just how superior Gmail is to all other Web-based e-mail, as well as how well its developers have used patterns for solving their problems.