Inventing in software

To invent, you need a good imagination and a pile of junk.

— Thomas Edison

This is what is so facinating about programming. Your “pile of junk” consists of digital assest instead of physical matterial, so the raw materials are not limited by the normal laws of supply and demand. In software, you are limited only by your imagination.

Does Size Matter to Search Engines

Yahoo and Google are trying to one up each other on whose database contains more pages. Does it really matter? Isn’t relevancy more important? It depends on who your user is.

If most of your queries on search engines return hundreds or thousands of results, then it probably doesn’t matter who has the biggest index. As long as Google and Yahoo get the popular pages, you’ll most likely find what you are looking for. If you get a bunch of results from your search you are probably aren’t looking for a specific document on the web, so it doesn’t matter which page you get as long as it has the information you are after.

Most internet users fall in the above category. I tend to find that many of my searches return less than 20 results and sometimes only 2 or 3 or even 0. For people like me the number of pages searched is much more important. In fact for those types of searches the method used to order the results (determine the relevancy) isn’t really important. With a small number of pages, it is easy to scan through the list and find the most relevant entry.

Today I was working on a piece of equipment and it started giving me a less than helpful error message. I typed the manufacturer (in quotes) and the error message (also in quotes) into Google hoping to find out how to fix my problem. There were zero results. I tried it on Yahoo and got the same thing. After poking around in some forums I was able to find a post that described the problem on a page that was missing from both Yahoo and Google’s databases. The post contained all the phrases that I had searched for it just wasn’t in either of the search engine’s indexes.

Five years ago I taught a community college class about the internet. I used Geocities to put up the tests, class outline, etc. Today I tried search for:
site:geocities "juco internet class"
In Google I get zero results. With Yahoo I get a single result that links to the page I was searching for.

I’m sure that there are other items that I could find with Google that won’t show up in Yahoo. I’m not trying to say one engine is better than the other. The point is that the index size does matter if you are looking for a specific document. If Yahoo decides that they are going to maintain a larger index than Google, then there are going to be pages that can be found in the Yahoo index, but not in Google’s. If you are looking for one of these pages then size is very important.

Storing your Maven Repository in CVS/Subversion

Brett Porter has hacked together a tool that will let you use a CVS or Subversion repository as your maven repository.

Brett Porter – Storing your Maven Repository in CVS/Subversion
It’s pretty rough, but is a working prototype that makes Maven 1.1/2.0 downloads a checkout/update, and deploy is an add/commit. I see this would be useful for snapshot repositories, where you could use one filename instead of transforming the version, so getting the latest would literally be an svn update.

If you are using Subversion with Apache, it is pretty easy to achieve most of this. The problem that I’m faced with is the fact that Maven can’t handle repositories that use SSL and a login.

Currently, I’m using a separate server to host our Maven repository because the Subversion server is using SSL. I hope that Maven will eventually come up with a way to work around this, but right now it looks like most of their efforts are being spent on Maven 2.

Ignoring Build Problems

I ran across this blog post that is probably typical of many people who are managing software projects.

Musings of a Software Development Manager » Blog Archive » CruiseControl Warnings
I get about 48 email messages from Cruisecontrol each day for one of our projects. This is not something I’m proud of since this situation has existed for at least 4 weeks now, we’ve had a broken build. The problem stems from some nasty functional tests that no one wants to investigate and we’ve sort of let our process slip.

There is a simple solution to this. Turn off the tests that are failing. People’s first reaction to this is “Oh no, we can’t turn off the tests! They indicate that something is wrong. Eventually we’ll have time to fix it.”

If you are actually going to fix it go ahead, but if something has been broken for more than a week, chances are no one is going to fix it any time soon. You should turn it off so it starts building without errors again.

Why is this better? If your team gets 10 emails each day saying that something is broken, they are going to ignore it. No one is really responsible for all of the problems, so no individual really works on fixing it. However, if the build is working correctly and someone checks in code that breaks a unit test and everyone gets and email, that person is probably going to try to fix it because it shows that he is responsible for the problem.

Think of it another way. Lets say I have 3 smoke alarms, 1 gas alarm, 1 CO2 alarm, and a flooded basement alarm in my house and they all sound pretty much the same. Now lets say that the flooded basement alarm goes off and I decide that it isn’t important enough to fix the cause…. So I just let the alarm go off. How likely do you think I am to notice if another alarm goes off once I get used to ignoring the first alarm.

If I’m not going to fix the problem, the best thing I can do is disable the flooded basement alarm until I have a chance to fix it. After a week of ignoring the alarm and nothing bad happening, I’m not suddenly notice it and decide I should do something about it.

One of the first things I did when I started at my current job, is go through and renamed every test that failed our automatic build process as “pending”. By the time the test would run, I had disabled about 2/3 of the tests. Since they were failing we ignored them anyway, so marking them as pending didn’t change anything. Before they were turned off, it would have been impossible to notice if one of the tests that were previously working broke because of a change.

Over time we’ve turned most of the pending tests back on one at a time as we’ve had more time to fix the code or fix the test.

When your tests fail, it should be unusual. I setup our builds to break if any test fails. I’ve got a lava lamp above my cubicle and everyone in the company know what it means. If something breaks people start asking the developers about it until it gets fixed.

Enabling Color Terminal on OS X 10.4

OS X has an environmental variable called CLICOLOR. If it is set, it turns on color for command line commands. So…

export CLICOLOR=true

will turn on color. If you want to enable this for everyone, you could do that by putting it in /etc/profile.