Joel, Unit Tests and Software Pricing

Every time I read JoelOnSoftware, I am reminded what a brilliant writer this guy is. He's dead on about most things. Occasionally, he gets it spectacularly wrong, like here:

For instance, in software development, we like to have people unit-test their code. However, a good, experienced developer is about 100 times less likely to write bugs that will be uncovered during unit tests than a beginner. It is therefore practically useless for the former to write these… but most methodologies would enforce that he has to, or else you don’t pass some phase.

(admittedly, Joel didn’t actually write this. One of his readers did. But Joel summarizes with “Exactly true.”)

I’m sure this has been discussed to death in the JoS forums, but it’s clear that neither of these two really understands what unit tests are for, at least in the XP sense of things.

Units tests are not about finding bugs in the code you just wrote. Instead, they’re for finding bugs in the code that someone else writes six months from now that’s going to break your feature. And don’t tell me that true “rockstars” would never break someone else’s code. On any system of more than trivial complexity and time-scale, all developers are going to make mistakes like that eventually. So good developers write unit tests so that they can move forward aggressively and waste less time on manual testing their new feature in order to prove that it didn’t break anyone else’s code. It is anything but a waste of time. In fact, it’s a net gain over the long run.

But anyway — most of the time, Joel is spectacularly right. As he is here. In particular, the quote below explains exactly what’s going on with our efforts to find a development tool.

The reason I bring this up is because software is priced three ways: free, cheap, and dear.

1. Free. Open source, etc. Not relevant to the current discussion. Nothing to see here. Move along.
2. Cheap. $10 – $1000, sold to a very large number of people at a low price without a salesforce. Most shrinkwrapped consumer and small business software falls into this category.
3. Dear. $75,000 – $1,000,000, sold to a handful of rich big companies using a team of slick salespeople that do six months of intense PowerPoint just to get one goddamn sale. The Oracle model.

All three methods work fine.

Notice the gap? There’s no software priced between $1000 and $75,000. I’ll tell you why. The minute you charge more than $1000 you need to get serious corporate signoffs. You need a line item in their budget. You need purchasing managers and CEO approval and competitive bids and paperwork. So you need to send a salesperson out to the customer to do PowerPoint, with his airfare, golf course memberships, and $19.95 porn movies at the Ritz Carlton. And with all this, the cost of making one successful sale is going to average about $50,000. If you’re sending salespeople out to customers and charging less than $75,000, you’re losing money.

The joke of it is, big companies protect themselves so well against the risk of buying something expensive that they actually drive up the cost of the expensive stuff, from $1000 to $75000, which mostly goes towards the cost of jumping all the hurdles that they set up to insure that no purchase can possibly go wrong.

Now, a quick glance around the Fog Creek website reveals that I’m firmly in camp #2. Why? Selling software at a low price means that I can get thousands of customers right away, some small, some large. And all those customers are going to be out there using my software and recommending it to their friends. When those customers grow, they’ll buy more licenses. When people working at those customers move to new companies, they’ll recommend my software to those new companies. Effectively I am willing to accept a lower price now in exchange for creating grassroots support. I see the low price of FogBugz as being an investment in advertising that I expect will pay off many times over in the long run. So far, it’s working very well: FogBugz sales have grown more than 100% for three years without marketing, solely based on word-of-mouth and existing customers buying additional licenses.

By comparison, look at BEA. Big company. Big price tag. The price alone means almost nobody has experience with their product. Nobody comes out of college and starts a dotcom using BEA technology, because they couldn’t afford BEA technology in college. A lot of other good technologies have doomed themselves with high prices: Apple WebObjects was irrelevant as an application server because it started at $50,000. Who cared how good it was? Nobody ever used it! Anything made by Rational. The only way these products get into the hands of users is with an expensive full-frontal sales pitch. At these prices, the sales pitch is made to the executive, not the techie. The techies may well actively resist bad technology with good sales that the executives force down their throats. We have lots of FogBugz customers who have high-priced Remedy, Rational, or Mercury products sitting on the shelves after investments of well over $100,000, because that software isn’t good enough to actually use. Then they buy a couple of thousand dollars worth of FogBugz and that’s the product they really use. The Rational salesperson is laughing at me, because I have $2000 in the bank and he has $100,000. But I have far more customers than he does, and they’re all using my product, and evangelizing it, and spreading it, while Rational customers either (a) don’t use it or (b) use it and can’t stand it. But he’s still laughing at me from his 40 foot yacht while I play with rubber duckies in the bathtub. Like I said, all three methods work fine. But cheaper prices is like buying advertising and as such is an investment in the future.

It’s a terrible Catch-22. The companies making good tools (ones that I’d want to use) price them in the thousands of dollars. But at a cost like that, they can’t afford to waste a week to respond to a RFP or to send out a sales team to our office. Thus they don’t even get considered by my company. Honestly, IBM sent out six people and bought us lunch for 40. That first meeting alone would have blown the entire profit margin on a sale of JIRA.

I read a quote a couple of years ago on usenet (that I can no longer find, or I’d link it) that ran something like: “Rational does a great job of recognizing that their customers are not the developers, but the CTOs — people who will never actually use their tools. So they spend all of their energy perfecting a demo that will wow the CTO and none of their enegry making developers’ lives better.” Too right.

FYI: My company decided not to go with SourceForge. The lack of workflow was the ultimate killer. Even my colleague, who was well disposed to like them at the beginning, said 1) “Our current mish-mash of tools are better than theirs” and 2) “They should be 2 years further along than they are.” Which is true. Ah, the dangers of the total-system-rewrite.