Time Capsule: Do Technology Surveys Rate the Hammer or the House?

I wrote weekly editorials for a boutique industry newsletter for several years, anxious for both audience and income. I learned a lot about coming up with ideas for the weekly grind, trying to be simultaneously opinionated and entertaining in a few hundred words, and not sleeping much because I was working all the time. They’re fun to read as a look back at what was important then (and often still important now).

I wrote this piece in January 2006.

Do Technology Surveys Rate the Hammer or the House?
By Mr. HIStalk

A disclaimer: I’ve worked at three organizations that were named Most Wired. In no case did we really believe that our IT was any better than anyone else’s, but we sure bragged about our victory at every chance.

The Most Wired survey season is upon us again. Eager candidates yearn for the recognition and improved job prospects they think the award will bring. Past-winner CIOs wake up in a cold sweat after having nightmares of opening the magazine and realizing that they’ve become a Former Most Wired (e.g., loser), while their cross-town CIO competitor bags a spot on the list and waves the rag triumphantly in their face.

I’ve seen the survey encourage the same sort of rationalized fabrication usually reserved for aggressive vendors imaginatively completing RFPs. Said one of my former employees to whom I turfed off completion of the survey form, “The survey’s vague enough that it’s not really about what we’re doing, it’s about how badly you want to win.”

For others, the survey’s detailed questions invite casual answers just to get the damned thing sent off in time. Think of those annoying “qualify for a free subscription” cards that require an hour of your time just to get a worthless rag that you’ll throw out unread anyway.

As I read over the 2006 survey form, I’m encouraged that it asks some really good and meaningful questions. If they’d just spot-check some winners and expose a few frauds, I’d be behind it 100 percent. Still, it is evolving fairly well.

As I remember, the original survey measured what you have. The emphasis was on buying stuff: wireless networks and PCs, for example. The vendor sponsors of the survey aren’t exactly against that concept, and even today, the incessant message is, “Good hospitals spend a lot on IT, bad ones don’t, so which one do you want to be?”

Today’s survey is more along the lines of, “How much of the stuff that you bought gets used?” Better.

Where it hasn’t gone yet is, “Did all that stuff make you better at patient care?”

If technology is something to be admired and honored with awards, then what’s the payoff, other than getting some cool Most Wired shirts? We should see a positive correlation to improved patient outcomes, reduced costs, and a happier work force and medical staff.

Indeed, lots of glossy paper will be sacrificed after the survey trying to prove some frankly questionable premises on why “Most Wired” means “Best” (unconvincingly, if you ask me). Was information technology the cause of the improved outcomes effect, or did they simply coexist in an unrelated way?

Maybe we’re doing this wrong. The hospitals I admire are those with tiny IT departments and budgets who, nonetheless, manage to meet non-IT criteria and instead are best in patient care.

I don’t admire carpenters with cool hammers. I reserve that judgment until I see what they’ve built with them. Maybe a few of the Lesser Wired could teach us all some lessons, after which we could still buy the cool stuff if we’re really convinced it would make us better.