I had several people calling me over the last couple of days, wanting to talk about the recent Allscripts ransomware issue. A couple wanted my advice on protecting themselves, even though they use different vendors and have different system configurations. I have some good friends who spend the majority of their time during security risk analysis and white-hat hacking, so was happy to hand them over to the experts. One was a physician liaison at my former hospital, who wondered if I would write a guest column for their newsletter to help community physicians be more aware of the risks of ransomware. Their deadline is a couple of weeks out, so I’m happy to help.
Another was from a friend who uses Allscripts and wasn’t sure if her practice was impacted or not, so I got to explain the difference between being self-hosted and vendor-hosted. It sounds like her system is self-hosted but connected to vendor-hosted subsystems that have been impacted. For the most part she was just glad that she could see her charts and also glad to have “someone who speaks IT and can translate” available. She had been getting emails from her practice that included language forwarded from Allscripts that didn’t meet the need for understanding.
I also received a call from a former colleague who now works for a vendor and who “just wanted to catch up.” The call quickly turned into the most glaring example of schadenfreude I’ve seen in a long time. He went on and on about how this is going to be the death knell for Allscripts and how he was going to hit his territory hard and try to make sales. I had to remind him that his company has had its own share of issues, not necessarily with ransomware, but with outages on its own hosting platform.
There is plenty of quicksand for any vendor to land in, and sometimes I think only dumb luck prevents vendors from falling into the pit. Not to mention, going into a practice that has been impacted by a major outage and trying to sell a replacement system might not be a good idea in the short term. The proverbial corpse isn’t even cold and practices are still down, so a little patience and respect might be in order.
I have always preferred vendors who sell their products based on their own merits rather than by tearing down their competitors. Trying to make a purchaser feel bad about their current vendor calls into question their past decision-making and isn’t a way to win friends, in my book. Outages aside, every system has flaws and there isn’t one perfect solution out there. For every rock-solid feature, it seems like there’s something clunky hiding in the background to haunt you after you’ve already signed the contract. EHRs aren’t different from any other technology. As features evolve, sometimes they hit the mark and sometimes they don’t. It’s like buying a car – there’s always something you miss from your old car, or something you didn’t find on the test drive that becomes a daily annoyance.
I feel bad for the hosted physicians who are having to deal with the consequences of the ransomware and are being told to plan to be down on Monday. Although Allscripts is working on a read-only solution, it’s not clear how they’re going to deploy it or what it will include. This should be a wake-up call to physicians and hospitals and a good prompt to review their downtime solutions and maybe even give them a test. At my practice, we have monthly reviews of the downtime process and site leads have to check weekly that their downtime supplies are ready to go, but despite the preparations it’s always at least a minimum level of mayhem when downtime hits. The reality is that although ransomware and hacking get the spotlight, the majority of downtime events have more conventional or mechanical causes.
I’ve personally been the victim of the guy with the backhoe that cuts the fiber, the guy who accidentally triggers the Halon fire suppression system, and the lady who crashed into the data center and knocked out the electrical transformer. There’s also the winter storm that took down power lines, the system that froze because the server was out of memory, and the person who triggered a giant report to run against the production server in the middle of the day. Any one of those issues can make a system unusable and lead to a downtime event.
In my career at Big Health System, we had a utility that created a “lite” version of charts each night, sending records for all the patients in my panel to a local desktop. The lite chart basically contained the medication list, allergies, diagnosis list, and six months’ worth of laboratory and radiology data. It didn’t include scanned documents, but was enough to field a patient’s phone call. The utility also sent a “full” version of the chart for each patient scheduled for an appointment in the next 72 hours, which included the lite chart plus six months’ of chart notes and scanned documents from the laboratory, radiology, and consults filing structure. Theoretically, that would be enough to get one through an office visit with enough essential information.
That solution was great for a network outage but not for a power outage, so we had to make sure we had a fully-charged laptop with either a wireless modem or the ability to tether to a cell phone in the event that we lost power. The belt-and-suspenders coverage provided by this combination served us well through a variety of challenging situations. Of course, we also had a full disaster recovery plan, with distant servers and near-real-time fail-over processes, but thankfully I only had to experience that situation a couple of times.
Not every practice is fortunate enough to have staff dedicated to ensuring a smooth downtime. Still, you’d think with all the natural disasters we’ve seen in the past two years, that people would be doing a better job of it. I look forward to the day when I no longer hear about a practice whose only downtime preparation includes some photocopied visit note forms and a hope that someone printed a copy of the patient schedule before they went home last night.
For vendors servicing smaller practices, offering services to help clients put together a solid downtime plan would be great. I’d be interested to hear what vendors offer support for that type of a solution, and what other organizations small practices look to for downtime advice.
In the short term, however, I’m wishing the best to my colleagues on Allscripts. I hope the outage is short lived and your sanity makes it through mostly intact.
Have you been impacted by ransomware? Email me.
Email Dr. Jayne.