Home » Dr. Jayne » Currently Reading:

Curbside Consult with Dr. Jayne 4/25/11

April 25, 2011 Dr. Jayne 2 Comments

In the last several weeks, tornadoes and other severe storms have ripped through various parts of the country. Based on a frantic phone call I received from a medical school colleague, this seems to be a good opportunity for a physician-friendly discussion of disaster preparedness for healthcare information technology. For those of you who are serious IT professionals, this may be boring, but on the other hand it may be a good conversation starter to e-mail (or even print if you have to) for the physicians in your lives.

Downtime and Disaster Recovery 101

The most important part of successfully dealing with an outage of your electronic health record is to have a plan. Most practices need both short-term and long-term plans, whether you’re in a well-known hurricane zone or tornado alley or not. Lots of things can happen: floods, fires, and earthquakes. No one is exempt and everyone needs a plan.

Downtime usually refers to a time when the system is unavailable, whether planned or unplanned. Downtimes can happen for a variety of reasons. Unplanned downtimes may include a local power outage, loss of Internet connectivity, or other nonspecific system issues that keep physicians from fully using the EHR. They may be limited — perhaps it’s just an outage of e-prescribing or faxing — or may affect the system across the board.

Limited downtime events often have simple workarounds. For example, if e-prescribing or faxing is down, one can always print prescriptions or documents, call medication orders to the pharmacy, or worst-case scenario (ugh) use a paper prescription pad and a pen. Loss of Internet connectivity can be overcome by using a cellular / wireless Internet card, provided the practice has planned ahead and such cards are available for use. If the local wireless network in the practice is out of commission, users may be able to plug in, assuming there are ports available.

For unplanned downtimes, unless they have in-house IT support 24×7, practices should ensure key personnel have checklists for troubleshooting issues and phone lists for Internet service providers, vendor help desks, etc. Make sure multiple people in the practice know how and where the information is stored — don’t count on a single employee to be the point of knowledge. Murphy’s Law dictates that if something goes wrong, it will go wrong when that employee is unavailable.

Planned downtimes are usually limited downtimes. This may include hardware upgrades, software upgrades, weekly or monthly maintenance, etc. When planning a downtime, physicians need to discuss their willingness to work without full access to the EHR. Many physicians may be willing to print summaries for patients who may be scheduled during an upgrade and ‘wing it’ for others. For some, being without data is unacceptable, and the office must be closed.

Careful planning can keep physicians from having to make this decision. Many vendors offer solutions where a copy of the database can be saved to a local computer and accessed in a read-only manner during an upgrade. There are several third-party solutions to this problem, and if you are interested in this for your practice, allow some time (often a few months) to make sure this is in place before a planned downtime.

Disaster recovery usually refers to a situation where something very, very bad has happened. This can include physical destruction of the practice, its servers, and its equipment due to a natural disaster. If the IT infrastructure is physically destroyed, it may be weeks before the practice can be up and running. Disasters can also occur due to poor planning, as my friend learned.

Practices need a plan to create backup copies of the data in the event of a disaster. If you use a Web-based or hosted EHR, often your vendor takes care of backups for you. However, you need to understand the interval at which backups are done. Daily, weekly, monthly? To determine how frequently you need to do a backup, ask yourself: how much data are you willing to lose? For a busy practice, backups should be done daily and practices should consider other strategies to continuously back up data throughout the day (but that’s beyond Disaster Recovery 101, so I’ll save the discussion of transaction log shipping vs. database mirroring for another day).

Backups should not be stored in the office. Think it through: if your office catches on fire and the backup copy is at the office, that’s not a great idea. Backups need to be stored securely under appropriate climate conditions — be mindful of temperature, humidity, etc. There is one important thing about backups that doesn’t cross most physician minds: the need to test the backup to make sure it works. Your IT professionals can do this by taking the backup copy of the database and restoring it to a test system, then checking it to make sure data is current and comprehensive.

Unfortunately, the solo physician who called me this morning learned this the hard way. When the power went out and the battery backup failed, the database was impacted. Her vendor recommended that they restore the database from the most recent backup. When this was attempted, the backup contained less than half the data they expected it to. Not a great situation. Although she was fortunate that the EF-4 tornado didn’t touch her building, it’s going to be a challenge to recover from the loss of so much data.

So physicians, heed this cautionary tale. Take a moment to discuss your downtime and disaster recovery strategies with your IT support staff, whether you work in a solo practice or for a large health system. Don’t be afraid of stepping on the IT team’s toes — many are proud of the downtime strategies they’ve created and will be happy to talk about them. If there is no written plan, make it a point to create and document the processes you need to practice should the system be unavailable. Make sure key staff have copies of the plan, and practice it. Use regular maintenance windows as an opportunity to practice what you would do if an unplanned outage occurred.

Preparing for system outages should be a regular part of the life of the practice, no different than fire drills, tornado drills, or the like. The odds of something bad happening may be slim, but if you’re in disaster’s crosshairs, you’ll be glad you took the time to prepare for the worst and to protect your patients and your practice.

E-mail Dr. Jayne.

View/Print Text Only View/Print Text Only

HIStalk Featured Sponsors


Currently there are "2 comments" on this Article:

  1. You mention fire drills and tornado drills, but IT disaster drills are just as important! The written plan means nothing if the staff doesn’t know how to execute it when the time comes. Of course, in a busy practice there’s never a good time for an IT disaster drill – but if you don’t set aside the time, you’re guaranteed for an even bigger disaster when the real on hits.

  2. You have raised here a very important topic of concern regarding the role of IT in crisis handling and physicians say on it. What I think is that this is one of the major factor one should consider while selecting the various Health IT applications and tools for deployment in your practice. In a good Disaster Recovery plan one should also consider the People factor . Preparing your organizational culture is as much as important as preparing for system outages . Unity and shared vision among Physician leaders and IT leaders is required. At the time of disaster/crisis proper communication channel should exist and there should be firm existence of cooperation with each other ( cooperation among IT and Clinical staff/ practice management and Vendors etc). Also not to forget to learn the lessons from the past events you experienced and/or related events happened elsewhere and how they combated ?

Subscribe to Updates



Text Ads

Report News and Rumors

No title

Anonymous online form
Rumor line: 801.HIT.NEWS



Founding Sponsors


Platinum Sponsors































































Gold Sponsors















Reader Comments

  • richie: Wonderful topic (I'm biased as I strive to implement systems I'd want, prior to my own long-term care becoming imminent)...
  • richie: Thanks Ed, I'm impressed by your ability to pursue new endeavors....
  • Melissa: I have enjoyed your posts every month, and have followed you through your life changes as well as mine. Thank you for a...
  • Edward Marx: That would make for a good post! Let me think on it and look for me on LinkedIn....
  • Rebecca: Thank you, Ed. I always looked forward to your posts - they were a validation to me that you don't have to have to be a...
  • Lee David Milligan: Rob: thanks for the comment. Agree with your point re: unique expectations and complexity....
  • Shelly: Thank you Ed for taking the time to invest in the industry as you have done - with this blog as a sounding board for CIO...
  • Appreciatve: Thanks,, Ed. This was a very nice piece, and telling of what you cherish most. There is one topic that I am not cert...
  • Rob Price: Excellent information and quite consistent with my experiences since 1999 working with three different software companie...
  • Cosmos: Thank you for the interesting article. FYI - the terms "Severe Sepsis" and "Septicemia" are no longer in common use, ...

Sponsor Quick Links