Home » Dr. Jayne » Currently Reading:

Curbside Consult with Dr. Jayne 10/21/13

October 21, 2013 Dr. Jayne 6 Comments

I mentioned a few weeks ago that I had been preparing to sit for the new Clinical Informatics subspecialty board certification exam. Now that the testing period is over, I can talk about my experience. The American Board of Preventive Medicine expressly forbids candidates from discussing the exam content or questions so I’m not going to get myself into trouble, although there have certainly been some pretty significant discussions about it via the virtual water cooler.

The core content outline for the clinical informatics exam includes fundamentals, clinical decision making, health information systems, and leadership/change management. I don’t think there’s anything in the outline that is unreasonable. A major problem in trying to test that fund of knowledge, however, is that clinical informatics is an evolving specialty. Very few of us who have been in the field for any length of time intentionally set out to become informaticists. Of course there are some younger colleagues who decided to pursue it during medical school and attended a masters programs or fellowship, but I don’t believe they make up the majority of the seasoned workforce. The fact of how our discipline has developed (and subsequently exploded during the era of Meaningful Use) means that our field is extremely heterogeneous. That makes any kind of standardized evaluation a daunting experience.

Just look at physician titles as an example: CMIO, medical informatics director, clinical informaticist, medical director of informatics, medical informatics specialist, medical director of EHR, and my favorite: process improvement consultant. All of those titles are held by close friends of mine who are essentially doing the same job. Others have the CMIO title, but do vastly different jobs. Some are almost exclusively administrative, where others are heads-down writing their own code. Most of us are somewhere in the middle. We may practice medicine or not (although to sit for the new exam you do have to be currently certified in another clinical specialty.)

Preparing for the test was a good exercise. learned some things I didn’t know and encountered a lot of minutiae that, although “fundamental,” has very little bearing on my daily work. The thing that struck me most about the actual exam was that depending on your work experience, the difficulty of the questions varied. Like most exams, there were many “which is the best” and “which is the most appropriate” type questions with multiple correct-appearing answers. If you were straight out of a fellowship with little real-world experience, these might be easier. For those of us who have encountered hundreds of different variations on the questions (depending on whether the setting is inpatient vs. outpatient, whether the physicians in question are independent or employed, whether we’re wearing our physician hat or CMIO hat, what kind of system we’re talking about, etc.) the subjective nature of the questions made them difficult. Some of the questions I’ve actually encountered in real life and the answers to them may have been different at different times in the same project lifecycle.

Whether or not I passed (and truly I have no idea because it’s a new test and hasn’t been normed yet) isn’t going to impact my skill as a CMIO or make me less knowledgeable about handling tough decisions in the real world. Preparing for the exam though was stressful and actually taking it was an exercise in lack of user friendliness when you consider the whole electronic test-taking process used a third-party vendor. I was herded into the exam room along with firefighters, real estate agents, aspiring graduate students, and a couple of people who almost got ejected because they wouldn’t cooperate. We had to turn our pockets inside out, lift our shirts to expose our waistlines, raise our sleeves above the elbow, take a palm scan, and pay hundreds of dollars for the privilege. It certainly wasn’t that bad at another facility where I took my primary board exam a few years ago, but I guess things change. They didn’t make me take my glasses off to ensure I wasn’t hiding a chip or camera in them though, but I bet they do next time given the emergence of Google Glass.

Once most people finish taking this kind of test (after several months of preparation and at least some measure of worry) there’s this letdown period where you don’t know what to do with yourself. I did my best to keep busy and not second guess myself, but there comes a point where you just start thinking about the questions and how if you were in charge of the test, you’d do it differently. I couldn’t help but think of all the questions that weren’t on the test. Let’s pose some sample test scenarios and see what HIStalk readers think. It may not be evidence-based, but it is certainly ripped from the pages of real-world experience.


Item 1: As a clinical informaticist, you must often put yourself in the place of your stakeholders and end users as you make complex decisions. Read the following scenario, then select the best response to the associated questions. You are the finance director of a large employed medical group. The IT Director has decided to go to Las Vegas for the bachelor party of one of the IT analysts. He “signs out” authority over the ambulatory EHR project to you while he is gone, because after all, you’re both directors. He does this via email after 5 p.m. on Friday night then turns his phone off.

What is the most appropriate step for your first course of action?

a) Call Human Resources because it’s completely inappropriate for a supervisor to go to a Las Vegas bachelor party with his subordinates.

b) Call the IT director’s supervising VP because you’re not even in the same vertical and it’s not appropriate for you to take responsibility for his IT portfolio.

c) Call the CMIO and the EHR application manager and make sure they have your back.

d) Proceed to the local supermarket, purchasing more vodka than cranberry juice.


The weekend passes smoothly and without incident. However at 4 a.m. Sunday, your cell phone rings. The CMIO’s name is on Caller ID. Which of the following best describes the scenario?

a) This is a bad thing.

b) The clammy sweat you just broke tells you this is a very, very bad thing.

c) Anytime the CMIO calls you in the middle of the night it is extremely ominous.

d) You are glad you went to bed early Saturday night, because you have the feeling it might be days before you sleep again.


The CMIO explains that some cowboy analysts decided to perform an unauthorized “data migration” on your ambulatory database last night. No one was aware of the proposed maneuver as it was absent from the weekly change control discussion. Additionally, contractors were involved and it may not have been tested fully before deployment. It appears there may be some patient charts which have been corrupted, so they called the CMIO for advice. What is the most appropriate next step?

a) Curse the IT director’s name then try to track him down in Las Vegas.

b) Try to stop shaking, and then eat an entire box of chalk to address the heartburn you know is coming.

c) Be glad the CMIO has extensive experience resuscitating patients and staying calm because her demeanor makes you feel like it’s going to be OK.

d) Start making coffee; you’re going to need it.


Item 2: Review the above scenario. You are now in the role of the CMIO. It is 4 a.m. and you have just called your colleague. You calm her down and explain that you’re not that worried about the corrupt data because the organization has a well-vetted backup strategy and the team should be able to restore the database from the nightly backup, then apply the transaction logs up until the point they forced users off the system and started the migration. The finance director approves your plan and you phone the DBA on call. What is the most appropriate information to provide first?

a) Loudly ask, “What were you thinking?” and why he allowed unauthorized access to the production system.

b) Tell him to wake up his boss “because there’s gonna be hell to pay.”

c) Remind the DBA of your no-fault policy for reporting errors. A root cause analysis will be performed later and systems will be evaluated to prevent this from happening in the future.

d) Recommend a good travel agent for a one way trip somewhere far, far away.


The DBA makes some vaguely disturbing comments, so you get all involved parties on a conference line. Using your best patient interview skills (gleaned from years of trying to get the truth from drug-seeking patients) you begin to piece together what actually happened before they called you. Of the following system failures that have occurred, which is the most significant?

a) The team started the untested migration during the nightly backup process.

b) Users were on the system documenting clinical encounters during the migration.

c) The DBA already tried to restore from the admittedly non-reliable backup and it’s been running for two hours, but he neglected to mention it or the fact that he’s not sure the transaction log shipping was running properly prior to the incident.

d) A Level I analyst decided to wait four and a half hours before calling the CMIO for help and only called her because he “didn’t know who else to call.”


You ask for a list of users who were on the system when the incident started in an attempt to determine what kind of documentation they might have been doing and how severe a data loss might be. Should there be an unrecoverable data loss, which user will provide the most severe tongue-lashing?

a) Chief medical officer.

b) Chief of surgery.

c) Self-described “EHR Hater” who will use this as an attempt to mothball the ambulatory EHR project.

d) Your partner, who doesn’t want to be named as actually using the system since he brags to everyone in the physician’s lounge how he refuses to use it.


You invoke downtime procedures and notify the lone urgent care site that will be opening in less than three hours that they should document on paper and use the read-only downtime server for critical patient information needs. You notify all users that the system is down and immediately disable remote login capability so that no one can access the system while you figure out the recovery strategy. What portion of your user base will claim they never received notification?

a) None, as your communication plan is thorough and multi-media with multiple layers of backup.

b) Less than 10 percent, because most users are diligent about checking emails.

c) 20 percent, but they will find the email notifications three months from now when they get caught up on email. They will not apologize for yelling at you, however.

d) 90 percent, because they will jump on the bandwagon and complain about anything EHR-related.


Item 3: Given all the facts above, calculate the odds that the system will be back up with full data integrity before patient care starts Monday morning. Show your work. Estimate the total length of downtime. Conclude by estimating the total cost (in resources, recovery, and lost productivity) of this change control misadventure.

Bonus question: Estimate the number of employees who will be reprimanded for the actions leading up and/or during to the incident.

If you answered “C” to the multiple choice questions, then you just might be a CMIO (or someone who does the job without the title). As for Item 3, the system was indeed back up and ready for business by Monday morning. The total cost of the event was TNTC – as they say under the microscope, “Too numerous to count.” Before it was all over, we had to involve multiple representatives from various vendors, including the regional HIE and other downstream entities that received corrupt information and also had to perform various rollback activities.

I’m not going to give the answer to the bonus question just yet. If you want to make a guess, use the comment feature below. I’ll provide the actual answer later this week. For those of you who sat for the exam, I hope this gave you a laugh because we need one while we wait for the results.

What did you think about the exam? Email me.

Print

E-mail Dr. Jayne.



HIStalk Featured Sponsors

     

Currently there are "6 comments" on this Article:

  1. In re: certifying tests. Recall the old adage, “Those that can do and those that can’t, teach.” This has expanded in the 21st century to “Those that can’t teach either consult or write exams.”

  2. Thanks, Dr. Jayne. I found the test reasonable, even if some questions were superficial memorization and others had grammatical issues. But overall the exam did for the most part reflect the core content outline.

    And for kmhelper, you have the quote wrong: Those who can do, those who understand teach. (Aristotle)

    Signed,

    The Informatics Professor

  3. The attribution to Aristotle is probably in error. I was quoting a bit of 20th century humor, Professor. “He who can, does. He who cannot, teaches.” ~George Bernard Shaw [Man and Superman (1903)

  4. Having written questions for national board exams myself in the past, I empathize with whoever was charged with writing questions for the clinical informatics exam.

    Good test questions require that core concepts can be identified that are important for all test takers to know and that can be expressed with a fairly high level of specificity. The topic outline, though it covers the breadth of clinical informatics, is not constructed in sufficient detail to be able to map the concepts into discrete, testable learning objectives.
    Questions can be framed on those core concepts that have a clearly correct answer as well as other answer choices (i.e., distractors) that are plausibly correct.
    if there are approaches to a scenario where reasonable people may differ then creating a question on that topic are difficult and often seem contrived.
    Alternatively, in the search for a question that has a clearly correct answer, exam writers may stoop to testing about increasingly inane factoids. These may technically match the topic outline but aren’t really crucial or fundamental knowledge for subspecialty practice. In other instances, such questions represent the personal biases or pet projects of the question writers.

    Given the nebulous nature of many of the outline topics, I actually thought this was a reasonable exam and especially good for a first effort. There were a number of questions that were actually thought-provoking. Of course there were some random factoids being tested that may or may not have been that important for any clinical informatician to commit to memory but I didn’t think it was any worse than other multiple choice exams that I’ve endured during my education and training.

    In terms of the testing center experience, I actually found it fairly painless. Of course the last time that I had to take a standardized exam, my colleagues and I showed up with a fist-full of sharpened #2 pencils and sat in alternating seats in a big lecture hall. The highlight of that exam was the patient management problems where we each got a special marker to highlight our management choices. Swiping the marker revealed additional information about the test findings or the change in patient status due to our (ill-advised) intervention. After that exam, i thought I may have been better off reading Good Housekeeping for medical advice rather than Harrison’s and after this exam, I was equally unsure that any legitimate studying made any difference. But as they say “It is what it is”.

  5. For those of you playing along with the home game, I promised to answer the bonus question later in the week. The answer is zero – no one was reprimanded for any of the incidents that happened. Given the culture of the organization in question, I wasn’t surprised.







Text Ads


RECENT COMMENTS

  1. It seems that every innovation in the past 50 years has claimed that it would save money and lives. There…

  2. Well, this is predicting the future, and my crystal ball is cloudy and cracked. But my basic thesis about Meditech?…

  3. RE Judy Faulkner's foundation wishes: Different area, but read up on the Barnes Foundation to see how things work out…

  4. Meditech certainly benefited from Cerner and Allscripts stumbles and before that the failures of ECW and Athena’s inpatient expansions. I…

  5. Yes, Meditech will talk your ears off about Expanse. There are multiple factors at play here which undercut both Meditech…

Founding Sponsors


 

Platinum Sponsors


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Gold Sponsors