HIStalk Interviews Jennifer Lyle, CEO, Software Testing Solutions

Jennifer Lyle is co-founder and CEO of Software Testing Solutions of Tucson, AZ.

Tell me about yourself and about the company.

I’m co-founder and CEO of Software Testing Solutions. We’ve been around since 1999. We specialize in building technology-based quality assurance solutions to hospitals. We provide an automated testing solution — focused on laboratories and the blood banks to date –– to help them test, re-test, and maintain regulatory compliance throughout those systems in a way that’s much more efficient, much more effective, and with a lot more coverage than was possible manually.

The hospitals I’ve worked at all tried to use off-the-shelf scripting tools to write their own software testing scripts, but all of them abandoned the idea because of some application quirk or Citrix problem. Or, they realized we would never get enough benefit to have been worth the analyst time required. How is your product different?

That’s absolutely true. I think the latest studies show that across all the companies that try, it’s only like a 37% success rate. It’s very difficult to take the tool off the shelf and to take people who are not automation experts and have them develop robust, maintainable, and reusable scripts.

That’s where we are different. My background is as an automation engineer. The other co-founders of the company were a programmer and a med tech. We were able to take our expertise and our years and years of industry experience of how to use that tool and build something that really became an expert in the functionality of the system under test, so it was completely reusable and maintainable for the client.

The other problem with automated testing tools as they come off the shelf is that the average medical person is not an automation engineer. They can’t sit down and figure out exactly how to programmatically get the script to do what they want or set up the variables to do the variations of the testing that they want. 

Our solutions have a very straightforward front end that makes the system look a lot like a Cerner application or a Sunquest application under test. They fill in the blanks and use drop-down boxes to tell the system how and what they want to test. We keep the underpinnings up to date, so as the system under test goes through release after release by the vendor, we maintain it. The user never sees the underlying testing tool. This way, they can use it from the very first time and use it for years and years. We have folks that have been using it for over eight years in testing and validation.

I assume that software vendors use automation for their in-house QA testing. Do they offer similar tools to their customers so they can do their own validation?

Not that we’ve found. The vendors work very hard to do a very good job of testing their application that they’re developing with the data that they have.

As you know, every hospital sets up their catalog and their procedures totally differently than the next hospital. The flags they’re going to use, the warnings they want turned on, and where they want them turned on vary. It’s hard for those application developers to write a scripting tool that’s robust enough to make it productive for the client.

We prefer to not partner too tightly with any one vendor, especially in the world of the laboratory and the blood bank. The regulatory agencies prefer that if you’re getting assistance in your testing that it not be from the vendor who’s providing the solution. They believe that the more eyes that are looking at the system with a different perspective, the better the chances are that you’re going to find errors. If the people who programmed it are the people who are testing it, it’s testing with blinders on.

Your flagship offering is for the Sunquest LIS, but you’re now offering similar products for Allscripts / Eclipsys Sunrise and Epic, right?

Right. We’ve just started a division called Ratio. The focus of that division is on meeting the testing and validation needs of hospitals implementing CPOE systems. We’re going to be in the GE market, the Allscripts / Eclipsys Sunrise market, and Epic and Cerner as well.

It just seems such a like a perfect, natural flow to go from the laboratory and the blood bank now into the CPOE area. We’re having such a massive rollout of CPOE systems that it’s getting very difficult for the hospitals to exhaustively test all the permutations of patients and orders, where warnings should fire, where messages should appear, and where something should be allowed but something else shouldn’t. Automation would serve that industry very, very well right now. We’ve got the technology to do that.

It’s laborious for analysts to have to do all that testing and documentation. But to automate the process, do you need the cooperation of the application vendors?

We make them standalone. We don’t have any tight relationships with the vendors. Our clients are the end-user hospitals. They provide access to the systems to help us develop our testing scripts and to help us understand the sets of conditions that they want to test — what therapeutic duplications they want to test, what allergies, what drug-diagnosis interaction.

That lets us tailor it to how the hospitals want to use it, since again, it varies so much from hospital to hospital of what they need and what they want. We really want to serve the end-user community here. The vendors are doing the absolute best job possible with testing their solutions in-house, but once those systems are out in the field, you have the variety that comes with the unique configuration of every single institution.

Hospitals would ideally test often, every time they or their vendor make a change. Is your product more of a turnkey solution than a toolkit?

It is turnkey. When we provide that the solution to the client, we train them on how to use it. It’s very, very straightforward and simple for them to use.

Maybe you’ve put a new laboratory interface in and you want to make sure your Epic lab orders are crossing correctly over to your downstream and ancillary systems. With a few clicks of the mouse, our solution will extract all of your lab’s procedures out of your Epic database and, with a click of the button, it will place the orders for you. Once the lab has resulted those orders, another button will go in and look them all up and take the screen prints of those transactions coming back. You can do your results review checking at the same time. We provide a basic set of the patient-procedure pairings or patient-medication order pairings that you want to do.

What we’d love to do over time is continue to work with other industry-leading groups to identify the most common serious medication errors out there so we can build an even bigger sampling of prepackaged conditions.  We can quickly tailor those to a site, let them test it that way, and also give them the ability to add their own. Such as, if in my institution, I want to make sure that if a patient comes in with these demographics and these particular drugs are ordered, I want to see this type of warning.

We want to do both. We want to give you that prepackaged capability, right now, right off-the-shelf within an hour’s worth of training … have it be there and be productive for you and have it grow with you as your institution changes.

I would assume the primary return on investment is to free up analyst time, plus the chance to avoid a software-caused medical disaster that could lead to a lawsuit. What ROI parameters do customers consider before purchasing?

You’re definitely looking at the time and labor savings. You’re looking at a much more accurate testing protocol because it is being done by a computer, not by a human. It allows you to thoroughly do regression testing, which is going to get your releases in quicker as well. As you mentioned earlier, that’s a big problem with the vendors coming out with new releases. It takes the client quite a while to be able to implement those and a lot of that piece is in the testing.

The other area we’re seeing more concern about is providing proof of testing and being able to document. There’s a stronger push by the government getting more into Meaningful Use criteria and mandating certain testing. Our tool provides great comprehensive documentation in the form of reports and screen prints that these combinations have been tested and have been exercised. We can repeat this again at any point that the hospital desires: monthly, quarterly, annually, or when they take a new release. Whatever they feel is appropriate for their site.

Who’s your competition?

We’re very innovative in this particular role. In this particular area, I don’t know of anybody who has this style of an automated solution. There are consultants that you can hire to come in and manually exercise your system and check on it. The Leapfrog Group has their great CPOE tool that you can use and take to see how you’re doing on that performance, but that’s a pretty limited scope of combinations that you’re going to be testing. 

I think this is the leading edge. This is the next place where we can take a great step with technology to make CPOE implementations faster, to make them stronger, to get the benefit out of them, and the Meaningful Use that we’re trying to get out there, and show that the patient safety element is still out there as we implement.

As CPOE is implemented properly, it drives the quality of care and efficiency. When it’s flawed, it can lead to more issues. If we’re going to do it, let’s do it right and let’s make sure it’s functioning as we expect.

What are the challenges and rewards of being a small company offering a niche product that is targeted to customers of specific application vendors?

We’re focusing on the key players in the CPOE world. We’ve leveraged off of our installed base from the laboratory and blood bank side, where a lot of those site have Epic and Allscripts / Eclipsys in them. We’re also developing it with another client for GE and for Cerner.

I think when we have those fully out there, that’s a good representative piece of the market. We’ll continue to look forward and build more solutions for other vendors out there as the client need appears.

You’ve been in business for ten years. What skills and characteristics does it take to succeed?

It’s very customer-focused. We need to deliver value to our end user. We have to make a difference in their software quality. They need to be able to see a meaningful return of their investment in the form of time freed up from the analysts. They have to feel that they are catching things that would have gotten into production and caused patient harm, and we need to provide this at a very cost-effective price point.

So far, that’s what we’ve been able to deliver. In the entire history of our business, we’ve offered a full one-year, 100% money-back guarantee to all of our clients. We guarantee to people who have invested in our product that this is going to work for you. If it doesn’t, then we’re going to make sure you have your funds back to go find something that will work for you.

Putting that hospital’s needs first, respecting their business, and earning a seat there and providing value is what’s kept us in business and kept with us very, very loyal customers.

Have customers contacted  you and said, “Wow … this would have been a disaster if your system hadn’t caught this problem.”

Yes. We’ve had a number of those in the blood bank. We’ve had a number of those with implementing CPOE and the results for review crossing back where certain laboratory flags on tests results were not being carried correctly back into the results viewer in the HIS system, so the physicians were not seeing appropriate results. All of which could have caused a lot of harm if they had gotten through.

Where do you see the healthcare IT industry going in the next five to ten years?

I think it’s explosively growing. ARRA and our move toward CPOE is going to give us an unprecedented opportunity to get more technology out there and to drive quality care. It’s going to provide some challenges along the way. I think as long as we keep making sure we’re focusing on solving the little challenges that come up as we implement these great steps, these great strides, we’re going to see a huge benefit going forward.

We just have to make sure that, as we implement it, it really is working correctly. The tolerance for error in our industry is very, very small. But I think we’re going to see care at great new levels and great more efficiency. That’s what we’re really looking for – patient safety and a more efficient use of resources.