Tidepool and the FDA Pre-Cert Program: Software Iteration FTW!

If there's one thing that successful startups in Silicon Valley will tell you, it's that your ability to iterate quickly defines your success. The faster you can iterate and deliver new ideas, the more quickly you can converge the winning ideas based on real feedback.

We were thrilled to learn that we had been accepted into the FDA's Digital Health Software Pre-certification Pilot program. Why? Because the goals of the program are exactly what you want to see the FDA doing: embracing the notion that modern software development techniques have changed DRAMATICALLY in the last 10-20 years, and that old school, "waterfall" methods of thinking about software regulation are actually harming public health. As Bakul Patel said during the announcement of the program, "We think in the world of software, iteration is actually better for public health." We couldn't say it better ourselves.

The current General Principles of Software Validation guidance document had been written in 2002, which replaced the guidance document that was written in 1997. (You might remember 1997: JavaScript was a thing, but certainly not Node or React. Test automation was a thing, but certainly not Travis or CircleCI. And Google was still a year away from looking like this:) [source]

But there's one important thing that you'll notice about every single FDA guidance document. They all have a glorious box on the front of them that says something like:

This guidance represents the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance.

(Emphasis mine.)

These guidance documents are not the law. Only the law is the law, and even then the FDA is pretty awesome about applying "enforcement discretion" when they know that the law hasn't kept up with modern times. We could not figure out how any company could possible deliver rapid, iterative solutions by following the old school software guidance.

When Tidepool was getting started in 2013, we thought "OK, the text in that glorious box sounds awesome! Let's go talk to the FDA about an alternative approach!" So we packed the kids in the station wagon (not really) and headed to Silver Spring, Maryland (really). We had gotten lots of great advice from people like Andy Balo at Dexcom saying "The FDA doesn't like surprises. Tell them what you are up to. Meet with them early and often."

So we did. And it was awesome.

We had a series of "Pre-sub" meetings with the FDA, a type of meeting that companies use to discuss their upcoming products (hence "pre-submission"). These are on-the-record meetings; you take diligent minutes, and those notes are reviewed and approved by the FDA. As a nonprofit, and an open source effort, one of our missions was to share everything we learn openly and publicly. So you can read our pre-sub submissions and approved minutes here.

In one of those pre-sub meetings, we said "We're going to show you our 21 CFR 820 compliant software quality system, but it's not going to look like any other quality system you've ever seen... We're using crazy new tools like GitHub and Travis and Google Docs. And we sometimes ship software every day." We walked the FDA team through how we did both automated and manual testing, and how we used Trello and Google Docs to document everything. We showed them how software changes can be meticulously tracked in GitHub, and how TravisCI would run hundreds, if not thousands of tests every time we changed code.

They loved it. They said "We wish every company would come and talk to us like this!" (They also said "Would you be willing to give talks to other startups?" Which we were, of course, happy to do…)

[To give credit where due, applying agile software development techniques to regulated medical software is not a new concept. In fact, Bakul Patel, who is leading the new FDA Pre-certification Pilot, was the author of AAMI TIR45, which was inspirational to us. He also contributed to many other great guidance documents like Mobile Medical Apps (MMA) and Medical Device Data Systems (MDDS), which have made things way better for companies delivering digital health solutions.]

We also said to the FDA, and we still say every day, that we do not have all of this figured out. We’re not even remotely close. Any company that says they are crushing their FDA quality system, much less their software development process, is a company that is not learning and adapting and making their process better over time. Again, iteration is key.

Which brings us to the FDA Pre-certification pilot program. The core goals of this program include developing a framework that "enables a modern and tailored approach that allows software iterations and changes to occur in a timely fashion" and "ensures high quality medical product software throughout the life of the product by enabling companies to demonstrate their embedded culture of quality and organization excellence".

That sounds pretty awesome to us, and we were honored to be selected to join the other eight companies in the program to help the FDA develop this new framework.

We had our first official meeting with the FDA last week. This is us in our offsite meeting room, and that's them on my iPhone, and cheesy poofs on the left…

During this kickoff meeting, we shared our story, and the FDA shared the goals of the program (presentation). They then followed up with a draft of a bunch of questions that they hope will draw out how we approach building our software. As you can see, the framework focuses on "Excellence Principles" like Patient Safety, Product Quality and Cybersecurity, and within each excellence principle there are "Common Validating Principles" from an organizational, customer, learning/growth and process perspectives. The goal of the program is to come up with key performance indicators (KPIs) to help evaluate the organization along these dimensions. The FDA made it clear that that these questions are still draft, and that they want to work with us and the other companies to come up with the right questions and KPIs.

We are currently scheduling an in-person visit with the FDA, which is only a little funny because we are a distributed software organization without a "facility" in the tradition sense...

We will continue to share everything that we learn, and we will be completely transparent about what transpires. If you have any feedback on the process or that you think we should share with the FDA, please shoot us an email or leave a comment and let us know!

Cheers!Howard

PS: While we are on the topic of the FDA, HUGE congratulations to Courtney Lias and Stayce Beck, who recently won a Service to America "Sammy" medal. Courtney and Stayce have been our primary contacts at the FDA until now, and we can affirm that they have been FABULOUS to work with. They are always pragmatic and wanting to move quickly to get new, safe, innovative treatments to the public.