Cloud software tools can be validated under the FDA’s software validation guidance using one of two approaches: continuous validation, which relies on automated tests that run on demand, or periodic validation, which relies on scheduled manual or automated testing. The right approach depends on whether the tool touches a medical device, the team’s technical skills, and cost.
That guidance, “General Principles of Software Validation; Final Guidance for Industry and FDA Staff,” dates to 2002, before smartphones, tablets, and streaming were part of daily computing. It was written to avoid referencing any specific technology or prescribing particular computing techniques, which is why it still applies in today’s era of cloud computing. What is dated is how firms have applied it.
Why the Traditional Validation Approach Does Not Fit Cloud Computing
Validation of software is typically a costly, time-consuming process, with many firms spending four to six months to validate a specific software product. This has led to the most common validation approach being one of “locking down” the software being validated to prevent any version changes that could force a re-validation. This approach limits firms to software that can be installed locally and controlled by internal IT staff. Cloud computing, by contrast, was invented to avoid the cost and complexity of installing and managing software locally, and the growing majority of software development and innovation is now focused there.
Before discussing how to take advantage of cloud computing while still following FDA validation guidance, it is worth describing where validation matters. There are three distinct areas where validation is needed when building a medical device:
- Validating software that is wholly a medical device or is part of a medical device
- Validating software that is used to build, test, or deploy medical device software
- Validating software that is used in maintaining records related to the medical device
This article focuses primarily on the second and third types, though the concepts apply to validating software that is part of a medical device as well.
Cloud Computing Tools and Validation
Current and upcoming generations of software developers have grown accustomed to cloud-based tools for developing software, as evidenced by the popularity of tools and systems like GitHub, Bitbucket, Jira, and the suites included in cloud vendor products like Amazon AWS, Microsoft Azure, and Google Cloud. A developer hired for their cloud tool experience who is then asked to use internal, locked-down development tools will often work around them, syncing their cloud-based work into the internal tool only occasionally. That is not ideal from a productivity, morale, or source code confidentiality standpoint, so it is best to find a way to incorporate the best-of-breed cloud tools into a validated tool set.
One of the main benefits of cloud computing tools (that someone else controls the installation, updating, and maintenance of the software) is also the biggest impediment to overcome when validating cloud software. The cloud vendor controls when new versions are introduced, generally with little or no notice to the end user. Because the software cannot be locked down, there are two general approaches for validating cloud software tools:
- Continuous validation
- Periodic validation
Continuous validation is validation that occurs on a continuous basis, characterized by automated tests that can be executed or triggered on demand and completed in a short timeframe, such as minutes or hours. FDA guidance does not specify whether testing is manual or automatic, only that testing evidence is generated to prove that all tests pass.
Periodic validation typically takes longer to execute, often because some portion is done through manual testing, and is scheduled to occur on a regular but not continuous basis. It is suitable for situations where cloud software validation needs to be refreshed based on a significant event, such as releasing a new version of medical device software, to prove the tool used to create it is still functioning as expected.
A key element of both approaches is that requirements exist documenting the user needs for the software being validated. The tests, whether automated, manual, or performed through code inspection, need to prove that the software meets those defined requirements.
Criteria for Selecting a Validation Approach
Selection of continuous versus periodic validation depends on several factors. One factor favoring continuous validation is whether the software under validation is wholly or partly a medical device. If medical device software is built using cloud computing, periodic re-validation alone is not sufficient, since cloud software version changes can occur at any time, and an unvalidated breaking change could affect the device’s behavior. Likewise, if the cloud software manages record-keeping data about a medical device, an automated approach is preferable to a periodic one, so there is never a risk to the integrity of the records from an unvalidated cloud software change.
Another factor is the technical skill and feasibility of writing fully automated tests of cloud software. A company without developers on staff who can write automated tests, but with verification testers capable of writing manual tests, may find periodic validation a logical approach, assuming the cloud software is not part of a medical device as described above.
A third factor is cost: the cost to develop the initial test suite and the ongoing cost to re-execute it. Automated tests tend to have a higher upfront cost but little or no cost to re-run beyond the infrastructure that supports them. Manual tests that are part of periodic validation tend to have a lower upfront cost but, given the labor required to re-execute them, can be more costly over the long term depending on how frequently re-validation occurs.
Conclusion
Adoption of cloud computing for software development, including medical device software, is here to stay. Fortunately, and due to the foresight of the FDA’s validation guidance, using cloud computing for medical device software is possible, so long as risks are evaluated and an appropriate validation strategy is established.