Signant Health

Eric Staib

Advancements in computing technology, and data governance improvements within the life sciences, is significantly changing the way organizations conduct business and improve the lives of patients. One such area of technology is Machine Learning (ML). ML has advanced our ability to process and learn from pharmacovigilance safety data. It has also enhanced existing technologies such as myoelectric prothesis control and adapted personalized medicines to treat and/or manage chronic diseases. Such advances place an even greater emphasis on computer software assurance (CSA), data integrity, and regulatory compliance. Life science industry regulators (e.g.., FDA, Health Canada, MHRA, and EMA) and non-profit organizations such as ISPE, are leading the way in industry to establish best practices (i.e., Good Machine Learning Practices).

ML relies heavily on the data it is provided to “learn” and ultimately make accurate predictions and/or decisions.  Systems, with models trained on “bad” data, are not likely to provide the expected results or desired outcomes (e.g., garbage in, garbage out). Industry guidance is centered on the importance and significance of “good data. In a similar manner, the development and ongoing operation of a system utilizing ML should be achieved through a life-cycle approach. Such an approach, or framework, is often iterative and results in multiple cycles of fine tuning a model’s hyper parameters, training/retraining of models, and modifications to algorithms prior to final verification.

Computerized systems using ML must be developed, deployed, and implemented responsibly. There must be considerations taken for human factors such as alert/alarm fatigue, in addition to cybersecurity, privacy, and legal liability. Those responsible for such systems must also guard against bias in both the systems use case(s), algorithm selection, and development. The data that is acquired, prepared, and partitioned to train and verify given models, must also be guarded against bias.

Such systems require multi-disciplinary expertise and an in-depth understanding of how ML shall be integrated, the desired benefits and associated risks.These systems must also be appropriately monitored for performance, re-training, and maintained in a validated state through robust change management. This becomes increasingly difficult with continuous learning systems with autonomously incremental development, and those that may leverage artificial neural networks.

Despite the tremendous gains in industry though ML, there is still along way to go for many in accepting the inherent risks, challenges, and regulatory implications of such technologies. Many companies still hold tight to paper or electronic documents to demonstrate compliance, but others are advancing their mindset to leverage the necessary tools, good software quality engineering practices, and real-time digital reporting to demonstrate fitness for intended use. It’s these trend setters that will reshape the paradigm of traditional computer systems compliance. Gone are the days of carts piled with physical binders containing protocols, reports, etc. and so should the time of electronic representations of such deliverables. Systems utilizing ML are in a constant state of change and so are the elements defining these systems. The time has come to embrace a new way of leveraging information to support their compliance, just as data itself is being used in new ways to improve our lives through Machine Learning.