Case Note Sections:
The first task was to review the reports and data acquired from the clinical studies. These made it clear that something useful was happening in the cohort, but it was getting lost in the midst of noise.
The schematic to the left shows that the variability involved in repeating measurements exceeds the average difference between healthy and fracture. What does this mean?
It is common place for one measurement to predict healthy and a repeat measurement to predict high fracture risk
Sampling variability and healthy-fracture contrast
Such a state of affairs is clearly unacceptable for a health screening tool. A strategic plan was drawn up to:
The strategy used the results of historical data analysis combined with existing documentation to identify 4 key suspects within 1 month. For each of these error bottlenecks a protocol was drawn up detailing what experiments needed to be carried out and how the different sources of variation could be isolated and controlled.
As part of the strategy the business case was reviewed to understand the required level of regulatory adherence, the demands of the end use case and the standards to which the test must ultimately perform. These then provided a benchmark to assess the outcome of the protocols against.
The protocols developed during the strategy formulation defined how to control the main conflicts in order to optimise the methodology and analysis to minimise error.
The experiments were carried out in controlled conditions in academic laboratories, exposing the methodology to extremes of variation. There is a natural desire in analytical sciences to protect one’s baby, but it is critical to do everything possible to break the methodology. Only then can you understand its true strength and what limitations exist.
Sampling variability and healthy-fracture contrast
Breaking the methodology allows a very high level of confidence to be placed on what conditions and processes will give a reliable, accurate and appropriate test. This may sound drastic, but often it reveals secrets that allow significant commercial gains to be made. It also significantly derisks investment in later stage clinical trials and in developing new technologies or approaches.
Prior to the evaluation the company were locked into using a £125K instrument that took half an hour per measurement. After the method was broken down into the bare bones requirements it was found that a £20K instrument could do a better job in 5 minutes.
The results of the protocols led to a raft of system improvements that :
A set of detailed system specifications was created to define the precise needs of the test
Thankfully human nails are very stable under even extreme storage conditions and it was possible to re-analyse each cohort of nails using the newly optimised procedures.
A total of 1500 nails across 5 clinical studies needed to be re-measured in triplicate (so 4500 measurements in total).
Using the existing technology with the newly updated methodology this would have taken 5 years to complete. An alternative had to be found.
Old and New
Using the specifications document a shortlist of suitable low cost devices was formed and a set of standard experiments carried out on systems from alternative suppliers. A standard set of calculations allowed comparison of the results across the different platforms and a winner emerged.
A new instrumental system was purchased and handling requirements for the nails were defined around the device’s operating characteristics and the performance targets required to meet the business need of the test. The result of all this activity was
Clever design of measurement protocols allowed
The data collected, it was time to evaluate the data and determine if the overall performance was sufficient to warrant ongoing commercial activity.
It rapidly became clear that the data still exhibited large amounts of variation not related to fracture risk. However, this variation was found to be inherent in the population, not a type that was introduced by the methodological approach. The new methodology embraced this by ensuring it collected all the variation possible within each individual.
Key Scientific Results
However, the data needed something unique to handle the level of variation present and to minimise its impact on the final result. The solution proved to be the advanced signal processing methods developed earlier in Dr Beattie’s career. Those interested in the science can see the variation in the paper linked here or can view a detailed introduction to the advanced methods here.
The clinical trials proved to be clinically significant and relevant (see publications list under heading of ‘Fracture’ for scientific references).
The test outperformed competitor products in the tests.
Osentia materials including test kit and contents plus typical reports adapted to the customer’s results
Following the successful completion of the clinical trails the test was commercialised by Crescent Ops Ltd and launched in September 2016. As a critical member of the scientific team developing the test Dr Beattie was heavily involved in a range of activities:
Professor Mark Towler : Professor at Ryerson University, Toronto and Founder of Crescent.
“At the time Dr Beattie was engaged by our team we had ran 3 clinical trials with the aim of progressing our test from a hypothetical possibility to a commercial reality. While the results of these studies were scientifically interesting they lacked the performance needed for a commercially viable screening or diagnostic test. Dr Beattie undertook a rigorous evaluation of our technical approaches and implemented a raft of improvements that transformed the reliability and repeatability of our platform.“
Alan Ransome : CEO Crescent Ops ltd
“It is difficult to visualise how Osentia could have been commercially viable without the significant improvements in the core processes resulting from Dr Beattie's work. Subsequently he has been integral to the commercialisation team that developed and brought to market the Osentia osteoporosis screening test kit in the UK. Dr Beattie was highly responsive to the commercial needs of the launch whilst retaining the scientific integrity needed to set up, calibrate and validate the system, complete SOP’s and train the operator staff. This he did admirably.
Being able to run accurate commercial test samples and produce personalised customer reports is only possible through Dr Beattie’s knowledge of the propriety algorithm Crescent developed and his capabilities within modern analytic methods.“
It is clear that when Dr Beattie joined the team, the method was struggling and without significantly better results the commercial endeavour would never materialise. Insufficient attention had been paid to developing the method, trusting that because it had worked on a simple situation that it would work again in a full clinical trial. Without strategic evaluation of the entire methodology and subsequent optimisation and implementation none of the clinical trails would have warranted development of a commercial system for clinical use.