The Osentia Test - a screening tool for fracture risk
The background:
When Dr Beattie commenced working on the Osentia project it was backed by numerous small scale preliminary academic studies that demonstrated a strong connection between the proteins in human nails and their bone health. A spin out company was created to develop and commercialise the promising work, which promptly set about designing and undertaking a number of follow on clinical studies in association with a range of 3rd party collaborators. The studies showed promising trends of scientific interest, where neither proving commercially nor clinically useful.
The first task was to review the reports and data acquired from the clinical studies. These made it clear that something useful was happening in the cohort, but it was getting lost in the midst of noise.
The schematic to the left shows that the variability involved in repeating measurements exceeds the average difference between healthy and fracture. What does this mean?
It is common place for one measurement to predict healthy and a repeat measurement to predict high fracture risk
Sampling variability and healthy-fracture contrast
Such a state of affairs is clearly unacceptable for a health screening tool. A strategic plan was drawn up to:
Identify the sources of error within the methodology
Assess what measures were required to control variability
Quantify the tolerance limits of the methodology
The strategy used the results of historical data analysis combined with existing documentation to identify 4 key suspects within 1 month. For each of these error bottlenecks a protocol was drawn up detailing what experiments needed to be carried out and how the different sources of variation could be isolated and controlled.
As part of the strategy the business case was reviewed to understand the required level of regulatory adherence, the demands of the end use case and the standards to which the test must ultimately perform. These then provided a benchmark to assess the outcome of the protocols against.
The protocols developed during the strategy formulation defined how to control the main conflicts in order to optimise the methodology and analysis to minimise error.
The experiments were carried out in controlled conditions in academic laboratories, exposing the methodology to extremes of variation. There is a natural desire in analytical sciences to protect one’s baby, but it is critical to do everything possible to break the methodology. Only then can you understand its true strength and what limitations exist.
Sampling variability and healthy-fracture contrast
Breaking the methodology allows a very high level of confidence to be placed on what conditions and processes will give a reliable, accurate and appropriate test. This may sound drastic, but often it reveals secrets that allow significant commercial gains to be made. It also significantly derisks investment in later stage clinical trials and in developing new technologies or approaches.
Prior to the evaluation the company were locked into using a £125K instrument that took half an hour per measurement. After the method was broken down into the bare bones requirements it was found that a £20K instrument could do a better job in 5 minutes.
The results of the protocols led to a raft of system improvements that :
Cost of technology reduce by factor of 6
Analysis time reduced by factor of 6
Resampling error reduced by factor of 12
Clinical performance improved (above random chance) by factor of 10
A set of detailed system specifications was created to define the precise needs of the test
Thankfully human nails are very stable under even extreme storage conditions and it was possible to re-analyse each cohort of nails using the newly optimised procedures.
A total of 1500 nails across 5 clinical studies needed to be re-measured in triplicate (so 4500 measurements in total).
Using the existing technology with the newly updated methodology this would have taken 5 years to complete. An alternative had to be found.
Old and New
Using the specifications document a shortlist of suitable low cost devices was formed and a set of standard experiments carried out on systems from alternative suppliers. A standard set of calculations allowed comparison of the results across the different platforms and a winner emerged.
A new instrumental system was purchased and handling requirements for the nails were defined around the device’s operating characteristics and the performance targets required to meet the business need of the test. The result of all this activity was
Recording all the data in less than a year on one system.
Refined methodology made it robust to operator error
The clinical trials met the strict standards required to be considered commercially viable
Clever design of measurement protocols allowed
Demonstration of the robustness and reliability of the new test
Updating system requirements based on improved estimates
Streamlining the methodology through identifying possible efficiency savings
The data collected, it was time to evaluate the data and determine if the overall performance was sufficient to warrant ongoing commercial activity.
It rapidly became clear that the data still exhibited large amounts of variation not related to fracture risk. However, this variation was found to be inherent in the population, not a type that was introduced by the methodological approach. The new methodology embraced this by ensuring it collected all the variation possible within each individual.
Key Scientific Results
However, the data needed something unique to handle the level of variation present and to minimise its impact on the final result. The solution proved to be the advanced signal processing methods developed earlier in Dr Beattie’s career. Those interested in the science can see the variation in the paper linked here or can view a detailed introduction to the advanced methods here.
The clinical trials proved to be clinically significant and relevant (see publications list under heading of ‘Fracture’ for scientific references).
The test outperformed competitor products in the tests.
4 manuscripts published in leading international peer-reviewed scientific publications, with several more in preparation, demonstrating a reliable scientific foundation for the test.
The test was launched commercially in autumn 2016
Osentia materials including test kit and contents plus typical reports adapted to the customer’s results
Following the successful completion of the clinical trails the test was commercialised by Crescent Ops Ltd and launched in September 2016. As a critical member of the scientific team developing the test Dr Beattie was heavily involved in a range of activities:
“At the time Dr Beattie was engaged by our team we had ran 3 clinical trials with the aim of progressing our test from a hypothetical possibility to a commercial reality. While the results of these studies were scientifically interesting they lacked the performance needed for a commercially viable screening or diagnostic test. Dr Beattie undertook a rigorous evaluation of our technical approaches and implemented a raft of improvements that transformed the reliability and repeatability of our platform.“
“It is difficult to visualise how Osentia could have been commercially viable without the significant improvements in the core processes resulting from Dr Beattie's work. Subsequently he has been integral to the commercialisation team that developed and brought to market the Osentia osteoporosis screening test kit in the UK. Dr Beattie was highly responsive to the commercial needs of the launch whilst retaining the scientific integrity needed to set up, calibrate and validate the system, complete SOP’s and train the operator staff. This he did admirably.
Being able to run accurate commercial test samples and produce personalised customer reports is only possible through Dr Beattie’s knowledge of the propriety algorithm Crescent developed and his capabilities within modern analytic methods.“
It is clear that when Dr Beattie joined the team, the method was struggling and without significantly better results the commercial endeavour would never materialise. Insufficient attention had been paid to developing the method, trusting that because it had worked on a simple situation that it would work again in a full clinical trial. Without strategic evaluation of the entire methodology and subsequent optimisation and implementation none of the clinical trails would have warranted development of a commercial system for clinical use.
Osentia
