Abay health center, Bahirdar, Ethiopia. ©UNICEF
The Modelling Consortium work package on characterisation of tests for recent infection involved benchmarking a variety of approaches to using seroconverter follow up data to optimally estimate the key characteristics of a candidate test under development. Given recent advances in the underlying theory, the application of recently published, generally applicable summary properties, to incomplete data such as obtained in real world follow up studies, is an urgent next step to ensure that test development proceeds against robust performance metrics.
The Modelling Consortium convened an ad hoc working group on July 19 - 20th 2012 to consider preliminary results of the benchmarking exercise, develop a joint manuscript and prioritise remaining gaps in community consensus.
Aims and objectives
The Modelling Consortium initiative on incidence work package had two components, both relating to cross sectional estimation using biomarkers.
Part 1 of the project focused on building consensus on methods to use for estimating the mean duration of recent infection from data of the kind usually available. This was done by collaboratively applying a number of existing and evolving methods to simulated data where the answer is known. Simulated data will capture:
- various possible test dynamics inspired by actual tests
- various patient follow up protocols
- various types of data problems from selection bias to meaningful missingness
Participants developed analysis plans which were applied to these scenarios where appropriate, and detailed performance measures will be obtained. The endpoint is a systematic evaluation of current methodology on the estimation of mean duration of recency, which is expected to be a compelling publication in peer reviewed forum.
Part 2 of the project focused on practical and conceptual aspects of application, which seek to translate the emerging theory into best practise. Again, the process intended to be collaborative and not competitive. Topics up for discussion:
- exploring the regimes and scale of the impact of violated assumptions such as normality, which are made in the computationally lightweight formulas which emerge from the theory
- choosing appropriate means to report uncertainty from various sources
- dealing with signals of data problems – such as confidence intervals which are strangely large or straddle zero.
- exploring issues of study design (sample size / power calculations)
- technical and conceptual aspects of finite population size effects when large surveys are done in small populations.
- subtleties of benchmarking, and harmonisation of estimates from multiple sources – especially the much desired ‘head to head’ comparisons of cohort and cross sectional estimates from ‘the same population’
A paper in relation to this work is currently under review and will be made available on this site once released.