SWGDAM outlines validation requirements for individual laboratories which include concordance, reproducibility, accuracy and precision, sensitivity and stochastic effects and mixture studies. The details of these requirements are informative but balancing of sample numbers versus robust results to establish conclusions and subsequent guidelines is more tricky.
In her presentation at ISHI, Kate Stevenson (a Senior Scientist at the Institute of Environmental Science and Research Ltd (ESR) will describe the lessons learnt from validation studies at the Institute of Environmental Science and Research Ltd (ESR). The goal being an understanding of the performance of each kit or instrument including its limitations when used on casework like samples.
We sat down with Kate and asked her what considerations a lab should think through prior to starting a validation and the differences between validating quantification kits versus STR amplification kits.
Hi Kate, thanks for sitting down with us today! As you mentioned in your abstract, validating new kits can be a tricky process, so it’s helpful to learn what has worked for other labs.
Why can it be challenging when balancing sample numbers versus robust results during a validation?
By using too few samples in a validation, any issues or limitations that may only affect a fraction of samples processed may not be observed during validation or simply get attributed to being an anomaly yet the impact of such issues may present itself more once it’s been implemented into casework with higher sample numbers and may cause downstream issues.
On the flip side, by testing every substrate and sample type at multiple sample concentrations, using every variation on extraction methods and processing in duplicate or triplicate you could have more than a hundred samples for concordance alone-this could be deemed excessive.
Based on your experience, are there certain considerations that a lab should think through prior to beginning a validation?
Planning time for validation is critical for determining what needs to be done in what time frame. Don’t underestimate time frames for interpretation of data particularly when one set of samples is being used for multiple validation experiments.
Try and evaluate different kits from different suppliers on paper. By this I mean, what was included in their experimental work done as part of the developmental validation, is a developmental validation paper available? have other labs published their findings of validation work? This information may assist with considering whether/how the chemistry/instrument can be integrated into current workflow. Consideration should also be given to how the new chemistry or instrument affects IT lab systems and the analysis of data and reporting of results.
Are there any key differences in validating quantification kits versus STR amplification?
Whilst both steps are PCR based and the validation of amplification and quant kits are intertwined; in that a key question is, how do the sensitivities of the kits compare? i.e does zero quant mean zero profiling results, the validation of the two systems do differ in other ways.
As amplification kits are being released with increased loci number, increased number of dyes detected and catered to degraded and inhibited DNA more, validations completed for STR kits should ensure the increased benefits do not result in compromises associated with data quality particularly regarding intra and inter dye balance.
Whilst the quantitation system has also been developed to better capture sample degradation and inhibition, and newer kits are typically able to detect male and total DNA in a sample, care should be taken to establish expected ranges from validation data for total DNA:male DNA ratios, effectiveness of the IPC and where is the value at which point the presence of human DNA can be confirmed.
Were there any common ‘quirks’ that came up when performing your validations? If so, do you have suggestions for other labs who may be experiencing the same thing?
Extraction method (chemistry as well as automation vs manual processing) can influence results obtained with new quant, amplification chemistries or instrumentation. Ensure your validation captures samples extracted using your typical lab methods.
Personally, what was the biggest learning that you took away from the project? How did you become involved?
Anomalies in data may be attributable to a bigger issue so it can be easier to take time to resolve possible issues during validation than stop casework processing to trouble shoot an issue after implementation.
I have been heavily involved in multiple validation projects in our team and it is my experience with previous projects which adds to my awareness of factors to consider as part of a new validation project.
How did you become interested in forensic science?
I learnt about it as a career choice at university and thought it was a great opportunity to apply my passion of science to a role. After 15 years I still have a great amount of job satisfaction and glad I took this path.
What do you feel is the biggest challenge that forensic laboratories are facing today?
Answering questions for court that are beyond current technology. How and when was DNA deposited?
What tips would you give to someone who is just starting out in the forensics field, or what is the best advice that you’ve received?
Ask questions if ever unsure or if you need advice/clarification. I have been able to extend my own knowledge through asking questions and learning information from those more experienced.
For those still deciding whether to attend, what is your favorite thing about ISHI, or what are you most looking forward to this year?
Favorite thing is networking with other forensic biologists who hold a wealth of information from their respective labs. I am looking forward to seeing how technologies are changing and the role of Massively Parallel sequencing in workflows and research.
What’s one thing that others may not know about you?
I have not been in US since 2010 when I last attended ISHI in San Antonio
WOULD YOU LIKE TO SEE MORE ARTICLES LIKE THIS? SUBSCRIBE TO THE ISHI BLOG BELOW!
