420 likes | 837 Vues
POCT - Assessing Method Validity Brian Smith (Bedlam Dischargee). POCT - Care" - Clinical - Collection" - GeneralClinical Hospitals, GPs, Pharmacies, Self Test.Workplace, Forces, Occ Health, Pre Employment, Random, Fit for Duty", For Cause", Re
E N D
1. POCT, Assessing Method Validityfor POCT by NON-Lab Staff Brian Smith, Consultant Biochemist Head of Pathology 1990-2003 (Locum 2003-05) Bethlem Royal & Maudsley Hospitals.
UKNEQAS Steering Group for Drug Assays
Probation Service POCT Pilot/Roll Out 1998 -2000
Draft MDA DB 2002 (02 & 03) Commentator
MDA POCT for DAT Evaluation 2002-03
NHS PASA Wk Gp, Nat Contract (DAT POCT)03/05
US NACB Wk Gp, Evidence Based Practice for POCT,Lab Med Practice Guideline (Drugs & EtOH) 2003-2006
2. POCT - Assessing Method ValidityBrian Smith (Bedlam Dischargee) POCT - Care - Clinical
- Collection - General
Clinical Hospitals, GPs, Pharmacies, Self Test.
Workplace, Forces, Occ Health, Pre Employment, Random, Fit for Duty, For Cause, Return to Work.
Schools, Prisons (Mandatory,Voluntary), Probation,
Police (Roadside, Arrestees, Custody Cells).
Non Human Veterinary. Environmental.
3. POCT - Some Examples Wet Chem, Repeat use Devices (mini-labs). (e.g. Blood Gas Analysers).
Dry Chem Detector, Repeat use Devices. (e.g. Glucose Meters, Rapiscan) ( Vitros ?? ).
Dry Chem, Single use Devices (Stix/Stick Tests) (e.g. Drugs of Abuse, CK/Myoglobin,Uristix).
If Dry Chem - Within Batch and Between Batch Manufacturing Consistency much more important (each strip is Unique).
4. POCT - some examples (cont) Machine Read, Permanent Detector
Blood - Gas,Na,K,Li,ICa,Lact,U,Gluc, INR/PT etc. Breath - EtOH, CO, CO2,O2.
Machine Read, Single Use Dry Chem Detector
Blood - Gluc,
Oral Fluid - Drugs of Abuse(DAT).
Visually Read, Single Use Dry Chem Detector
Serum - Myoglobin, Saliva/Oral F - EtOH,HIV,DAT, Sweat - DAT, Urine - Hb/RBC,Prot,Gluc,Bili,NO2(Bugs) etc, DAT.
5. POCT POCT typically involves Rapid On-Site Testing and Result Interpretation, so that Rapid Actions (Clinical Management) are taken.
Those Authorising, Managing and Performing the POCT, have a Duty of Care to be able to Demonstrate, that their system Produced & Interpreted the results Reliably.
They are both Corporately and Personally Liable for this.
POCT should never be initiated Casually.
6. POCT Corporate (& Clinical) Governance Requires a Formal : -
Cost/Benefit Analysis & Business Plan.
Central & Field Evaluation (your Sites,your Operators).
POCT Management System ( with accountabilities).
System Operating Procedures (SOPs).
Operator/Interpretor Competence Testing and Authorisation (Licence to Kill) - and Continuing Competence monitoring including EQA.
( Mirrors routine systems in Accredited Labs )
7. POCT Evaluation Guidance(Clinical) Official Guidance is given in MDA DB2002 (03)
Management and Use of IVD Point of Care Test Devices.
(MDA Medical Devices Agency) now, the
Medicines and Healthcare products Regulatory Authority (MHRA).
This is the UK Competent Authority for IVDs.
MDA DB2002 (03) has a status similar to the Highway Code it may not be the Law, but it does define Good Practice so if you dont follow it you are wrong.
8. MDA DB2002 (03) User/Purchaser (NOT the Manufacturer) must formally : -
Assess if a device is Clinically useful.
Evaluate the devices (for their particular needs).
(e.g. Ketoacidosis interfering with Glucose Meter result)
Put in place a formal POCT Procurement, Management & Monitoring system for all POCTs
Allow only locally evaluated & Authorised POCTs to be used
(continued)
9. MDA DB2002 (03) - continued POCTs to be operated only by specifically Trained &
Competence Tested staff, Corporately Authorised
(Certified)
to perform that POCT,(& non authorised staff prohibited),
Using a locally written SOP, including IQC & EQA
and incorporating the Manufacturers Instructions
Has a duty to participate in an EQA Scheme
10. POCT Manufacturers Instructions- usually include statements such as : - The user must verify the use of the device for their particular purposes.
Suspect results should be repeated.
Suspect results should be verified by another laboratory method. (How do you know its suspect?)
It is good (laboratory) practice to test Positive & Negative Control samples at appropriate intervals [ less commonly ] and to participate in (External) Proficiency Testing / QA schemes.
11. Selling POCTs Suppliers usually NOT selling to Lab Staff,
& often dont understand (& dont promulgate) : -
Limitations of their devices
Likely Field Error Rates
Official Guidance on POCT
Management , Staffing & Time needed for Testing and the Ancillary activities.
Time, Accountability & , for KSF Certification
Contin Whole System Performance Assessment
12. POCT Evaluation Overview( Assessing Analytical Method Validity only a Minor part ? ) Literature / Info Sources Search
Suppliers Training & Technical Support
Fabric, Safety and Disposal (inc Confidentiality)
Connectivity / Result Communication
Evaluation (and Continuing) Record Keeping
Analytical Performance [Central & On-Site(s) ]
Ease of Use (robustness)
Fault Detection ( including IQC)
Ease & Accuracy of Interpretation
Cost including Management,Fabric,Training etc
13. POCT Literature / Info Source Search(changing Rapidly must be recent) Info from Local Path Lab (or Reference Lab).
MHRA Evaluation Reports (www.pasa.nhs.uk.evaluation/details.asp?id=1093)
Other (Peer Reviewed) Evaluations.
NACB Evidence Based Guidelines for POCTs (2006).( www.nacb.org )
Info from EQAs,
Info from PASA** (www.pasa.nhs.uk) - POCT Managers Database & Procurement Toolkits.
Info from Literature on Known Interferents.
Info from Manufacturers inc list of Users.
14. Suppliers Training & Technical Support In my experience Suppliers (Manufacturers) are NOT prepared to offer Training and Competence Testing to MDA standards,
nor are they prepared to Indemnify the User, for the Trainee as a Certified Operator of their POCT not just for their Device but for Compliance with the SOP for the Whole process.
i.e the User Organisation has to pick up the tab for Training, Continuing Competence Testing & Authorisation & the Management & Cost of this.
15. POCT Fabric Requirements(you are making minilabs so Each Site) Storage - Records, Consumables, Samples, Samples for Reference method testing.
Testing & Recording & Communicating
Space, Security(Staff & Records), Confidentiality, Washable Surfaces, Handbasins, Services / Lighting / Connectivity, (All H&S and Infection Control approved).
Disposal (Clinical Waste Licenses, Confidentiality Patient ID on POCTs/Records)
16. Connectivity Validation Recording, Checking, Releasing & Communicating Results Appropriate Space & Services must be present at each site for this.
MACHINE READ Devices should meet the NCCLS POCT1-A Connectivity Standard (www.nccls.org). This allows the Device to connect to a Central (e.g. Hospital) Network allowing - Automated
Result Recording (patients,QC,EQA), (approved) Operator ID, Operator performance Audit, etc.
VISUALLY READ Devices are more difficult to reliably Read, Record & Performance Manage.
17. (POCT Method Validation) Evaluation Record Keeping for POCT, QC samples and Reference Method
Batch Nos, Expiry Dates, Shelf Life/Storage,
Reference Method EQA data
(i.e. Reference lab must be asked, before Eval)
Precision, Bias (particularly for Odd samples)
All Results (POCT & Ref) Patients, QC, QA, Duplicates.
All Analytical Procedures ( & Results) including All Variant tried ( & Results)
All Training & Competence Testing ( & Results)
both for Central Eval and, for each Site Eval.
18. POCT Evaluation Central Trialwith Test Analyte in Matrix Imprecision, Linearity, Sensitivity all in Duplicate at least.
typically at - Zero, Below, Mid, Above and Way Above (? Hook effects) Ref Range (or Path range), & particularly, +/- any Decision Points.
For Quantitative POCTs
say - Zero ( for LOD), 0.2, 0.5 & 1x bottom ref range (for LOQ), mid, 1, 3 & 10x top of ref range.
19. POCT EvaluationTest Analyte in Matrix (cont) Qualitative & Semi Quantitative POCTs
Usually a Neg / Pos Cut-Off or Threshold (T) or, a Neg , + , ++ , +++ result.
typically test Imprecision at Zero, - 50%T, - 25%T, +25%T, +50%T, +1000%T and at, - 30% & +30% of ++ and at, - 30% & +30% of +++
Check Batch to Batch variation at Zero, Below , Above and Way Above ref range. Zero, -25%T, +25%T, 20xT. - all in Duplicate.
20. Within and Between Batch VariationReal Examples(with Test Analyte) Vitros Salicylate.
Method said, if Over 500 mg/l, dilute sample with zero calibrator.
Found from EQA, that some batches gave a result of < 400 mg/l , when true result was 600mg/l so user unaware that sample even needed dilution.
MHRA Eval of 16 POCTs for Drugs of Abuse.
For two Devices (Acon & DrugSCREEN), some batches gave Negative Opiate results at +25% of Cut Off.
21. Formal Device / Method Evaluation(Parallel Testing) This must involve Parallel Testing of POCTs vs reference (local lab) Method, of a WIDE range of Patient samples plus IQC and EQA samples
- first in a local(Central) Trial , and then by the Operators at Each Site.
Sample Collection (& preparation, if any) is part of the POCT Method. Sample often unprocessed & different to the reference method sample. Duplicate (precision) testing is also needed so larger Patient (& IQC,EQA) samples required during Evaluation.
22. POCT - IQC and EQA(Parallel Testing) Standard Laboratory EQA and IQC samples may not be suitable for POCTs. ( Anti Coags, Preservatives, pH, Viscosity etc)
A permanent Local Lab based Parallel Testing EQA scheme may be a viable alternative. (COST, )
MDA DB2002 (03) recommends that POCT services should be Accredited (e.g. by CPA). (COST,)
The Traceability to National analyte values would be via the Laboratorys EQA performance.
23. POCT EvaluationParallel Testing of Patient Samples Sample Collection and Storage
for Parallel Testing.
Sample Vol for Duplicate testing by POCT(s) & Lab.
Sample AntiCoag/Preservative for POCT(s) & Lab.
Sample Stability (need a library) POCT(s) & Lab.
e.g. INR, POCT - No AntiCoag, Lab - Citrate.
Gluc, POCT - No AntiCoag, Lab - F/Ox Plasma
Urine Cannabinoids, Analyte Adsorption (Vol/SA)
24. Parallel Testing Patient Samples (cont) Concentration as wide a range as possible particularly above & below any decision points.
Specificity
Analyte Variants (from Patients & EQA samples) e.g. Chole, - HDL/LDL,(& different TriG concs).
HbA1c, - Hb variants.
Analyte Precursors / Metabolites e.g. Heroin, 6MAM, Morph, M3G, M6G etc.
any Ethnic, Gender or Age Variants
25. Parallel Testing Patient Samples & Other (spiked) Interferents Matrix Variants ( Look at Method Principle & Matrix)
General
Blood - Hct, Viscosity, Haemolysis, TriG, Bili,
Urine - Urea, pH, Colour, Turbidity, Precipitates.
Possible (U Gluc) Fruc, Suc, Lactulose, Vit C etc.
Known (from other Methods)
Na conc (Li), DLIS (Digoxin), Labetalol (Amfets)
26. POCT Method Robustness Evaluation(Central Trial AND Site Trials Effect on Results of Variations in : -
Device Storage, Preparation, Humidity.
Sample Volume .
Device/Testing Temperature .
Test Timings .
Lighting (Time of Day will cause changes).
Ease of Fault Detection (IQC would help) .
Ease of Reading result.
Ease of Interpretation, Recording, (etc)
27. POCT IQC Samples & Test Frequency MDA/MHRA advice is to use 3rd party QC samples - after problems with Manufacturers own QCs working, when Patient samples did not.
Use Pos & Neg QC before each batch of Patient samples (reason to believe POCT OK when used). (this is what happens routinely in Lab Testing)
NB . You are testing the Operator and their Technique, under their Field conditions, at the time performed - NOT just the POCT kits when the Batch is opened as suggested by some kit Suppliers/Manufacturers.
28. POCT Method & Error Rates real Example 1 Probation Service (PS) DTTO POCT Pilot, 98-2000.
PS worried about False Results (go to Prison)().
Modified Site, got Clinical Waste Licences, set up Confidential Chain of Custody system.
In Lab, Trained & Competence Tested 2 P Officers & 2 Senior Drug Abuse Specialist Nurses.
Had Written SOPs, + & - IQCs with each batch.
Knew all initial samples would be Lab retested by both same POCT (& Chromatog).
29. Error Rates, Example 1 (cont 1) Despite all the above precautions, Pos & Neg POCT mistakes in 19 of the first 35 sample (54%)
Busy setting(frequent distraction) with 6 Drug results to Visually read & Record per sample.
Operators needed 10-20 mins per sample for - Client & Sample Registration, POC Testing, Interpretation, & Result Recording
Max of 30 samples per day per Operator doing effectively Nothing else.
30. Error Rates, Example 1(cont 2) This left no time for : -
Admin Actions, Client Interviews,
Counselling Therapy or Court Reporting.
By performing POCT, effectively lost one Staff
say 3 / sample in Direct Labour (+ O/H & POCT)
Work pattern changed, - No distractions(phones) until test recorded - & each test fully completed before next started. (US HHS 2004 same).
Error Rate now 6% ( >1 in 20 samples ) !!!
but similar to other UK & US rates.
31. Error Rates, Example1 (cont 3) After 2.5yr Pilot, PS decided to roll out this (POCT) model Nationally in 2000.
(i.e. it was deemed Highly Successful)
Within 18 months, vast majority of PS switched to Lab based testing. (Occasional POCT,few sites).
32. EXAMPLE 2 Similar Experience Set Up POCT for Drug Abuse In-Patient Ward .
(for Nights, Weekends & Bank Hols)
Little Used Why ?
Because POCT cant answer several common Clinical queries and, because of Staffing needed (2 on at night)
but caused a 30% increase in their Lab tests.
33. Example 2, Clinical Queries Qualitative POCT cant answer Urine Opiates can stay Positive for up to 10 days.
Query ? Used on Ward (since admission on 8/1)
Date POCT Opiates Opi/Creat Ratio
8/1 + 24,000 4,363
10/1 + 2,550 230
14/1 + 3,550 1,044*
15/1 + 2,100 288
16/1 + 1,500 183
17/1 + 350 85
* Gap of 3 days but Conc (and Ratio) Increase.
34. Example 2, Clinical Query Qual POCT cant answer Not is he Pos ? But is he More Pos ? Subject DS, Officially smoking Days 1 6
Urine Cannabinoids Positive Days 1 - 8 Days 8.5, 9 Negative Days 9.5, 10, 10.5, 11 Positive Day 11.5 Negative Day 12 Positive Days 13, 14, 15, etc. Negative
? Did he Re-use on Days 9 and 11.5 ???
Again resolved by Quant Cann*/Creat* Ratio
( Two * Quantitative (Lab) Measurements )
35. Example 3 Glucose POCT, 2000Safety Notice MDA SN 9616 issued 1996, says - POCT Operators Must be Competence Tested & use QC Phone call from Adolescent Psych Ward
We need new stix Urgently. (Lab didnt supply)
For which Meter ?. For Our Meter.
A Glucose Meter ?. Yes. Which Type/Make ?.
Dont Know, Cant you send someone over ?
(8 miles away), Get Ward Manager to Phone me.
Glucosense . Will contact Maker for Stix & Stds.
Why are you doing Assays on (Psych) Ward ?
Adolescent (13),DM, misusing Insulin for Attention and to get out of things .
36. Example 3 Glucose POCT (cont 1) ? Training,Supplies,Stds,QC,QA,Record Keeping .
We are all State Registered Nurses
(i.e. Not Psych Nurses). Weve done these for years
(i.e. We dont need Training).
We went through all this with the Sidcup DM Nurse,
who came with the Patient & Showed us .
The Stix (& Makers Stds & Instructions) came.
I took them to the Ward & asked to see Meter
(to check if Stix correct) & Records of Use.
37. Example 3, Glucose POCT (cont 2) Found beautifully written Records of 10pm levels
only. Recorded as 0.7 mmol/l on all 6 occasions
by a Psych HCA (not SRN). [Ward Mgr in meeting]
Phoned Ward Manager.
Do you have an Action Plan ? What Action Taken ?
Why No Daytime Records ?.
Do you know about DoH Good Practice Guidance ?
How can you get 0.7 on a Meter that only measures
down to 1.8 ??? Why are all the values 0.7 ???
38. Example 3, Glucose POCT (cont 3) No Action Plan, The Patient (not the Levels) would
be discussed at the next Clinical Meeting
If we thought the Patient was Hypo or Hyper, we
would call the Duty Dr. (Hadnt Happened).
So, Why are you Measuring Glucose on the Ward ??
Said I would contact the SHO & Consultant.
39. Example 3, Glucose POCT (cont 4) Asked SRN to show me how they used Meter
Humph !!! . - Do my Glucose.
Stick Insufficiently covered with Blood. See 0.7
Meter shows LO, but held upside down,
so read as 07. OH .
40. Example 3, Glucose POCT (cont 5) Immediate DEMAND for Training,Supplies etc.
I Said,
(a)No Budget or Resources
(b)3 Shifts to train, how will they call in Night Staff ?
Youll have to come when we want .
I said NO !!
You will have to send Glucoses to Lab until Staff
Trained & Certified. You are NOT actually using
Ward POCT Levels Now !!! .
41. Example 3, Gucose POCT (cont 6) Will discuss Training with your Unit Manager ()
and with Director of Nursing (State Registration ?) .
Training will have to be in Sessions with several
Staff together (backfill problem) who will have to
be Certified in the use of the particular Meter the
Trust has selected. PLUS, a Ward Manager,
(you) Responsible for SOPs, Training, Re-Training,
Supplies, Prohibiting use by Non-trained Staff etc.
42. Example 3, Glucose POCT (cont 7) We cant possibly do all that .
Then you cant do POCT even by BM Stix.
Then well get the Duty Dr. .
No, They would need Certifying.
OUTCOME
Assays sent to the Lab, inc On Call, (Better Care).
No more Stix were used.
An Action Plan was apparently developed.
Patient discharged (alive) some 6 weeks later.
43. Example 3, Glucose POCT (cont 8) Director of Nursing had discussions with all Senior
Nurse Advisors (responsibilities, advance planning)
Medical Director (Psychiatrist)(Head of Clin Gov)
did not respond. [ Both left within a year ]
Next Time a similar case arose, Ward asked for
Training,QC etc, In Advance !. 4 separate Recorded
Certification/Training sessions (3-4 Staff at each).
[For 1 patient in for 3 months(Psych Ward problem)]