1 / 23

From Population to Individual Drug Dosing in Chronic Illness

From Population to Individual Drug Dosing in Chronic Illness. Intelligent Control for Management of Renal Anemia. Adam E Gaweda University of Louisville Department of Medicine. Challenges in Dynamic Treatment Regimes and Multistage Decision-Making. Overview. Anemia management

amalie
Télécharger la présentation

From Population to Individual Drug Dosing in Chronic Illness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From Population to Individual Drug Dosing in Chronic Illness Intelligent Control for Management of Renal Anemia Adam E Gaweda University of Louisville Department of Medicine Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  2. Overview • Anemia management • Dose-response modeling • Model-based control in drug dosing • Model-free control in drug dosing Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  3. rHuEPO Anemia ManagementBiological vs. clinical Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  4. Anemia ManagementClinical guidelines • Dosing guidelines (NKF – KDOQI) • Maintain Haemoglobin (Hb) between 11 and 12 g/dL ( Hematocrit (Hct) between 33 – 36 % ). • Titration of EPO: “If the increase in Hb after EPO initiation or after a dose increase has been less than 1 g/dL over a 2- to 4-week period, the dose of EPO should be increased by 50%. If the absolute rate of increase of Hb after EPO initiation or after a dose increase exceeds 3 g/dL per month (eg, an increase from a Hgb 7 to 10 g/dL), or if the Hgb exceeds the target, reduce the weekly dose of EPO by 25%. When the weekly EPO dose is being increased or decreased, a change may be made in the amount administered in a given dose and/or the frequency of dosing.” Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  5. Anemia ManagementCurrent state-of-the-art • Anemia Management Protocols (AMP) • Frequency of Hb observation: • Every 4 weeks if Hb within the target • Every 2 weeks if Hb outside of the target • EPO dose adjustment: • Minimum adjustment amount 10% (of current dose) • Maximum decrease 50% (if Hb > 15 g/dL) • Maximum increase 70% (if Hb < 9 g/dL) • Problem with AMP • Based on average response. • Only 1/3 of the patient population achieve the target. • Can we improve the outcome of anemia management by making it patient-specific using control theory and machine learning techniques ? Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  6. Dose-response modelingOverview • In control system design and simulation, a good process model is priceless. • Models of erythropoiesis: • Physiological model (Uehlinger et al. 1992) • PK / PD model(Brockmöller et al. 1992) • Bayesian network model (Bellazzi et al. 1993) • Artificial Neural Network (ANN) models (Martin Guerrero et al. 2003, Gaweda et al. 2003, Gabutti et al. 2006) Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  7. Subpopulation 1e.g. responders(EPO/Hb < a) Model 1 dose data set (batch) Whole population selection Model 1 response response data subsets (batch) dose Subpopulation 2e.g. non-responders(EPO/Hb ≥a) Model 2 Dose-response modelingPopulation vs. subpopulation modeling Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  8. Dose-response modelingExample of response prediction Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  9. Dose-response modelingOpen problems • Prediction seems to “lag” behind the actual value • Do our data allow us to build a model that shows the true effect of EPO on Hb ( Hct ) ? • Let’s estimate a dynamic linear model Hb(k+1) = f( Hb(k), EPO(k) ) Hbm(k+1) = 0.82 Hb(k) + 0.011 EPO(k) + 1.91 • Let’s now estimate a model of ΔHb(k+1) = f( EPO(k) ) ΔHbm(k+1) = 0.015 EPO(k) - 0.23 Both models achieve comparable accuracy. The second model “explains” the dose effect better. Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  10. output distribution absolute prediction error vs. output Dose-response modelingOpen problems • Our data come from clinical treatment (“closed-loop system”) • How does that affect the model ? Martin Guerrero et al. report the same phenomenon. Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  11. Model-based controlModel Predictive Control (MPC) • Rationale for using Model Predictive Control • There is a delay between EPO administration and Hb response(about 17 days – from EPO manufacturer information). • The relationship between EPO dose and Hb increase is nonlinear (monotonically increasing with saturation – Uehlinger et al. 1992). • The effect of EPO continues throughout the lifetime of red blood cells (up to 120 days). • We plan to include constraints on EPO dose (in the future)(such as minimization of the total dose or minimization of dose changes). Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  12. Model-based controlMPC - Schematic diagram MODEL(population) Hb(k+1) = Hb(k) + FNN(EPO(k),EPO(k-1),EPO(k-2)) EPO Hbm CONTROLLER PATIENT Hb EPO* Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  13. Model-based controlMPC Clinical trial - setup • Trial population: • 60 patients: • 30 controls (dosed by physicians) / 30 treatment (dosed by MPC) • 45 African-American / 15 Caucasian • 35 males / 25 females • Average age 58, min 21, max 84 • Trial length: • 8 months • 2 months “wash-out” period / 6 months for outcome analysis • Treatment goal: • maintain Hb at 11.5 g/dL • performance measure: mean absolute deviation from 11.5 Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  14. Model-based controlMPC - Clinical trial results (thus far) Mean |11.5-Hb| Month Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  15. Model-based controlOpen problems • Simulating MPC • How do we accurately represent the mismatch between the model and the patient ? • How do we effectively simulate adverse events ? • Measuring success • We try to individualize the treatment yet we use a mean performance measure – what are the alternatives ? • Individual performance measures (e.g. within-subject StDevof Hb ) ???? • How do we eliminate influence of Hb changes due to adverse events on the performance measure ? Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  16. Model-free controlReinforcement Learning • Drug administration in chronic conditions is a trial-and-error control process that resembles reinforcement learning disease symptoms – initial state (s0) (standard) initial dose – action (a0) k = 1 Repeat (infinitely) evaluate patient (remission/progression/side effects) – new state (sk), reward (rk) adjust dosing strategy – update state-action table/function (Qk), extract policy (k) administer new dose – action (ak) k = k + 1 End Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  17. Model-free controlQ-Learning simulation - Schematic diagram Q-LEARNING AGENT POLICY ()Ri: IF Hb = Hbi THEN EPO = EPOi Hb(s) EPO(a) PATIENT SIMULATOR(subpopulation model)Hb(k+1) = F( Hb(k), EPO(k), IRON(k) ) IRON(disturbance) Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  18. Model-free controlReward function 11.5 11.5 11.5 11.5 11.5 Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  19. Model-free controlQ-table update • Dose-response relationship (EPO to ΔHb) is monotonically increasing with saturation (Uehlinger et al. 1992). • Let’s update multiple entries in the Q-table at a time : • If Hb(k) < 11.5 and Hb(k+1)  Hb(k) or Hb(k) = 11.5 and Hb(k+1) < Hb(k)thenupdate Q( s, a ) for all s  Hb(k) and all a  EPO(k) • If Hb(k) > 11.5 and Hb(k+1) ≥ Hb(k) or Hb(k) = 11.5 and Hb(k+1) > Hb(k)thenupdate Q( s, a ) for all s ≥ Hb(k) and all a ≥ EPO(k) Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  20. Model-free controlQ-Learning - Simulated clinical trial • Trial population: • 200 individuals with various degrees of response to EPO • 100 distinct responders / 100 distinct non-responders • In the first run, all individuals dosed by AMP • In the second run, all individuals dosed by policy updatedon-line by Q-learning • Trial length: • 24 months • Treatment goal: • drive Hb to, and maintain at 11.5 g/dL • performance measure: mean absolute deviation from 11.5 Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  21. Model-free controlQ-Learning - Simulation results Mean |11.5-Hb| Month Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  22. Conclusionsand open problems • We believe that we are on a good path to successfully individualize anemia management using presented techniques. However, we need to address the following: • How do we produce reliable dose-response models that perform well on under-represented data instances ? • What performance measure do we need to use in order to adequately evaluate the success of an individualized treatment ? Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

  23. UofL Division of Nephrology George R Aronoff Michael E Brier Alfred A Jacobs UofL Dept Electrical and Computer Engineering Mehmet K Muezzinoglu Jacek M Zurada Acknowledgments Michael E Brier has been sponsored by Department of Veterans Affairs Merit Review Grant. Adam E Gaweda is sponsored by NIDDK (1K25DK072085-01A2). Challenges in Dynamic Treatment Regimes and Multistage Decision-Making

More Related