FDA and Pharmaceutical Manufacturing Research Projects Jeffrey T. Macher Jackson A. Nickerson Co-Principal Investigator

1. FDA and Pharmaceutical Manufacturing Research ProjectsJeffrey T. Macher Jackson A. NickersonCo-Principal Investigators July 21, 2004

2. Presentation Overview • FDA Research Project status report • History • Project Goals • Approach and Expected Outcomes • Current Status • Pharmaceutical Manufacturing Research Project (PMRP) status report • History • Project Goals • Approach and Expected Outcomes • Current Status

3. FDA Research Project

4. FDA Research Project History • Research project idea emerged in Fall, 2001. • Observed significant increases in both number and severity of cGMP violations. • Approached FDA in Spring, 2002. • Dozens of meetings with CBER, CDER, ORA and district offices. • Formalized relationship with FDA in Fall, 2003. • Signed Material Transfer Agreement (MTA) in October, 2003.

5. FDA Research Project Goals • Risk-based assessment of FDA cGMP outcomes. • Identify attributes (currently recorded by the FDA) that impact inspection outcomes. • Transfer “learning” to FDA.

6. FDA Project Approach • Compile and link FDA databases. • Estimate the likelihood of various outcomes: • NAI, VAI, OAI; Warning Letters; Field Alerts; Product Recalls. • based on… • compound/product, facility, firm, FDA district, investigator and training derived factors. • in order to … • evaluate the allocation of investigational resources. • inform effectiveness of investigator training and management.

7. FDA Databases • COMIS (Supplement filings) • DQRS (Field alerts) • EES (Outsourcing) • FACTS (Inspections) • Product Listing • Product Recalls • Product Shortages • Registration • Warning letters • Training

8. FDA-related Performance Outcomes • Inspection outcomes: • NAI • VAI • OAI • Warning letters • Field reports • Product recalls • Product availability

9. FDA-related Key Factors • Product-level and Process-level variables • NDA/ANDA. • Prescription/non-prescription. • Product class. • Product subclass. • Process indicator code. • Supplement history. • Extent of vertical integration. • History of regulatory outcomes for the product. • History of regulatory actions for related products.

10. FDA-related Key Factors • Facility-level variables • Age and Size. • Number of products produced. • Diversity of products produced: • Prescription/non-prescription; Product class; Product subclass; Process indicator code. • Increases/decreases in number of products and diversity of production n years prior. • Time since last ownership change. • Regulatory history (e.g., inspectional outcomes, warning letters and type).

11. FDA-related Key Factors • Firm-level variables • Age and Size. • Number of manufacturing locations. • Number of products produced. • Diversity of products produced. • Number of products introduced in past n years. • Number of NAI, VAI, and OAI at non-focal facilities in the past n years. • Number and type of warning letters at non-focal facilities in the past n years.

12. FDA-related Key Factors • FDA related variables • FDA District Identifier and annual allocated inspection hours. • Inspections • Length of cGMP inspection; Number of investigators; Reason for inspection; Time since last inspection. • Investigators • Annual experience of Pharma versus non-Pharma inspections. • Training: number and frequency of formal training classes. • Policy shifts (e.g., SUPACs)

13. Expected Outcomes of Analysis • Statistical analysis will estimate the probability of various outcomes based on counterfactuals. • Counterfactual analysis allows a risk-based assessment of changes in at least some FDA-related oversight policies. • Results will improve understanding of certain inspection outcomes based on attributes examined. • Risk assessments (in terms of increasing probability of occurrence) will be used to: • Inform oversight choices of FDA. • Identify underlying quality of manufacturers’ production and regulatory processes.

14. Status of FDA Research Project • Phase 1: Exploratory pilot study. • Completed Summer 2003. • Phase 2: Data collection • Complete for CDER and awaiting transfer. • Incomplete for CBER. • Phase 3: Data analysis • Awaiting transfer of data. • Analysis will require 3-6 months.

15. Pharmaceutical Manufacturing Research Project (PMRP)

16. PMRP Project History • Research project idea emerged in Fall, 2001. • Explore whether cGMP violations related to managerial, organizational and technical practices. • Attempt to translate lessons learned from similar research study of manufacturing in semiconductor industry. • Began interviewing manufacturers in Spring, 2002. • Dozens of interviews with pharmaceutical, biotechnology, API, and contract manufacturers. • Launched internet-based questionnaire in Fall, 2003. • Went live in November, 2003. • Ongoing marketing and participation solicitation since. • Expect to close first round of survey shortly.

17. PMRP Goals • Develop standard set of benchmarks for measuring manufacturing and regulatory performance. • Identify managerial, organizational, and technical practices underlying manufacturing and regulatory performance. • Provide confidential “scorecard” to manufacturing facilities on how they perform against anonymous others.

18. PMRP Approach • Develop focused questionnaire of potential factors that influence manufacturing and regulatory performance. • Administer questionnaire over internet on secure-site. • Analyze data collected using a variety of econometric techniques. • Provide summary of findings and facility scorecard to participating manufacturers.

19. PMRP Database • Secured participation from cross section of U.S. and EU manufacturers. • 21 firms. • 60 manufacturing facilities. • Online survey with each manufacturing facility providing detailed data on between 1 and 5 compounds.

20. PMRP Performance Outcomes • Manufacturing Performance • Theoretical / Actual yield. • Batches started / failed. • Cycle time. • Regulatory Performance. • Field alerts/Biologic deviation reports. • Warning letters. • Consent decrees. • Deviations. • Supplement approval.

21. PMRP-related Key Factors • Company / SBU • Financial information. • Demographic information. • Manufacturing Facility • Financial information. • Demographic information (size, location, age, employees, etc.). • Product information (number, type, etc.). • Regulatory inspection information. • Extent of Outsourcing (development, manufacturing, APIs, etc.).

22. PMRP-related Key Factors • Product and Process Development • Location. • Organization. • Timing. • Human Resource Management • Appraisal, Promotion, Mobility, Demographics, Training, etc. • Extent and use of teams. • Deviation and Supplement Management • Extent and use of information technology. • Extent and use of Process Analytic Technology (PAT). • Organization.

23. Status of PMRP Project • Phase 1: Exploratory pilot study • Completed Summer, 2003. • Phase 2: Data collection • First round concluding shortly. • Phase 3: Data analysis • Analysis will require 3-6 months. • Statistical and econometric analysis of data. • Final reports.