1 / 31

Byeong-Uk Kim * , William Vizuete, and Harvey E. Jeffries

Development, implementation, and application of an improved model performance evaluation and diagnostics approach. Byeong-Uk Kim * , William Vizuete, and Harvey E. Jeffries Department of Environmental Sciences & Engineering, University of North Carolina at Chapel Hill

erins
Télécharger la présentation

Byeong-Uk Kim * , William Vizuete, and Harvey E. Jeffries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development, implementation, and application of an improved model performance evaluation and diagnostics approach Byeong-Uk Kim*, William Vizuete, and Harvey E. Jeffries Department of Environmental Sciences & Engineering, University of North Carolina at Chapel Hill *Georgia Department of Natural Resources 5th Annual CMAS Conference October 17, 2006

  2. Typical SIP modeling Base case Base case emissions Model Performance Evaluation Future projected emissions Future case Base case meteorology Future case emissions Preset controls = + If attainment demonstration failed Future control case Future control case emissions Proposed controls Preset controls More controls until passing attainment demonstration Model performance evaluation (MPE) is the process for assessing the “reliability of model predictions.”

  3. Issues with typical MPE practice • MPE for only (if not, mostly) ozone signals • No systematic evaluation about if models get right answers for likely right reasons • No evaluation of winds with respect to chemical signals • “Waterfall” procedures and no explicit consideration of the impact of model performance on policy choices • No further MPE for model inputs/outputs with respect to proposed policy options once a MPE is done by following the EPA guidance literally • Probable diagnostic evaluation after many ad hoc analyses • Over-dependence on statistical tests • No acceptance for partially useful modeling results • No systematic analysis for graphical measures • Needs for investigation of possible causes of poor performance

  4. The expected outcomes of MPE • Is the formulation of a model scientifically acceptable in general? (i.e. what is the adequacy and quality of model formulation for this use?) • Concerning if models simulate general causes • Does a model replicate the observations adequately? (i.e. does it make predictions that match history?) • Examining if models get right answers for right reasons • Is a model usable for answering specific (e.g. policy) questions? (i.e. does the model fulfill the designed task?) • Assessing if models are usable for target purposes Modified from the original questions in Beck, 2002

  5. Protocol for Regulatory Ozone Modeling Performance Tests (PROMPT) • PROMPT is a meta-protocol; it is a protocol for protocols. • Four phases of evaluation procedures • Does this model show or have all necessary components to produce the phenomena that I can expect from the current best perceptual/conceptual model? (Evaluation Phase One) • Can this model distinguish which precursor(s) to control for ozone reduction? (Evaluation Phase Two) • How precisely can the model estimate control requirements? (Evaluation Phase Three) • What are the possible biases in the prediction and the impact of biases on the policy choice? (Evaluation Phase Four) • Performance measures will be examined in a “progressive” manner. • In later evaluation phases, more information will be investigated than earlier phases of evaluation. • PROMPT emphasizes “day-by-day” and “site-by-site” performance analyses and requires evaluators to examine meteorological inputs, ozone, NOx and VOCs as well as geographical features.

  6. Importance of consideration of control options in MPE • Given two winds, A and B, control options for R can be evaluated if the target emission source is the grey area. • Assuming the emission intensity in the grey area is homogeneous in time and space

  7. Illustration of PROMPT application • Houston-Galveston-Brazoria 8-hour Ozone SIP Modeling (“base1b” was used for this example although “base1c” is the newest case) • Modeling period: 2000-08-16 ~ 2000-09-06 • Extensive observational data available through TexAQS 2000 campaign • Almost same period of Houston-Galveston Mid-Course Review (MCR) modeling for the 1-hour ozone (“base5b”) • In general, this episode shows a very Houston-specific ozone problem; Transient High Ozone Events (THOEs) that are often characterized by hourly ozone concentration changes more or equal than 40 ppb. • THOEs are often caused by epidemic highly reactive volatile organic compounds (HRVOCs) emission events under ozone-conducive conditions. • No official HRVOC emission event record available for 2000 • The possible existence of event emissions in 2000 can be inferred from a study conducted by UT researchers.

  8. Transient High Ozone Event in Houston >10,000 lbs/hr ethylene release at La Porte, (6700 lbs between 11:00 AM and 11:25 AM) 3/27/2002

  9. Modeling domain Ship Channel Galveston Bay 36 km 36 km 12 km 4 km 1 km 1 km

  10. Modeling with improved model inputs • Issues in base5b (1-hour MCR): • Poor surface wind predictions • PBL height and vertical mixing • Over-predictions in NOx, CO, and HRVOCs at surface monitors • Insufficient ozone formation • Major changes in inputs • Meteorology, Emissions, Chemistry, Boundary conditions • Yet, questionable 4-km grid resolution • Does the set of new inputs make base1b (8-hour SIP) more “useful” for assisting decision makers in choosing control options between NOx control and VOCs control (or both)?

  11. RegEvnt1 base5b with CMAQ base5b (1km) base5b

  12. Older 1-h model Newer 8-h model

  13. Clinton (C35C) Base5b (MCR) Base1b (8-hr SIP)

  14. NOx emissions Bush Airport Only 6AM~5PM Decrease ~4 ppb/h downtown/west Houston; increase perimeter counties

  15. CO emissions Decrease ~50 ppb in downtown/west Houston

  16. ETH emissions Only 7AM on 25, 29, and 31 Some increases in downtown/west Houston

  17. NO2 Barchart and time series

  18. CO timeseries Level 4 in model Drops to model boundary condition at night.

  19. ISOP timeseries

  20. ETH timeseries

  21. O3 peak 8/25

  22. base5b Aloft NO2 and CO base1b

  23. Summary of Process Analysis • Overall, OH distribution of reaction with NO2, CO, CH4 ranges 41%~ 48%; very similar new OH radical source strength across HG domain • This is somewhat low compared to other PA results in other areas. • A significant portion of the total OH reaction (=new OH x chain length) is with NO2, CO, CH4, and other non-NO oxidizing paths. • The absolutely maximum amount of O3 that can be formed at the four sites ranged from 127 ppb to 150 ppb minus the emitted NO which ranged from 22 to 123 ppb, thus limiting chemical ozone to values between 36 and 103 ppb of ozone. • Thus, the chemical production of O3 is inversely proportional to the NOx at these four sites. • PAN is predicted to be very low at these sites, so is RNO3.

  24. Major SIP-related question • What are the implications from insufficient radical source? • The deficient radical sources result in insensitivity to VOC precursors and inhibited due to elevated levels of NOx. • With current model configuration, VOC controls will have little to no effect in future control strategies.

  25. Acknowledgement • The Houston Advanced Research Center and the 8-hour ozone Coalition Group • Texas Commission on Environmental Quality: Dr. Jim Smith for base1b and base5b files • University of Houston: Dr. Daewon Byun and Dr. Soontae Kim for Q20 files

  26. Missing FORM? • Observational Evidence

  27. Two potential sources of HCHO • Flares • 98-99% combustion assumed, 1% to 2% emitted VOC composition is assumed same as that fed to flare; rest assumed to be CO2. We assumed that HCHO emitted was equal to VOC emitted. • “Flare case” - Assumed that VOCs fed to flares were partially converted to HCHO and that an amount equal to another 1% was emitted as HCHO. This added a total of 55, 58, and 59 tons on 25th, 30th and 31st. to 13 flares located mainly in the eastern part of Houston • Mobile sources • New data (SWRI, 2005) on Heavy Duty Diesel show that HCHO is 23% of VOC and ethene is 18% of THC. HCHO was 5% of CO. We added HCHO at 4% of low level CO • “Mobile case” - Based on AC obs, assumed that MV emissions did not have enough HCHO. An appropriate factor appeared to be 4% of CO. This added 167, 156, and 145 tons on 25, 30, and 31.

  28. Delta Ozone,ppb O8/25 13-16h Flare case

  29. Delta Ozone,ppb O8/25 09-12h Mobile case

  30. Summary • Flare imputation caused >30 ppb increase in ozone concentrations • CO ratio caused >18 ppb increase ozone concentrations, more distributed • Increased peak ozone at almost every monitor causing 4 monitors to match observations • ~20% increase in new OH and ~30% in ozone production • Still did not match observed HCHO.

  31. base1b

More Related