CE 451 - Urban Transportation Planning and Modeling Iowa State University Calibration, Adjustment and Validation

1. CE 451 - Urban Transportation Planning and Modeling Iowa State University Calibration, Adjustment and Validation Sources: Calibration and Adjustment of System Planning Models Note: Date = 1990 (need to adjust for inflation, other changes) Model Validation and Reasonableness Checking Manual NHI course on Travel Demand Forecasting (152054A)

2. Objectives: • Identify and interpret trends affecting travel demand • Explain difference between calibration and validation • Identify critical reasonableness checks • socioeconomic • travel survey • network • trip generation • mode split • trip assignment

3. Terminology • Model Calibration • Estimate parameters • Match observations (OD, AADT) • Model Validation • Reasonableness checks • Sensitivity checks • Special generators • Screen lines, cut lines, cordons Is the model sensitive to policy options?

4. Planner responsibilities • Actively involve all participants • Modelers • Planners • Decision makers • Public • Fairly present all alternatives • Timely • Unbiased • Identify (clearly) the decision making process • Who, when, and how • Allows input from all interested groups • You must rely on the TDM • Therefore, must be validated • Accurate and easy to understand (documented) www.readysetpresent.com

5. How do you judge a model/recommend improvement? Scrutinize these characteristics: • Data requirements • Logic of structure and conceptual appeal • Ease of calibration • Effectiveness of the model (accuracy, sensitivity) • Flexibility in application • Types of available outputs • Operational costs • Experience and successes to date • Public or private domain availability cio.com

6. Trends Affecting Travel Demand • Planners should monitor the following trends: • Demographics • Composition of the labor force • Immigration and emigration • Regional economic development • Modal shares • Vehicle occupancy • Average trip length • Freight transport • Are trends consistent with assumptions made in the modeling process? Must be aware of trends to ensure reasonable forecasts Image sources: scu.edu; usda.gov; illinois.edu; uwex.edu; mwcog.org; fhwa.gov; transportation1.org

7. How sensitive is travel to fuel price? doe.gov http://www.eia.doe.gov/oiaf/aeo/pdf/trend_4.pdf

9. Tips for building a good model* • Build accurate road network • Use aerial photos behind • Make sure road attributes are correct, esp. traffic • Use hourly counts • Income and auto ownership don’t fully explain travel • age, gender, life cycle and personal interest come into play • Use survey data • visualize these data • Survey can be done cheaply • Cooperation will be good if there’s a good reason for it – mayor sends letter, e.g. • Employer based surveys get good response (but may be biased) • Some will give home addresses, customer addresses, license plates • Use trip chaining (tour based) and activity based trip generation • We don’t know much about attractions – ITE sample too small – do your own • Drive the network using GPS • Get some data and do some statistics to derive your parameters *Howard Slavin, Caliper Corp. 3/13/04 peer review

10. Tips for building a good model* • Some models are completely made up except traffic counts • See if you really believe the counts • Create your OD matrix from ground counts • May be better than trip gen/dist if you “made up” the whole model (no surveys) • TransCAD has a tool for this • If still want to use trip gen/dist, this method can be used to determine K factors • Could also use the row and column totals as the dependent variables in your trip gen model • Examine individual links after model run • Where are the trips coming from and going to that use the link? • In TransCAD, what is the process used to determine this (for a particular link)? • In TransCAD, what is the process used to show where traffic from a particular zone is going to? • Familiarity with your region is helpful *Howard Slavin, Caliper Corp. 3/13/04

11. Sources of Error • Coding • Sampling • Computation (if done by hand) • Specification • Data Transfer • Data aggregation Improper structure of model, e.g., wrong variables

12. Key Concepts • Not enough attention on model evaluation and reasonableness checks • Checks should be performed after each step • reduces error propagation Errors can also “cancel”

14. Evaluation and Reasonableness Checks Overview Complete? Level of Detail? Reasonable? Methodology? Source? Sensitive? Documentation of calibration? Valid for base year? Current? Reasonable?

15. CALIBRATION and VALIDATION are sometimes confused. • Model development is sometimes called calibration or estimation as we are estimating parameters and constants for the particular model structure. • estimating is a statistical process … want high correlation coefficients and significant parameter values • can "import" a model - or borrow structure and parameters from a "similar" area • VALIDATION is checking if the model is accurately estimating traffic volumes by calculated measures (like RMSE) Feedback Loop

16. Model Validation • Validation of new model • Model applied to complete model chain • Base year model compared to observed travel • Judgment as to model suitability, return to calibration if not • Validation of a previously calibrated model • Compare to a new base year, with new … • SE data • Special gen. • Network • Counts “Transportation Conformity Guidelines” (Air Quality) require model validated < 10 years ago

17. Validation suggestions • - Systemwide • - compare traffic counts across … • - Screenlines • (long lines, check major flows) check trip interchange (distribution) between large sections or quadrants • need a survey; local knowledge of commute patterns helps • - Cordon lines (surround a major generator, e.g. university, CBD...) • - Cutlines (shorter, verify corridor flows, fine tuning) • if "importing" should validate all borrowed parameters and constants

18. IT IS VERY IMPORTANT TO HAVE A GOOD COUNT PROGRAM DESIGNED TO SUPPORT VALIDATION! iowadotmaps.com

19. To "calibrate" the model, need an OD database from a survey. This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%) The Calibration and Adjustment manual is not intended to replace good OD data, and is intended more for small urban areas. (and has some old data in it! – more recent data area available in the Barton-Ashman publication).

20. Calibration and Adjustment Steps: 1) verify network and socioeconomic data 2) run the model 3) develop region-wide values (e.g. trips/person, vmt/person) 4) compare region wide values with “Appendix A” values 5) develop screenlines and cutlines 6) compare model results with ground counts for crossings 7) determine problems (system level, local, combination) 8) modify one or more equations, parameters or variables according to chapters on: - networks - trip generation - auto occupancy - trip distribution - traffic assignment

21. Other chapters focus on: - transit - external stations - system vs. local changes - expected vs. required accuracy - conclusions - trouble shooting

22. Network Data Reasonableness Checks • Check Trees for 2-3 major attractions* • Check coded facility types – how used (BPR?)? • Verify speed and capacity look-up table (what LOS used for capacity?)* • Speed adjust (can lower the freeway speed if it is being overloaded – tweak?) • Significant transportation projects – narrative included? Still viable? • Consistency with MTP • Plot (facility types, # lanes, speeds, area types) to detect coding errors* * Items we can check in labs

23. Details 2. Network Errors 2.1 Centroid Connectors - represent local streets - check access (all 4 sides?) - not connected to intersections - make sure they are not blocked by a physical barrier (river, etc.)

24. Des Moines Model Capacity Look-up Table

25. Des Moines Model Capacity Look-up Table (cont.)

26. Des Moines Model Capacity Look-up Table (cont.)

27. Des Moines Model Capacity Look-up Table (cont.)

28. 2.3 Intersection Penalties (check them!) • - most congestion here • - more important in sub-area modeling • - turn penalties • account for congestion • - speed volume function • - can include delay on approach links • - can do it manually for small networks • check for circuity (correct with small turn penalties!)

29. See TransCAD Manual B “Chapter 10: Traffic Assignment with Volume Dependent Turning Delays”

30. 2.4 Intrazonal times • increasing intrazonal trips (in distribution) decreases interzonal trips (useful if too many trips are being loaded on the network) • number of trips is a function of travel time (gravity model) • can adjust travel time on intrazonals • can adjust friction factor curve to produce more shorter trips (which intrazonals usually are) • can change definition of zones (size, land use) • Air quality analysis implications???

31. 3.1 Trip generation - socioeconomic data can be a source of error - initial step is to check system trip totals, compare w/ Table 4 and A1 and A2 (next pages) - if there is a problem, check the system number of dwelling units - still a problem?, check production/attraction rates

32. Trip Generation Calibration Reasonableness checks – compare to other cities, check future trends • Population 503,345 • Households 201,116 • Average Household Size 2.50 • Basic employment 76,795 (33%) • Retail employment 50,465 (24%) • Service employment 101,697 (43%) • Military employment 42,800 • Population per employee 1.81 • Person trips per person 4.26 • Person trips per household 10.65 • HBW attractions per employee 1.44 • HBW productions per household 1.74 • HB shopping attractions per retail employee 5.99 Colorado Springs 1996 Travel Demand Model Calibration

34. Table A2

35. More recent data … From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN

37. 3.2 Income - be sure you are using “real” dollars

38. 3.3 P and A rates • Problems: old, borrowed, small survey • may work OK at the system level, but not for sub-areas • check system-wide values (see tables, next pages) • raise or lower trip generation rates • Person trip or vehicle trip rates used? • we usually have person trip by purpose, but can apply occupancy factor and check against vehicle rates (ITE) • later, screen line counts can be adjusted by varying trip generation rates (post assignment) • check cutlines and cordon counts • coordinate all of the above

42. Non- home

43. Trip Generation CalibrationTypical Values More recent data … • Person trips per household: 8.5 to 10.5 • HBW person trips per household: 1.7 to 2.3 • HBO person trips per household: 3.5 to 4.8 • NHB person trips per household: 1.7 to 2.9 • HBW trips: 18% to 27% of all trips • HBO trips: 47% to 54% of all trips • NHB trips: 22% to 31% of all trips

44. Trip Generation Reasonableness Checks • Examine trip production and attraction models • Form? • sensitivity? • IMPORTANT: keep parameters reasonable (e.g. don't use negative coefficients in regression models just because they provide the best fit.) • If you think you need to use unintuitive parameters, check the whole process... • Check models for … • External-through and external-local trips • Truck trips • To calibrate trip generation and trip distribution, sometimes we may use ... • default values from past surveys • very limited new surveys • census journey to work data (CTPP)

45. Examine trip purposes used … Use more trip purposes? TRIP PURPOSES Scaling Factor HBW low income 0.795 HBW low-middle income 0.823 HBW middle income 0.861 HBW upper middle income 0.908 HBW high income 0.936 HB elementary school 0.733 HB high school 1.991 HB university 0.895 HB shopping 0.698 HB social-recreation 0.945 HB other 0.875 NHB work-related 0.858 NHB other 0.820 Truck 0.985 Internal-external 0.591 Note: each income class is a purpose! Scale survey for participation (relative participation) Colorado Springs 1996 Travel Demand Model Calibration

46. Travel Survey Data Reasonableness Checks • Determine source of travel survey data • Types of survey conducted • Year of survey • Scale survey for participation • If no survey (borrowed) • Check source of trip rates, lengths, TLFD • Is area similar • Geographic area? • pop/HH/empl. characteristics? • Urban density and trans system? • Compare to similar regions and to same region in earlier times: • Person trip rates by trip purpose • Mean trip lengths by trip purpose • HBW longest? HBO shortest? • TLFDs by trip purpose

47. Socioeconomic Data: Check Reasonableness • Review source for estimates and forecasts • Visualize (plot) trends … • Population and household size • Household income • automotive availability • distribution of employment by type (basic, retail, service) • employees per household and per capita … rate of increase is decreasing • Check future household and employment changes by zone

48. 3.4 Special generators • e.g. universities, airports, malls, ... • Use ITE or survey

49. 3.5 trip balancing factors4.0 Auto occupancy • initially, Ps and As should balance to should be 0.9 to 1.1; if not, check your PA rates and socioeconomic data • NHB is usually out of balance • Automobile occupancy • by trip purpose? • Basis? • Constant? • see table 6 and A9 (next pages … are these still good?)