Calibrating and validating a traffic microsimulation model

1. The problem

A microsimulation model is only as credible as the evidence that it reproduces the system it represents. FHWA’s 2019 update of Traffic Analysis Toolbox Volume III states that software default parameters “do not represent a calibrated model” [1]. Before comparing alternatives, the analyst must show with measured data that the base model reproduces observed volumes, travel times and bottlenecks, and that results are not artifacts of a random seed.

Reviews of simulation studies keep returning to four questions: which acceptance criteria were used and who set them, whether the model was checked against data not used to tune it, how many runs were averaged, and whether the warm-up was long enough. This article separates what the governing documents say from what has become habit.

2. Definitions

• Error checking – removing coding errors (geometry, signal timing, demand) before any parameter is adjusted. FHWA treats this as a separate step that precedes calibration [1].
• Calibration – adjusting model parameters so the model reproduces field-measured local traffic conditions as closely as possible [1, glossary].
• Validation – the term is used in two different senses, which is a frequent source of confusion:
  • In the FHWA 2019 glossary, validation is the comparison of the software’s intrinsic predicted performance against field measurements, and the report assumes the software developer has already done it [1, glossary].
  • In project practice and in other guidance, validation means comparing the calibrated model against observed data that are independent of the data used in calibration. The UK Department for Transport’s TAG Unit M3.1 states this directly for highway assignment models [4]. The FDOT Traffic Analysis Handbook (June 2026) requires calibration (and validation) data for microsimulation (Section 4.7) and has a dedicated “Calibration and Validation Report” section (8.6) [3].
• Replication (run) – one execution of the model with a given random number seed.
• Warm-up (network loading) period – simulated time before the analysis period, during which the network fills with vehicles; its outputs are discarded.

3. Method

3.1 What FHWA 2019 asks for

The 2019 update (FHWA-HOP-18-036, April 2019) [1] replaces subjective 2004 language such as “to the analyst’s satisfaction” with criteria “that are statistically valid and derived from observed data”, and replaces the single “average” day with multiple travel conditions identified by cluster analysis [1, foreword; ch. 2].

The calibration chapter [1, ch. 5] has three steps:

1. Identify a representative day for each travel condition: the observed day whose 15-minute profile of each key measure deviates least from the cluster average. Calibrating to a synthetic averaged day is discouraged, because averaging produces unrealistically smooth targets.
2. Build variation envelopes around the representative day for at least two key measures: one travel time or speed on key paths, and one bottleneck measure (throughput or duration). The ~2 sigma band is the representative-day value ± 1.96·σ(t); the 1 sigma band is ± σ(t), where σ(t) is the day-to-day standard deviation in interval t within the cluster.
3. Calibrate model variants until four acceptability criteria are met, individually for each key measure and travel condition, in a single model run:
   • Criterion I – 95% of simulated outputs fall within the ~2 sigma band (with fewer than 20 time intervals, one value may fall outside).
   • Criterion II – two-thirds of simulated results, and both critical time intervals, fall within the 1 sigma band.
   • Criterion III – the average absolute error does not exceed the Bounded Dynamic Absolute Error (BDAE) threshold, defined from the average absolute difference between the representative day and the other days in the cluster.
   • Criterion IV – the absolute value of the average signed error does not exceed one-third of the BDAE threshold (a check for systematic bias).

The recommended sequence is to calibrate bottleneck throughput first (globally, then locally: e.g. mean following headway, reaction time, start-up lost time), then demand and route choice [1, ch. 5].

The calibration chapter of the 2019 update does not use the GEH statistic. The targets are therefore derived from the variability of the observed data at the study site, not from a universal number.

3.2 Where GEH targets come from

The GEH statistic for hourly flows is

GEH = sqrt( 2 · (M − C)^2 / (M + C) )

where M is the modelled hourly volume and C the observed count (veh/h). This is algebraically the same as the form printed in the 2004 Toolbox Volume III, sqrt( (E − V)^2 / ((E + V)/2) ), with E the model estimate and V the field count [2].

The familiar “GEH < 5 for more than 85% of links” comes from Table 4 of the 2004 version of Volume III, presented there as “an example of calibration targets that were developed by Wisconsin DOT”, based on UK guidelines, with the caveat that targets “will vary according to the purpose” of the model [2]. That table also lists:

Criterion (2004 Vol. III, Table 4, Wisconsin DOT example)	Target
Individual hourly link flows within 100 veh/h, for flow < 700 veh/h	> 85% of cases
Within 15%, for 700–2,700 veh/h	> 85% of cases
Within 400 veh/h, for flow > 2,700 veh/h	> 85% of cases
Sum of all link flows	within 5% of sum of counts
GEH < 5 for individual link flows	> 85% of cases
GEH for sum of all link flows	GEH < 4
Journey times within 15% (or 1 min, if higher)	> 85% of cases
Speed-flow relationship and queuing	visual audit, “to analyst’s satisfaction”

Two points matter for reviewers:

• The UK source family is about assignment models. The current TAG Unit M3.1 (May 2026) carries the same flow bands, GEH < 5 for > 85% of cases, and journey times within 15% (or 1 minute if higher) on > 85% of routes [4]. It also says the share of criteria met “should not be interpreted as a target that the model should be constrained to achieve”, and that meeting them does not guarantee a model is fit for purpose [4, paras 3.2.2, 3.3.2].
• Agencies set their own versions. TxDOT’s manual, for example, requires the FHWA 2019 criteria on FHWA-related projects. On other projects, where criteria are set at scoping, it allows a 2004-style approach with tighter numbers: GEH < 3.0 on all state facility segments and ramps, and GEH < 5.0 on at least 85% of local roadway segments [5].

GEH targets are therefore guidelines adopted by agencies, not a national standard; what governs is what the reviewing agency agreed in the methods-and-assumptions document. The FDOT handbook (June 2026) has its own “Classical Model Calibration Targets” (Table 8-3) and a GEH-based volume comparison table (Table 8-4) [3]. Those numbers were not verified for this article and should be read from the handbook directly.

3.3 Number of runs

Stochastic models need multiple replications with different random seeds. FHWA 2019 [1, ch. 6] gives:

N_min = ( t(n−1, 95%) · s / (e · x̄) )^2

where n is the number of initial runs, x̄ and s are the mean and standard deviation of the key measure across those runs, t(n−1, 95%) is the Student’s t value for n − 1 degrees of freedom at 95% confidence, and e is the tolerance error expressed as a fraction of the mean. Practical rules from the same chapter:

• Start with four runs (t(3, 95%) = 3.182) per model sub-variant.
• The tolerance e is taken from observed day-to-day variability at the two critical time intervals, e = t(n−1, 95%) · (s / √n) / x̄, where here n, s and x̄ refer to the observed days in the cluster at that interval (not to the simulation runs), using the larger of the two. A floor of 5% is applied “as a practical guideline”.
• Round N up; repeat for every key measure and cluster and take the maximum.
• If N exceeds about 20, look for model instability (gridlock, coding errors) rather than adding runs.
• The confidence level (95% unless otherwise set) should be decided by the decision-maker, not the analyst, before alternatives analysis.

FHWA’s own example gives N = (3.182 × 1.46 / (0.05 × 32.7))² = 8.07, i.e. 9 runs [1, ch. 6] (reproduced in Python).

3.4 Seeds and warm-up

• Seed-to-seed variation reflects tactical driver behaviour. It is “not indicative of first order system variation”, which comes from travel conditions [1, ch. 6]. Calibration criteria are tested on a single run; large seed-to-seed differences point to coding errors or gridlock, and an unstable run “should not be used” [1, ch. 5].
• Simulation duration is the analysis period plus time for loading, congestion build-up and dissipation; FHWA’s example allows one hour each side, and the extra time “should be determined by examining the data” [1, ch. 3]. A practical check is that vehicles in the network have levelled off before the analysis period.

4. Worked example (hypothetical)

All numbers below are invented for illustration and refer to no real project. Computed in Python; values rounded to two decimals.

4.1 Volume comparison at 8 count stations (peak hour)

Station	Observed C (veh/h)	Modelled M (veh/h)	M − C	% diff	GEH	GEH < 5	2004 flow band met
S1	1,850	1,795	−55	−3.0%	1.29	yes	yes
S2	2,420	2,700	+280	+11.6%	5.53	no	yes
S3	640	705	+65	+10.2%	2.51	yes	yes
S4	1,180	1,150	−30	−2.5%	0.88	yes	yes
S5	3,050	2,880	−170	−5.6%	3.12	yes	yes
S6	410	395	−15	−3.7%	0.75	yes	yes
S7	960	1,105	+145	+15.1%	4.51	yes	no
S8	1,520	1,540	+20	+1.3%	0.51	yes	yes

Example for S2: GEH = √(2 · 280² / (2,700 + 2,420)) = √(156,800 / 5,120) = 5.53.

• Stations with GEH < 5: 7 of 8 = 87.5%, above the 85% example target.
• Sum of counts 12,030 veh/h; sum modelled 12,270 veh/h; difference +2.0% (within 5%); GEH of the sums = 2.18 (< 4).
• The two tests disagree: S2 fails GEH but is within 15%; S7 passes GEH but misses the 15% band by 0.1 point. With eight stations, one station moves the pass rate by 12.5 points, so report every station and explain why S2 is over-assigned (e.g. upstream route choice or capacity) rather than tuning until the count matches.

4.2 Number of runs from 10 initial replications

Network travel time on the key path (s), runs 1–10: 612, 598, 640, 605, 627, 589, 651, 603, 618, 634.

• x̄ = 617.70 s; s = 19.95 s; t(9, 95%) = 2.262.
• Relative 95% confidence half-width of the mean achieved with 10 runs (same algebra as FHWA’s e, applied to the runs): 2.262 × 19.95 / √10 / 617.70 = 14.27 s / 617.70 s = 2.31%.
• Required runs for e = 5% (the FHWA floor): N = (2.262 × 19.95 / (0.05 × 617.70))² = 2.13, rounded up to 3. The 10 runs already done are sufficient.
• If the agency sets e = 2% (tighter than the FHWA 5% floor): N = 13.34, so 14 runs, i.e. 4 more, after which s and x̄ are recomputed and N checked again.
• If FHWA’s four-run start is used (first four runs: x̄ = 613.75 s, s = 18.41 s, t = 3.182), e = 5% gives N = 3.64, so 4; e = 2% gives N = 22.77, which is above 20. That is the point at which FHWA advises checking for instability rather than running more. It also shows why the tolerance has to be agreed before the runs are made.

5. Limitations

• Count matching is not behavioural validity. A model can reach GEH < 5 everywhere through demand adjustments while bottlenecks sit in the wrong place, queues propagate at the wrong time, or lane use is unrealistic. This is why FHWA 2019 requires a bottleneck measure alongside travel time [1], and why visual audits of queuing remain necessary.
• Overfitting. Tuning link-specific parameters or demand until every count matches produces a model that reproduces the base year and nothing else. TAG M3.1 warns against constraining a model, or letting matrix estimation make significant changes, just to meet validation criteria [4]. Independent validation data (different days, locations or measures) are the defence.
• Hourly GEH hides time dynamics. Hourly totals can match while the 15-minute onset and recovery of congestion are missed; the 2019 envelopes work on time intervals for this reason.
• Micro vs meso. On large networks dominated by route choice, assignment-style criteria may matter more than corridor envelopes; mesoscopic and dynamic assignment calibration is not covered here.
• Data contemporaneity. Calibration data should be collected at the same time and locations as the model-development data. The FDOT handbook lists this among QA reviewer checks (Section 4.9) [3].

6. Practitioner checklist

1. Before modelling, agree criteria, key measures, check locations, confidence level and tolerance in the methods-and-assumptions document, naming the guidance followed and the meaning of “validation”.
2. Keep an independent data set (other days, locations or measures) for validation.
3. Finish error checking before touching behavioural parameters; document every change from defaults.
4. Calibrate bottleneck throughput first, then demand and route choice.
5. Set and report the warm-up from data (network occupancy stabilised).
6. Size runs with the FHWA equation per measure and condition; report seeds, x̄, s, t, e, N.
7. Report per-station GEH and differences, travel-time and bottleneck comparisons, and reasons for remaining failures.
8. State software and version so another analyst can reproduce the work [1, ch. 7].
9. Structure the report to the FHWA final-report outline (objectives and scope, approach, data, travel conditions, calibration tests and results, forecasts, alternatives, results, conclusions) [1, ch. 7] or to the agency template, e.g. FDOT Section 8.6 and the reviewer’s checklist in Table 8-8 [3].

For an independent review of a study like this, see peer review or get in touch.

Sources

1. Federal Highway Administration (Wunderlich, Vasudevan, Wang). Traffic Analysis Toolbox Volume III: Guidelines for Applying Traffic Microsimulation Modeling Software, 2019 Update to the 2004 Version. FHWA-HOP-18-036, April 2019. https://ops.fhwa.dot.gov/publications/fhwahop18036/index.htm (foreword, ch. 3, 5, 6, 7 and Appendix B glossary consulted; report record: https://rosap.ntl.bts.gov/view/dot/43570).
2. Federal Highway Administration. Traffic Analysis Toolbox Volume III, 2004 version, Section 5.0 “Calibration of Microsimulation Models”, Table 4 “Wisconsin DOT freeway model calibration criteria” and GEH definition. https://ops.fhwa.dot.gov/trafficanalysistools/tat_vol3/sect5.htm
3. Florida Department of Transportation. Traffic Analysis Handbook, June 2026 edition. https://fdot.gov/planning/systems/systems-management/traffic-analysis; PDF: https://fdotwww.blob.core.windows.net/sitefinity/docs/default-source/planning/systems/systems-management/document-repository/traffic-analysis/traffic_analysis_handbook.pdf (Sections 4.7, 4.9, 8.3, 8.6, 8.7; Tables 8-3, 8-4, 8-8 cited by title only).
4. UK Department for Transport. TAG Unit M3.1: Highway Assignment Modelling, May 2026. https://www.gov.uk/government/publications/tag-unit-m3-1-highway-assignment-modelling
5. Texas Department of Transportation. Traffic and Safety Analysis Procedures Manual (online), Chapter 13 “Microsimulation Analysis”, Section 13.5.2 “Acceptability Criteria” (accessed October 2026). https://www.txdot.gov/manuals/des/tsp/chapter-13-microsimulation-analysis/13-5-calibration/13-5-2-acceptability-criteria.html