General info

The results from the Model Validation Kit should be interpreted with care, because it does not explicitly address the question of stochastic nature of observed concentrations. This is further explained in the document Key to the Model Validation Kit (pdf file).

In order to cope with this type of problems John Irwin of the US NOAA proposed an approach, which was discussed and further developed at several Harmonisation conferences. The methodology is not as simple as that of the Model Validation Kit methodology, but addresses some of its shortcomings in a relevant way. Irwin's work acquired an official status in 2000 when the American standards organisation ASTM published it as an ASTM standard guide. It is designated Standard Guide for Statistical Evaluation of Atmospheric Dispersion Models. D6589-00. The standard guide is available through the ASTM web page at a price of $ 35. The first page can be viewed here.

There are some issues with the ASTM procedure that are not fully resolved and deserve further attention. Such issues are discussed in Chapter 11 of the User's Guide to the Model Validation Kit.

The ASTM standard guide contains detailed discussions on the framework and procedures for model evaluation. The framework is general in the sense that it does not assume that one deals with a certain type of model or with a certain concentration variable. However, there is an appendix to the guide, which specifies an example where the framework is used. This example deals with the classic problem of a plume being emitted from an isolated point source.

The software and data used for this example are available through a web page for the ASTM methodology.