Performance Evaluation of AERM

Performance Evaluation of AERMOD and ADMS-Urban Models in a Tropical Urban Environment
Manju Mohan1, Shweta Bhati 1 and Pallavi Marrapu 2
1Centre for Atmospheric Sciences, Indian Institute of Technology,
Hauz Khas, New Delhi-110016; (Email: mmohan65@yahoo.com)
2 Center for Global and Regional Environmental Research
The University of Iowa, 424 IATL, Iowa City, USA
Abstract
In this paper an attempt has been made to undertake performance evaluation of the two well-known regulatory models, AERMOD (07026) and ADMS-Urban (2.2) for a tropical city such as Delhi in India. The model is applied to estimate ambient air concentrations of SO2 and NO2 for the years 2000 and 2004 over seven sites in Delhi. Both model evaluation and inter-comparison is performed. It has been found that though both the models have a tendency towards over-prediction, most of estimated values by both models agree with the observed concentrations within a factor of two. The models include all the urban sources (ie. elevated point sources, vehicular traffic, domestic etc.) in a city. Model validation is discussed in the light of emission inventory, requisite meteorological inputs and state-of-the art performance measures.
Keywords: Air Quality Modeling, Model Evaluation, AERMOD, ADMS-Urban, Regulatory Models
1. Introduction
The presence of sulphur and nitrogen oxides in ambient air is known to increase the risk of visibility impairment and respiratory and cardiovascular morbidity as well as mortality (USEPA, 2008). Though, sulphur oxides are present in low levels in ambient air in Delhi, an increasing trend has been observed in nitrogen oxide levels (CES, 2006 and 2007). In a mega city like Delhi where more than 1, 00,000 petrol and diesel based vehicles are added every year to the roads (DES, 2006), air quality management becomes an area of significant concern. Since modeling of pollutant concentrations is an essential tool for air quality management and thus there arises a need to evaluate the available models as suited to local site conditions.
The present study evaluates the performance of USEPA model AERMOD (07026) and ADMS-Urban (version 2.2) for modeling concentrations of Sulphur dioxide (SO2) and Nitrogen dioxide (NO2) in Delhi for two years, 2000 and 2004 for the winter months of November, December January and February. The years 2000 and 2004 correspond to pre and post phase respectively of implementation of compressed natural gas (CNG) thereby represent two different air quality scenarios in Delhi. On a mandate by Supreme Court of India, public transport vehicles in New Delhi were required to switch their fuel to natural gas in an attempt to reduce their air pollution impacts. This switch was initiated in 2001 and was completed by 2003 (Reynolds and Kandlikar, 2008). The estimated concentrations by both models have been compared with the observed values and their performance has been evaluated based on various statistical parameters.

Background of the study
The capital city of Delhi is located at latitude 28° 38′ 17” N and longitude 77° 15′ 51” E with an altitude of 215 m above sea level. With a population of about 13.85 million (DES, 2006), it is one of the rapidly growing megacities in the world. The scenario of population in Delhi is well explained by the fact that Delhi stands sixth among the principal agglomerations of the world in terms of population (Brinkhoff, 2007). This overpopulation causes strain on urban infrastructure that consequently results in deterioration of environmental condition. In recognition of the severity of air pollution due to vehicular, industrial and domestic sources, Delhi has been designated as an air pollution control area by Ministry of Environment and Forests (MoEF, 1998).
The climate in Delhi is mainly influenced by its inland position and the prevalence of continental air during a major part of the year and extreme climatic conditions. The city has three distinct seasons namely summer, monsoon and winter. The summer season (March, April, May and June) is governed by high temperature and hot, high speed winds. The monsoon (July, August, and September) is dominated by rains and since rain acts as a cleaning device for the environment, incidences of rain result in lower levels of pollutants in air. The most important season in Delhi, from air quality point of view, is the winter, which starts in November and ends with the month of February. This period is dominated by cold, dry air and ground-based inversion with low wind conditions, which occur very frequently and increase the concentrations of pollutants. Thus Air Quality Index values in Delhi range from unhealthy to hazardous levels in the winter season (Mohan and Kandya 2007). Based on this premise, the models were used to estimate SO2 and NO2 concentrations in winter months in Delhi.
The Central Pollution Control Board (CPCB), the nodal agency responsible for monitoring and regulating the pollution scenario, carries out regular monitoring of ambient levels of various air pollutants at seven monitoring stations in Delhi viz. Ashok Vihar, Siri Fort, Nizamuddin, Shahzada Baug, Janak Puri, Shahadara and ITO (Figure 1). Shahdara and Shahzada Baug are industrial areas and ITO is one of the busiest traffic intersections of Delhi. Ashok Vihar and Janakpuri represent areas of mixed residential and commercial use. Other monitoring sites are located in residential areas.
Figure 1: Location of Ambient Air Quality Monitoring Stations in Delhi

3. Applied Models
AERMOD was developed at the United States Environmental Protection Agency. The model is used for regulatory purposes in the United States and is a highly recommended model in many countries. In India, ISCST is still recommended by Ministry of Environment and Forests, Government of India (EIA Manual, 2001) for impact prediction in air environment. This may be due to lack of sufficient model validation studies for AERMOD in rural and urban areas in India as also the lack of detailed meteorological data required by this model.
AERMOD models a system with two separate components: AERMOD (Aermic Dispersion Model) and AERMET (AERMOD Meteorological Preprocessor). The AERMET is the meteorological processor for the AERMOD. Input data for AERMET includes hourly cloud cover observations, surface meteorological observations such as wind speed and direction, temperature, dew point, humidity and sea level pressure and twice-a-day upper air soundings. Meteorological data is accepted from multiple heights and wind, temperature, and turbulence are treated as vertical profiles.
AERMOD calculates the convective and mechanical mixing height. Plume growth is determined by turbulence profiles that vary with height. Under unstable conditions, AERMOD plume displacement is caused by random convective velocities. AERMOD is capable of estimating pollutant concentration from point, line and area sources. Sources can be individually modeled as rural or urban. The model incorporates the effects of increased surface heating from an urban area on pollutant dispersion under stable atmospheric conditions. This treatment is a function of city population.
ADMS-Urban (Atmospheric Dispersion Modelling System) was developed by Cambridge Environmental Research Consultants, United Kingdom. It is widely used for regulatory purposes in the United Kingdom and other countries across the world. It is a model of dispersion in the atmosphere of pollutants released from industrial, domestic and road traffic sources in urban areas. The model incorporates parameterization of boundary layer based on Monin-Obukhov Length and boundary layer height. This local Gaussian type model is nested within a trajectory model for areas beyond 50km × 50km. In this model non-Gaussian vertical profile of concentration is created in convective conditions, which allows for the skewed nature of turbulence within the atmospheric boundary layer that can lead to high surface concentrations near the source.
4. Data
Data for ambient air concentrations of SO2 and NO2 was collected for the years 2000 and 2004 in terms of average daily concentrations from the seven monitoring stations of CPCB in Delhi. Hourly values of meteorological data were obtained from Indian Meteorological Department (IMD) for the time period of two years under study i.e. 2000 and 2004. The upper air data was accessed from online global Radiosonde Database of National Climatic Data Center (NCDC) of National Oceanic and Atmospheric Administration (US-NOAA). The estimate of emissions for the year 2000 was based on the study by Gurjar et al, 2004. For year 2004, data for emission estimates has been collected from different sources such as Delhi Statistical Hand Book-2006 and government agencies like Central Pollution Control Board and Central Electricity Authority.
5. Methodology
Both AERMOD and ADMS-Urban were used to predict 24 hour average and monthly average concentrations of SO2 and NO2, by using the meteorological data and emission inventory for the winter months of the year 2000 and 2004 for Delhi. Three source categories have been considered: Transportation Sector, Power plants, and other sources which includes domestic and waste sector. In the present study, a grid network was constructed which comprised of 173 cells (2 km X 2 km) covering 26 X 30 sq km area of Delhi, where most of the urban activities take place. This area covers all the sources, receptors, seven monitoring stations and most part of urban Delhi. Estimation of emissions for the year 2000 is based on an earlier study by Gurjar et al, 2004. The data for emissions of year 2004 has been gathered from the sectors such as transportation (CPCB, 2006), power plants (CEA, 2006) and other i.e. domestic and waste etc. (DES, 2006) sources were calculated for each 2 km x 2 km cell of the grid. Methodology for the preparation of the gridded emission inventory is based on an earlier work by Mohan et al, 2007.
The emissions from small scale industries have not been taken into account for this study because, according to a Supreme Court decision in 1996, polluting industries in Delhi were closed in year 2000 and other non hazardous units were relocated and there is absence of factual information about emissions of relocated industries.
The models were run for two types of receptor options: (i) for discrete specified points i.e. for the location of monitoring stations so that comparisons between estimated and observed concentration could be made; and (ii) over the entire grid network of Delhi. The output was generated in the form of 24 hour average and monthly average concentrations.
Data for observed ambient air concentrations of SO2 and NO2 was collected for the years 2000 and 2004 in terms of average daily concentrations from the seven monitoring stations of CPCB in Delhi. The monitored daily averaged air quality data of the above pollutants for seven stations in the city was used for comparison with the model results. Analysis was performed both for daily averaged and monthly averaged concentrations. Statistical performance measures used in earlier studies related to model validation (Hanna et al, 1993, Mohan et al., 1995), were used to evaluate the performance of the models. These parameters include

Fractional Bias (FB): The fraction bias is a non-linear operator which is used to represent the relative difference between model and observation in a bounded range (Cooper, 1999). Since fractional bias is a symmetrical measure and a dimensionless number, it is convenient for comparing the results from studies involving different concentration levels, or even different chemical parameter values for the fractional bias range between 2.0 which indicates extreme under-prediction to -2.0 which is extreme over-prediction (Radonjic et al, 2004).

Normalized Root Mean Square Error (NMSE): Normalized mean square error emphasizes the scatter in the entire set and is an estimator of the overall deviations between the observed and predicted values (Kumar et al, 2006). Smaller values of NMSE indicate a better performance and it is not biased toward models that over-predict or under-predict.

FAC2: FAC2 determines fraction of predicted concentrations within factor of two of observed concentration. Ideal value of FAC2 is 1.

Index of Agreement (IA): It is the ratio of the cumulative difference between the model estimates and the corresponding observations to the sum of two differences between the estimates and observed mean and the observations and the observed mean. The index of agreement is a measure of how well the model estimates depart from the observed mean matches, case by case, the observations’ departure from the observed mean (Doty et al, 2002). The index of agreement has a theoretical range of 0 to 1, the latter score suggesting perfect agreement.

RMSE, S-RMSE and U-RMSE: The root mean square error (RMSE) is a measure of difference between the observed and predicted values. RMSE can be further analyzed as composed of systematic and unsystematic error. A measure of the model’s linear (or systematic) bias may be estimated from the systematic root mean square error(S-RMSE) and a measure of the model’s unsystematic bias is given by the unsystematic root mean square error. The unsystematic difference is a measure of how much of the discrepancy between estimates and observations is due to random processes or influences outside the legitimate range of the model. A “good” model will provide low values of the root mean square error, RMSE, explaining most of the variation in the observations. The systematic error, S-RMSE should approach zero and the unsystematic error U-RMSE should approach RMSE since:
RMSE2 = U-RMSE2 + S-RMSE2
The RMSE, however, may be a misleading overall measure of model performance. Since large errors are weighted heavily, these can produce large RMSE even though the errors may be small elsewhere.

Geometric Mean Bias and Variance: Geometric Mean Bias (MG) and Geometric Mean Variance (VG) are measures of dispersion which find application when values in a data set follow a log-normal distribution. A perfect model would have both MG and VG equal to 1.0
6. Results and Discussions
A comparison of monthly average observed and estimated values of SO2 and NO2 at all seven monitoring stations of Delhi for years 2000 and 2004 by both ADMS-Urban and AERMOD is depicted in Figures 2 and 3. It can be seen that both the models have a tendency towards over-prediction of the concentrations.
The results for performance of statistical indicators can be seen in Tables 1 to 4. Summarily, FB varies from -0.88 to -0.04, NMSE varies from 0.15 to 2.36, MG varies from 0.46 to 2.1, IA varies from 0.04 to 0.45, VG from 1.18 to 8.96 and FAC2 varies from 0.35 to 0.91 in terms of predictions by AERMOD. For ADMS-Urban predictions, FB varies from -0.79 to -0.18, NMSE varies from 0.15 to 0.98, MG varies from 0.44 to 1.24, IA varies from 0.07 to 0.49, VG from 1.17 to 4.96 and FAC2 varies from 0.35 to 0.96.
To determine the reliability of the model, the criteria used is as set in a study by Kumar et al (1993) and Chang et al (2004). According to Kumar et al (1993), the performance of the model can be deemed acceptable if;
NMSE < 0.5 and -0.5 < FB < +0.5
In case of NO2, the above criteria were satisfied for results of only two stations (Ashok Vihar and Janakpuri) for the year 2000 and for three sites (ITO, Janakpuri, Shahdra) for the year 2004 for

Table 1: Performance of statistical indicators for NO2 concentration predictions by ADMS-Urban at different monitoring sites in Delhi. (Figures in brackets indicate ideal values)

Parameter Name Ashok Vihar ITO Janakpuri Nizamuddin Shahdra Shahzada Baug Siri Fort 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 Index of Agreement
(1) 0.29 0.18 0.35 0.49 0.24 0.18 0.14 0.16 0.27 0.09 0.07 0.11 0.41 0.41 Fractional Bias
(0) -0.36 -0.79 -0.52 -0.31 -0.47 -0.49 -0.67 -0.65 -0.59 -0.36 -0.79 -0.77 -0.65 -0.46 NMSE
(0) 0.21 0.83 0.44 0.18 0.35 0.45 0.59 0.52 0.49 0.25 0.98 0.92 0.54 0.26 Geometric Mean Bias
(1) 0.75 0.44 0.66 0.77 0.70 0.73 0.56 0.53 0.60 0.74 0.59 0.52 0.53 0.63 Geometric
Variance
(1) 1.24 2.34 1.64 1.23 1.53 1.80 1.88 1.71 1.76 1.32 3.81 4.96 1.75 1.3 RMSE
(0) 15.14 59.85 61.31 49.53 28.81 39.99 43.94 43.01 43.74 25.14 68.54 78.06 27.40 22.00 S-RMSE
(0) 8.04 17.43 34.86 30.45 19.96 25.45 21.63 16.57 21.3 13.44 41.25 48.7 9.83 5.15 U-RMSE
(0) 12.83 57.26 50.43 39.07 20.77 30.85 38.25 39.69 38.2 21.25 54.74 61.0 25.58 21.39 FAC2
(1) 0.92 0.46 0.61 0.41 0.75 0.35 0.42 0.88 0.42 0.89 0.40 0.46 0.60 0.76
Table 2: Performance of statistical indicators for NO2 concentration estimations by AERMOD at different monitoring sites in Delhi.

Parameter Name Ashok Vihar ITO Janakpuri Nizamuddin Shahdra Shahzada Baug Siri Fort 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 Index of Agreement
(1) 0.08 0.16 0.31 0.30 0.24 0.06 0.17 0.21 0.17 0.22 0.11 0.11 0.22 0.24 Fractional Bias
(0) -0.88 -0.69 -0.07 -0.74 -0.21 -0.84 -0.39 -0.39 -0.79 -0.38 -0.09 -0.71 -0.32 -0.40 NMSE
(0) 1.10 0.64 0.34 0.80 1.36 2.36 0.34 0.25 0.93 0.23 0.52 0.74 0.47 0.32 Geometric Mean Bias
(1) 0.48 0.51 1.16 0.49 2.10 0.46 0.83 0.74 0.50 0.71 1.41 0.55 0.92 0.75 Geometric
Variance
(1) 4.04 2.01 1.49 2.66 7.27 8.96 1.42 1.31 2.86 1.29 2.13 2.30 1.89 1.43 RMSE (0) 56.65 50.51 38.45 131.91 36.58 155.59 28.42 26.71 39.87 25.58 28.52 61.06 21.51 26.95 S-RMSE
(0) 34.35 19.91 23.03 54.16 9.43 115.96 21.98 17.20 21.72 15.24 8.64 31.20 15.93 18.49 U-RMSE
(0) 45.05 46.42 30.79 120.28 35.34 103.74 18.02 20.43 33.43 20.55 27.18 52.49 14.45 19.6 FAC2
(1) 0.40 0.46 0.74 0.41 0.35 0.35 0.69 0.87 0.35 0.88 0.63 0.46 0.65 0.76
Table 3: Performance of statistical indicators for SO2 concentration estimations by ADMS-Urban at different monitoring sites in Delhi.

Parameter Name Ashok Vihar ITO Janakpuri Nizamuddin Shahdra Shahzada Baug Siri Fort 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 Index of Agreement
(1) 0.15 0.27 0.24 0.49 0.22 0.09 0.16 0.24 0.24 0.11 0.15 0.24 0.32 0.27 Fractional Bias
(0) -0.75 -0.51 -0.18 -0.48 -0.25 -0.53 -0.65 -0.62 -0.29 -0.21 -0.18 -0.35 -0.42 -0.74 NMSE
(0) 0.73 0.36 0.34 0.39 0.20 0.61 0.54 0.48 0.25 0.15 0.61 0.17 0.27 0.69 Geometric Mean Bias
(1) 0.47 0.61 0.99 0.63 0.89 0.77 0.57 0.54 0.84 0.85 1.24 0.73 0.71 0.47 Geometric
Variance
(1) 2.21 1.49 2.43 1.57 1.25 2.50 1.75 1.67 1.32 1.17 3.73 1.22 1.34 2.01 RMSE (0) 14.75 6.72 13.39 6.62 12.10 11.50 21.30 8.97 11.24 4.30 11.30 4.88 11.05 9.38 S-RMSE
(0) 6.11 2.99 12.75 2.82 5.97 8.90 10.04 3.43 6.16 2.18 3.11 3.13 6.65 2.79 U-RMSE
(0) 13.42 6.02 4.09 5.99 10.53 7.28 18.79 8.29 9.40 3.71 10.86 3.74 8.82 8.96 FAC2
(1) 0.38 0.67 0.79 0.70 0.83 0.61 0.50 0.62 0.83 0.96 0.72 0.91 0.80 0.36

Table 4: Performance of statistical indicators for SO2 concentration estimations by AERMOD at different monitoring sites in Delhi.

Parameter Name Ashok Vihar ITO Janakpuri Nizamuddin Shahdra Shahzada Baug Siri Fort 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 2000 2004 Index of Agreement
(1) 0.09 0.16 0.13 0.45 0.19 0.16 0.13 0.32 0.08 0.08 0.10 0.04 0.06 0.19 Fractional Bias
(0) -0.80 -0.74 -0.66 -0.55 -0.10 -0.21 -0.47 -0.19 -0.56 -0.36 -0.57 -0.55 -0.80 -0.04 NMSE
(0) 1.01 0.93 1.08 0.50 0.71 0.54 0.53 0.15 0.79 0.25 0.94 0.60 1.29 0.41 Geometric Mean Bias
(1) 0.60 0.48 0.62 0.62 1.60 1.22 0.69 0.86 0.62 0.72 0.65 0.61 0.47 0.82 Geometric
Variance
(1) 3.26 2.98 4.41 1.85 2.83 2.23 1.83 1.18 2.66 1.33 3.44 2.08 4.81 1.63 RMSE (0) 19.47 13.85 31.20 8.65 16.65 8.51 20.58 4.30 23.77 6.64 23.87 10.77 33.29 7.21 S-RMSE
(0) 12.42 7.53 16.14 5.25 16.28 7.98 13.79 3.37 13.57 4.26 15.29 6.16 18.35 5.71 U-RMSE
(0) 14.99 11.63 26.7 6.87 3.48 2.97 15.28 2.67 19.52 5.09 18.33 8.84 27.78 4.4 FAC2
(1) 0.38 0.46 0.36 0.65 0.52 0.62 0.61 0.91 0.52 0.85 0.50 0.62 0.39 0.64
ADMS-Urban predictions. For AERMOD estimations, ITO, Nizamuddin and SiriFort satisfy this condition for the year 2000 and ITO, Shahdra and Shahzada Baug for the 2004.

Figure 2: Plot of observed and estimated monthly average NO2 concentrations for the study period
Figure 3: Plot of observed and estimated monthly average SO2 concentrations for the study period
For SO2 estimations by ADMS-Urban, four monitoring stations (ITO, Janakpuri, Shahdra and Sirifort) satisfied the Kumar et al condition for the year 2000 while three sites (ITO, Shadra, Shahzada Baug) satisfied this condition for the year 2004. AERMOD failed to satisfy the limits of this condition on FB and NMSE for SO2 estimations at all sites for the year 2000. For the year 2004, however, three sites (Nizamuddin, Shahdara and Sirifort) were able to satisfy this condition.
Further, according to Chang et al, a ”good” model would be expected to have about 50% of the predictions within a factor of two of the observations and a relative mean bias within ± 30% or FB as within ± 0.3. This criterion is fulfilled at ITO and Shahzada Baug for the year 2000 for NO2 estimations by AERMOD. In case of SO2 estimations, ITO, Janakpuri, Shahdra and Shahzada Baug satisfy the Chang et al condition for the year 2000 for ADMS-Urban predictions. Janakpuri, Nizamuddin and SiriFort fulfill this condition for the year 2004 for AERMOD estimations.
Consistent prevalence of negative Fractional Bias values for both the models indicates that both the models have a tendency towards over-prediction as compared to observed values. Quantile-Quantile (Q-Q) plots also explain the model behavior in terms of over-prediction or under-prediction. In Q-Q plots, the sorted predicted concentrations are plotted against the sorted observed values (i.e. independent of time and position) to determine whether the observed and predicted concentrations datasets come from populations with a common distribution. If the two sets come from a population with the same distribution, the points should fall approximately along the 1:1 reference line. The greater the departure from this reference line, the greater the evidence for the conclusion that the two data sets have come from populations with different distributions (Luhar et al, 2006; Venkatram et al, 2001). Figures 4 to 7 show Q-Q plots for 24 hour average observed and estimated concentrations of SO2 and NO2. The plots reveal the general tendency of both models towards over-prediction at the higher end of the observed concentration distributions. Performance of both models is quite similar for observed concentrations up to roughly one fourth of the maximum observed concentration in all cases. However, except for NO2 concentration estimations for the year 2000, it can be seen that AERMOD has greater tendency towards over-prediction of concentrations as compared to ADMS-Urban.

Figure 4: Q-Q plot of observed and estimated 24 hour average concentration (µg m-3) of NO2 for year 2000

Figure 5: Q-Q plot of observed and estimated 24 hour average concentration (µg m-3) of NO2 for year 2004

Figure 6: Q-Q plot of observed and estimated 24 hour average concentration (µg m-3) of SO2 for year 2000

Figure 7: Q-Q plot of observed and estimated 24 hour average concentration (µg m-3) of SO2 for year 2004

Table 5: Overall performance of statistical indicators for NO2 and SO2 monthly averages of both years and all monitoring sites combined
Parameter Name NO2 SO2 AERMOD ADMS-Urban AERMOD ADMS-Urban Correlation Coefficient
(1) 0.73 0.76 0.71 0.56 Index of Agreement
(1) 0.55 0.57 0.49 0.47 Fractional Bias
(0) -0.51 -0.55 -0.53 -0.47 NMSE
(0) 0.42 0.49 0.46 0.38 Geometric Mean Bias
(1) 0.64 0.57 0.61 0.64 Geometric Variance
(1) 1.84 1.61 1.67 1.55 FAC2
(1) 0.61 0.71 0.69 0.69

Table 5 depicts comparison of statistical measures of monthly average concentration estimation by both models for all the monitoring sites and for both the years 2000 and 2004. In case of SO2, estimated concentrations by AERMOD compare well with observed concentrations in terms of correlation and agreement. However bias and error are lower for ADMS-Urban estimations. The opposite is observed for NO2 estimations where bias and error values are lower for AERMOD predictions while trend comparison between observed and estimated concentrations is better for ADMS-Urban.
Over-prediction by dispersion models can be attributed to various reasons, the prime cause being discrepancies in emission inventory. There are always differences between available factual data and existing real scenario and such differences are, at times, difficult to account for in the inventory. For instance, growth in number of vehicles in the city over the years need not correspond to their actual usage on road in the same proportion. The emission data which serves as an input to the models has been derived from suitable averaging of the annual emission data. Hence, the emissions data for each grid is taken to be constant through out the year. But this is not possible in the real scenario, hence at times, when the emissions decrease, the monitored values might tend to be lower than the modeled values.
For comparison purposes, concentrations were also estimated using emission data available from REAS (Regional Emission Inventory in Asia) for the year 2000. Table 6 lists the statistical analysis of the estimated concentrations. Comparatively lower values of bias and NMSE are obtained for most cases. For NO2, fractional bias lies within the limits as specified by Chang et al (2004). However, an overall under-prediction is observed in case of AERMOD estimations for NO2 yielding a positive fractional bias. In case of SO2, AERMOD estimations still fail to satisfy both Kumar et al (1993) and Chang et al (2004) condition in terms of fractional bias.

Table 6: Comparison of performance of statistical indicators for SO2 and NO2 concentration estimations using emission data from present study and REAS 2000 emissions

Parameter
Name SO2 NO2 AERMOD ADMS-Urban AERMOD ADMS-Urban Present Study REAS Emissions Present Study REAS Emissions Present Study REAS Emissions Present Study REAS Emissions Index of Agreement
(1) 0.154 0.161 0.256 0.260 0.498 0.544 0.559 0.784 Fractional Bias
(0) -0.681 -0.637 -0.331 -0.283 -0.256 0.185 -0.570 -0.184 NMSE
(0) 0.965 0.893 0.396 0.367 0.606 0.644 0.504 0.20 Geometric Mean Bias
(1) 0.711 0.746 0.871 0.915 1.088 1.632 0.637 0.955 Geometric
Variance
(1) 3.661 3.354 2.143 2.230 2.547 5.136 1.862 1.324 RMSE (0) 10.45 9.81 13.98 13.13 38.24 28.78 50.47 23.32 S-RMSE
(0) 5.658 5.389 9.308 8.865 20.826 19.262 31.574 10.037 U-RMSE
(0) 8.785 8.196 10.434 9.691 32.077 21.385 39.375 21.050 FAC2
(1) 0.53 0.54 0.72 0.77 0.66 0.61 0.61 0.91
Hanna et al. (2001) compared the results of ADMS and AERMOD to five sets of field measurements, which represent a cross-section of scenarios common in modeling studies. Though, in general both models performed well for all scenarios, the ADMS performance was slightly better than the AERMOD performance. On average in the study by Hanna et al (2001), ADMS under-predicted by about 20% and AERMOD under-predicted by about 40%, and both had a scatter of about a factor of two. Approximately 53% and 46% of the ADMS and AERMOD predictions, respectively, are within a factor of two of observations in their study. Overall, both ADMS and AERMOD models tend to under-predict the mean and maximum concentrations. Hall et al (2001) compared applicability of AERMOD, ADMS and ISC for regulatory purposes. Overall, ADMS produced maximum concentrations that were a little higher than AERMOD. However the differences in many individual cases were quite large by regulatory standards. In the present study, 53 % and 67 % of AERMOD predictions lie within a factor of 2 for SO2 for the years 2000 and 2004 respectively, while 72% and 69% predictions lie within a factor of two for the years 2000 and 2004 respectively for ADMS-Urban. For NO2, 66% and 60 % estimations lie within a factor of 2 for the years 2000 and 2004 for AERMOD estimations. 61 % and 74 % estimations lie within FAC2 limits for the years 2000 and 2004 respectively for NO2 estimations by ADMS-Urban.
The difference between estimated concentrations by both models arises due to processing of the meteorological data which result in different estimations of the depth of boundary layer. Since winter months are characterized by low boundary layer conditions, the estimated concentrations are very sensitive to any change in this parameter. Meteorological processing in AERMOD modeling system includes upper air soundings and many times the quality and unavailability of upper air data may affect the model computations. ADMS-Urban is dependent only on surface data for boundary layer parameterizations. This also reveals the fact that sophisticated parameterizations are not always helpful in improving the model performance for the lack of appropriate good quality data. On the whole, both the models have comparable performance and the differences between estimated concentrations are likely due to the processing of the meteorological data and many times the quality and availability of upper air data.
7. Conclusions
> Performance evaluation of two commonly used regulatory air quality models across the world namely AERMOD (07026) and ADMS-Urban (2.2) have been successfully performed using state-of-the-art practice.
> Additional case studies for model performance evaluation always enhance the credibility of the models for both model users and developers. It is helpful from the standpoint of the modeling community targeting their application in tropical urban areas as well as to provide insight for further improvement of these models.
> Agreement of monthly average estimated concentrations with observed monthly average concentrations is better as compared to 24 hour daily average concentrations.
> Both the models have a tendency towards over-prediction of concentrations. Model tendency and to an extent assumptions in emission input, can be a possible cause.
> AERMOD has a greater tendency towards over-prediction of concentrations as compared to ADMS-Urban. However both models perform similar for lower end of the observed concentrations and the differences are largely attributed to their deviations from higher end of the concentrations.
> Greater differences in these models for high concentrations are likely due to differences in the treatment of atmospheric stability conditions as highly stable conditions are associated with higher concentrations. Further work is required to understand the differences
> Comparable performance of both AERMOD and ADMS-Urban reveals that use of sophisticated parameterizations to describe boundary layer physics in AERMOD do not always help in improving the model performance due to lack of appropriate good quality meteorological data. The surface layer parameterizations based on similarity theory that requires only the surface data, are often available and of good quality has worked equally well in ADMS-Urban.
> Overall, considering the statistical evaluation, both the models can be said to have performed satisfactorily for both SO2 and NO2.

Acknowledgements
This research was partially supported by Ministry of Human Resource Development (MHRD), Govt. of India as a research grant. We thankfully acknowledge India Meteorological Department (IMD), Delhi, for providing the meteorological data. We also thank Central Pollution Control Board (CPCB), Delhi for making available the air quality data.

References
* Brinkhoff Th., 2007: The Principal Agglomerations of the World, http://www.citypopulation.de, 2007-09-30
* Central Electricity Authority (CEA), 2006: All India Electricity Statistics: General Review 2006, Ministry Of Power, Government of India.
* Central Pollution Control Board (CPCB), 2006: Vehicular Air Pollution in Delhi-A Preliminary Study, Government of India.
* Chang J.C., Hanna S.R., 2004: Air quality model performance evaluation, Meteorology and Atmospheric Physics (2004) 87, pp 167-196
* Compendium of Environmental Statistics (CES), 2006 and 2007: Ministry of Statistics and Programme Implementation, Government of India
* Cooper N.S., 1999: A review of the ‘Model Validation Kit’ (BOOT) and the draft ASTM validation procedure, The Sixth Harmonisation Conference, Rouen, 11-14 October 1999
* Directorate of Economics & Statistics (DES), Delhi Statistical Handbook- 2006, Govt. of NCT of Delhi.
* Doty K., Tesche T. W., McNally D. E., Timin B., Mueller S. F., 2002: Meteorological Modeling for the Southern Appalachian Mountains Initiative (SAMI), July 2002, Accessed at http://www.tva.gov/sami/met/eval/SAMI_v2A.pdf
* Environmental Impact Assessment, Manual, 2001: Ministry of Environment and Forests,Government of India, http://envfor.nic.in/divisions/iass/eia/Annex5.htm.
* Gurjar B.R., Aardenne J.A., Lelieveld J., Mohan M., 2004: Emission estimates and trends (1990-2000) for the mega city Delhi and implications, Atmospheric Environment 38, 5663-5681
* Hall D.J., Spanton A.M., Dunkerley F., Bennett M., Griffiths R.F., 2001: An intercomparison of the AERMOD, ADMS and ISC dispersion models for regulatory applications, The seventh conference on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes, May 2001, Belgirate, Italy.
* Hanna S.R., Egan B.A., Purdum J., Wagler J., 2001: Evaluation of the ADMS, AERMOD, and ISC3 dispersion models with the OPTEX, Duke Forest, Kincaid, Indianapolis and Lovett field datasets, International Journal of Environment and Pollution 2001 – Vol. 16, pp. 301-314
* Hanna S.R., Chang J.C., Strimaitis D.G., 1993: Hazardous Gas Model Evaluation With Field Observations, Atmospheric Environment (1993) 27A, pp 2265-2285
* Kumar A., Dixit S., Varadarajan C.,Vijayan A.,and MasurahaA. , 2006: Evaluation of the AERMOD dispersion model as a function of atmospheric stability for an urban area, Environmental Progress (2006), vol 25 pp 141 – 151
* Kumar A., Luo J., Bennett G., 1993: Statistical evaluation of lower flammability distance (LFD) using four hazardous release models, Process Safety Progress 12 (1993), pp. 1-11
* Luhar Ashok K., Galbally Ian E., Keywood Melita, 2006: Modelling PM10 concentrations and carrying capacity associated with woodheater emissions in Launceston, Tasmania, Atmospheric Environment 40 (2006) 5543-5557
* Ministry of Environment & Forests (MoEF), 1998: White Paper on Pollution in Delhi with an Action Plan, 1998, Government of India.
* Mohan M, Panwar T.S., Singh M.P., 1995: Development of dense gas dispersion model for emergency preparedness, Atmospheric Environment 29, pp 2075-2087
* Mohan M., Dagar L., Gurjar B.R., 2007: Preparation and Validation of Gridded Emission Inventory of Criteria Air Pollutants and Identification of Emission Hotspots for Megacity Delhi, Environmental Monitoring and Assessment 130, 323-339
* Mohan M., Kandya A., 2007: An Analysis of the Annual and Seasonal Trends of Air Quality Index of Delhi, Environmental Monitoring and Assessment 131, 267-277
* Radonjic Z. and Garisto N.C., 2004: An assessment of the suitability of Air Dispersion Models to predict contaminant concentrations in air due to industrial emissions, International Conference on the 20th Anniversary of the Bhopal Gas Tragedy, Indian Institute of Technology, Kanpur, India, December 1 to 3.
* Reynolds Conor C. O., Kandlikar M., 2008: Climate Impacts of Air Quality Policy: Switching to a Natural Gas-Fueled Public Transportation System in New Delhi, Environmental Science and Technology, Web Release Date: 25-Jun-2008; DOI: 10.1021/es702863p
* USEPA, 2008: http://www.epa.gov/air/urbanair/index.html
* Venkatram A, Brode R., Cimorelli A, Lee R, Robert Paine, Steven Perry, Warren Peters, Jeffrey Weil, Robert Wilson, 2001: A complex terrain dispersion model for regulatory applications, Atmospheric Environment 35, 4211-4221
48
Indian Journal of Air Pollution Control Vol. IX No.1 March 2009 pp 47-62