TDI-Brooks International, Inc.

Back to Analytical Services

1998 - 2002 NIST Intercalibration Results

10 April, 2003

Figure 1 - Intercalibration Results

In support of marine monitoring measurement programs, the National Institute of Standards and Technology (NIST), in cooperation with the NOAA National Status and Trends Program (NS&T), and the EPA Environmental Monitoring and Assessment Program (EMAP), has conducted yearly interlaboratory comparison exercises to provide one mechanism for participating laboratories (and monitoring programs) to evaluate their quality and comparability of performance in measuring selected organic contaminates in environmental samples.

NIST efforts focus on providing mechanisms for assessing the interlaboratory and temporal comparability of data, and on improving measurements for the monitoring of organic contaminants such as polycyclic aromatic hydrocarbons (PAHs), polychlorinated biphenyl congeners (PCBs), and chlorinated pesticides in bivalve, sediment and fish samples. This program includes the development of improved analytical methods, production of needed NIST Standard Reference Materials (SRMs) and other control materials, conduct of semi-annual interlaboratory comparison exercises, and the coordination of workshops to discuss the results of these exercises and to provide a forum for cooperative problem-solving efforts by participants. Current participants represent multi-laboratory monitoring programs as well as a number of individual programs, and include federal, state/municipal, university/college, private sector and international laboratories. In this performance-based program, each participating laboratory uses the methods currently being used by that laboratory for analysis of similar materials for its program customers. The target analytes are listed in Table 1.

Table 1. Analytes of Interest in NIST Intercomparison Exercise Program for Organic Contaminants in the Marine Environment

Chlorinated Pesticides

hexachlorobenzene

2,4'-DDE

alpha-HCH (alpha-BHC)

4,4'-DDE

gamma-HCH (gamma-BHC, Lindane)

2,4'-DDD

heptachlor

4,4'-DDD

heptachlor epoxide

2,4'-DDT

cis-chlordane (alpha-chlordane)

4,4'-DDT

trans-chlordane (gamma-chlordane)

aldrin

oxychlordane

dieldrin

cis-nonachlor

endrin

trans-nonachlor

endosulfan I

mirex

endosulfan II

Polychlorinated Biphenyl Congeners

PCB No.

Compound Name

8

2,4'-dichlorobiphenyl

18

2,2',5-trichlorobiphenyl

28

2,4,4'-trichlorobiphenyl

44

2,2',3,5'-tetrachlorobiphenyl

52

2,2',5,5'-tetrachlorobiphenyl

66

2,3',4,4'-tetrachlorobiphenyl

101

2,2',4,5,5'-pentachlorobiphenyl

105

2,3,3',4,4'-pentachlorobiphenyl

118

2,3',4,4',5-pentachlorobiphenyl

128

2,2',3,3',4,4'-hexachlorobiphenyl

138

2,2',3,4,4',5'-hexachlorobiphenyl

153

2,2',4,4',5,5'-hexachlorobiphenyl

170

2,2',3,3',4,4',5-heptachlorobiphenyl

180

2,2',3,4,4',5,5'-heptachlorobiphenyl

187

2,2',3,4',5,5',6-heptachlorobiphenyl

195

2,2',3,3',4,4',5,6-octachlorobiphenyl

206

2,2',3,3',4,4',5,5',6-nonachlorobiphenyl

209

decachlorobiphenyl

Polycyclic aromatic hydrocarbons (PAH)

naphthalene

fluoranthene

2-methylnaphthalene

pyrene

1-methylnaphthalene

benz[a]anthracene

biphenyl

chrysene

2,6-dimethylnaphthalen

benzofluoranthenes[b+j+k]

acenaphthylene

benzo[e]pyrene

acenaphthene

benzo[a]pyrene

1,6,7-trimethylnaphthalene

perylene

fluorene

indeno[1,2,3-cd]pyrene

phenanthrene

dibenz[a,h]anthracene

anthracene

benzo[ghi]perylene

1-methylphenanthrene

 

Note that the following are typically reported by exercise participants as the sums of the indicated components:

PAH

chrysene + triphenylene
benzo[b]- + benzo[j]- + benzo[k]fluoranthene
dibenz[a,h]anthracene + dibenz[a,c]anthracene

PCB congeners

*PCB 66 + PCB 95
PCB 101 + PCB 90
PCB 138 + PCB 163 + PCB 164
PCB 187 + PCB 182 + PCB 159
PCB 170 + PCB 190

*Note: Because PCB 66 and PCB 95 can now be separated by a significant number of participants, NIST has changed the table for reporting results so that participants may report these as two separate concentrations or as the sum of PCBs 66 and 95.

B&B Laboratories, (the lab affiliate of TDI-Brooks,) participated in the last four NIST intercalibration exercises for trace organics along with over 30 other submitting laboratories (including the NIST laboratory). For each exercise, NIST prepares and sends blind check samples to each lab, each to be analyzed for selected PAH, chlorinated pesticides, and PCB congeners. Each lab submits the results of these determinations to NIST. Laboratories are assigned numerical identification codes as they submit their results, but are otherwise not identified. Included in each NIST post-analysis report are exercise-assigned values along with the standard deviation, the % relative standard deviation, and the calculated 95% confidence interval of the assigned value for each analyte.

For each annual intercomparison exercise, samples of two natural-matrix-based homogeneous materials derived from the marine environment (not fortified with any of the target analytes) are analyzed by the participating laboratories.  Typical materials, such as mussel or fish homogenates and/or wetted marine sediment, typically have target contaminant levels in the 1 to 15,000 ng/g range.  Different materials have been used each year.

The following guidelines were used by the NIST exercise coordinators for the establishment of the exercise "assigned values" for these exercises.  In essence, the laboratory's performance on concurrent reference material analyses was used to determine if that laboratory's results would be included in the calculation of the exercise assigned value for the unknown material for a particular analyte.  The results reported for the unknown materials from laboratories that did not report results for the reference materials were not used in these calculations.  After the exercise assigned values, assigned values standard deviations, and 95% confidence limits were calculated, all reported results for the test materials were evaluated relative to these exercise "assigned values."

Click here for further discussion about the NIST guidelines used in the establishment of “Assigned Values”. 

Numerical indices (z- and p-scores) were used to assess and track laboratory performance (for accuracy and precision, respectively) and provide a mechanism for assessing the comparability of data being produced by the participating laboratories for target analytes. IUPAC guidelines describe the use of z-scores and p-scores for assessment of accuracy and precision in intercomparison exercises such as these.  These indices assess the difference between the result of the laboratory and the exercise assigned value and can be used, with caution, to compare performance on different analytes and on different materials.

Click here for more information about z- and p-scores. 

Laboratories were assigned numerical identification codes in order of receipt of data with the exception of NIST, which is Lab 1 in these exercises. A laboratory was assigned the same code for each material.  In the NIST reports, the triplicate results, as reported by the laboratories for both the exercise materials and the two reference materials, are reported along with reference values for each of the materials and performance scores (numerical indicators of accuracy (bias) and precision (reproducibility)).

B&B Laboratories was assigned the following identifier numbers for the four NIST intercalibration exercises in which we have participated:

1997 Lab 27 (partial participation)
1998 Lab 31
1999 Lab 16
2000 Lab 9
2002 Lab 27

The NIST exercise coordinators recognized that different programs have different data quality needs. The acceptability of the results submitted by a particular Organics laboratory is decided by the individual program(s) for which the particular laboratory provides data. Typically, the program will use these exercise results in conjunction with the laboratory's performance in the analysis of certified reference materials and/or control materials, and of other quality assurance samples. The exercise results are shown in the NIST report in a number of ways to facilitate their use by these programs in their acceptability assessment. B&B Laboratories has consequently developed an objective method of numerically assessing the relative performance of each participating laboratory. This assessment follows:

For each lab in an annual exercise, we sum the ratings from each of the two z-scores with the p-score. An analyte score rated by NIST as Satisfactory (see discussing of z- and p-scores) was counted as one point. An analyte rated as Questionable or one not reported by the lab was not counted. An analyte rated as Unsatisfactory was counted as one negative point. Using this convention, (1) satisfactory accuracy accounts for 2/3 of the points and precision for 1/3, (2) a lab is neither penalized nor rewarded for not reporting an analyte, and (3) negative overall scores are possible. Summed points are plotted below for each lab, in order of decreasing total. Using the data from the reports issued by NIST, we have calculated and plotted the resulting scores for each participating laboratory. Plots of the performance rankings for the 1998, 1999, 2000, and 2002 exercises are shown in Figure 1. In these plots, the Lab Performance Rank numbers along the X-axis represent the relative order of performance, not the Laboratory number assigned by NIST. As these plots show, our lab has consistently performed well in the NIST intercalibration exercises.




Figure 1. 1998, 1999, 2000, and 2002 NIST Intercalibration results

Copyright © 2003, TDI-Brooks International, Inc. All rights reserved.
Send questions or comments about this site to mk@tdi-bi.com.