Guidelines for Peer Review of Toxicological Profiles

Overview

The toxicological profiles provide ATSDR’s evaluations concerning whether adverse health effects occur and/or at what levels of exposure. Profiles are written with an emphasis on human health effects. They also contain information about health effects in animals, potential for human exposure, and environmental fate that may help the reader to determine the significance of levels found in the environment.

In these profiles, the emphasis is on providing succinct interpretations of the key literature. This distinguishes “profiles” from comprehensive criteria documents. The interpretations are expected to be useful to the informed public and health professionals who need a succinct interpretation of the toxicological data but may not have the resources to gather and consider all of the toxicological data themselves. Specifically, the profiles incorporate ATSDR’s evaluations concerning the validity of particular studies and the inferences that can be made from them. The profile is not meant to contain all of the details necessary to support these interpretations. It is beyond the intended scope of the profile to present extensive details for users to weigh all the evidence themselves; such data are incompatible with the concept of a “profile.” However, the environmental health community has expressed growing interest in the use of systematic review (SR) principles to increase transparency and objectivity of complex environmental questions. Therefore, ATSDR is integrating SR principles into the development of the health effects section of the profiles. This information will provide users with more detail regarding how authors chose the studies included in the profile and how they arrived at their conclusions about exposure health effects.

The authors have been instructed to avoid lengthy descriptions of studies. If there is uncertainty or controversy about a conclusion, however, a more detailed description of the studies that are the basis for the uncertainty may be included in the text. The description should be limited to those factors that are necessary to summarize the issue. Also, the “Supplemental Document” contains detailed descriptions of studies that provide no-observed-adverse-effect levels (NOAELs) and lowest-observed-adverse-effect levels (LOAELs).

As you review the profile, if you wish to comment or suggest specific changes, please annotate directly in the text where the change or additional work is needed. After reviewing the document, prepare a summary report that addresses your major issues. Please present your comments in a constructive manner, be specific about the issues/changes suggested, and cite the section numbers whenever possible. If an issue has been missed or addressed improperly, please give specific information as to how it should be addressed. If you are citing a new reference, please provide a copy and indicate where in the text it should be included. Do not cite secondary sources except when the facts are widely accepted and non-controversial (as in the case of chemical identity information and physical property values).

Please note that there is a standard format for the profiles, including introductory standard language in some sections (in bold), and certain tables, figures, headings, etc. Comments that relate to general format are welcome, and they will be considered in future revisions of the “Guidance for the Preparation of a Toxicological Profile.”

Chapter 1. Relevance to Public Health

The purpose of this section is to evaluate and interpret the significance of existing toxicity data and, in some cases, speculate regarding the significance of this information as it relates to human health. Specifically, the text should address: effects known to occur in humans; effects observed in animals but not in humans; and exposure conditions (route, duration, or level) that are likely to be of concern to humans, especially around hazardous waste sites.

• Do you agree with those effects known to occur in humans as reported in the text? If not, provide a copy of additional references you would cite and indicate where (in the text) these references should be included.
• Are the effects only observed in animals likely to be of concern to humans? Why or why not? If you do not agree, please explain.
• Have exposure conditions been adequately described? If you disagree, please explain.
• If MRLs have been derived, are the values justifiable? If no MRLs have been derived, do you agree that the data do not support such a derivation?

Chapter 2. Health Effects

The intended audience for this chapter includes community-level public health officials, physicians, and concerned citizens. It is not intended to be a data review for toxicologists. Emphasis is placed on providing a summary evaluation of the weight of evidence, rather than on providing detailed descriptions of every relevant study. Scientifically prudent judgments and interpretations are both appropriate and desirable. To increase the transparency of ATSDR’s process of identifying, evaluating, synthesizing, and interpreting the scientific evidence on the health effects, ATSDR is integrating systematic review (SR) into the profiles. ATSDR’s SR framework is a slight modification of NTP’s Office of Health Assessment and Translation (OHAT) systematic review methodology (NTP 2013, 2015; Rooney et al. 2014).

Section 2.1 Introduction

This introduction is standard language (in bold). A brief substance-specific discussion may be added to explain a complex topic.

Sections 2.2-2.20 INFORMATION ON HEALTH EFFECTS

The purpose of this section is to specify the health effects that are associated with the substance and the degree of certainty attached to that association. Negative data are also presented. The text should contain conclusions about whether or not the effect occurs and whether the studies are reliable. Human data should be presented before animal data. When information suggests that an effect occurs, but the dose/response relationship is unclear, the issue should be discussed in the text.

In this section, toxicological effects are organized according to health effect category (death, body weight, respiratory, cardiovascular, gastrointestinal, hematological, musculoskeletal, hepatic, renal, dermal, ocular, endocrine, immunological, neurological, reproductive, developmental, other non-cancer, cancer, and genotoxicity). Most of the information describing reliable studies is presented in the levels of significant exposure (LSE) tables. Text should be reserved for conclusions, discussions, explanations, etc.

ATSDR follows the National Research Council’s “Guidelines for Assessing the Quality of Individual Studies,” in Toxicity Testing: Strategies to Determine Needs and Priorities (NRC 1984).

Toxicity – Quality of Human Studies

• Were adequately designed human studies identified in the text (i.e., good exposure data, sufficiently long period of exposure to account for observed health effects, adequate control for confounding factors)? Were the major study limitations sufficiently described in the text without going into lengthy discussions? If study limitations were not adequately addressed, please suggest appropriate changes.
• Were the conclusions drawn by the authors of the studies appropriate and accurately reflected in the profile? If not, did the text provide adequate justification for including the study (e.g., citing study limitations)? Please suggest appropriate changes.
• Were all appropriate NOAELs and/or LOAELs identified for each study? If not, did the text provide adequate justification for excluding NOAELs/LOAELs including, but not limited to, citing study limitations?
  Please suggest appropriate changes.
• Were the appropriate statistical tests used in the studies? Would other statistical tests have been more appropriate? Were statistical test results of study data evaluated properly? NOTE: As a rule, statistical values are not reported in the text, but proper statistical analyses contribute to the reliability of the data.
• Are you aware of other studies which may be important in evaluating the toxicity of the substance? Please provide a copy of each study and indicate where in the text each study should be included.

Health Effects in Humans Exposed Tables

These tables are used to summarize health effects evaluated in epidemiological studies.

• Are the study details and author conclusions presented accurately?

Toxicity – Quality of Animal Studies

• Were adequately designed animal studies identified in the text (i.e., adequate number of animals, good animal care, accounting for competing causes of death, sufficient number of dose groups, and sufficient magnitude of dose levels)? If not, does the inadequate design negate the utility of the study?
  Please explain.
• Were the animal species appropriate for the most significant toxicological endpoint of the study? If not, which animal species would be more appropriate and why?
• Were the conclusions drawn by the authors of the studies appropriate and accurately reflected in the text? If not, did the text provide adequate justification for including the study (e.g., citing study limitations)?
• Were all appropriate NOAELs and LOAELs identified for each study? Were all appropriate toxicological effects identified for the studies? If not, please explain.
• If appropriate, is there a discussion of the toxicities of the various forms of the substance? If not, please give examples of toxicological effects that might be important for forms of the substance.
• Were the appropriate statistical tests used in the interpretation of the studies? If not, which statistical tests would have been more appropriate? Were statistical test results of study data evaluated properly? NOTE: As a rule, statistical values are not reported in the text, but proper statistical analyses contribute to the reliability of the data.
• Are you aware of other studies that may be important in evaluating the toxicity of the substance? If you are citing a new reference, please provide a copy and indicate where (in the text) it should be included.

Levels of Significant Exposure (LSE) Tables and Figures

These tables and figures are used to summarize health effects and graphically illustrate levels of exposure associated with those effects. These tables and figures present information on health effects by route, duration, increasing dose concentration, differences in response by species, minimal risk levels (MRLs) to humans for noncancer endpoints, cancer effect levels (CELs), and EPA’s estimated range associated with an upper-bound cancer risk of 1 in 10,000 to 1 in 10,000,000.

All studies that are identified in the text are not presented in the LSE tables and figures. Studies that lack quantitative estimates of NOAELs and LOAELs, or that are not reliable, should not be selected for inclusion. All data in an LSE table must be plotted on the corresponding LSE figure, with the exception that dermal data are presented in an LSE table without an accompanying LSE figure. For a description of MRLs and how to use the LSE tables and figures, see the “User’s Guide” in the profile.

• Are the LSE tables and figures complete and self-explanatory? Does the "Users Guide" explain clearly how to use them? Are exposure levels (units, dose) accurately presented for the route of exposure? Please offer suggestions to improve the effectiveness of the LSE tables and figures and the "User's Guide."
• Do you agree with the categorization of "less serious" or "serious" for the effects cited in the LSE tables?
  Evaluation of Text
• Have the major limitations of the studies been adequately and accurately discussed? How might discussions be changed to improve or more accurately reflect the proper interpretation of the studies?
• Has the effect, or key endpoint, been critically evaluated for its relevance in both humans and animals?
• Have "bottom-line" statements been made regarding the relevance of the endpoint for human health?
• Are the conclusions appropriate given the overall database? If not, please discuss your own conclusions based on the data provided and other data provided to you but not presented in the text.
• Has adequate attention been paid to dose-response relationships for both human and animal data? Please explain.
• Has the animal data been used to draw support for any known human effects? If so, critique the validity of the support.

Mechanisms of Action

Within each health effect category, there should be a discussion of the mechanisms of action underlying the toxicity. The purpose is to provide a brief overview of known mechanisms of metabolism, absorption, distribution, and excretion, and then a discussion of any substance reactions or physiological processes that may affect these mechanisms.

• Have all possible mechanisms of action been discussed within their relevant health effect section? If not, please explain.

Hazard Identification/Systematic Review Information

Within each relevant health effect category, there should be a discussion of the hazard identification derived from the systematic review protocol. Not every section will contain this because the SR focuses only on the most relevant health effect categories.

ATSDR utilized a slight modification of NTP’s Office of Health Assessment and Translation (OHAT) systematic review methodology (NTP 2013, 2015; Rooney et al. 2014). ATSDR’s framework is an eight-step process for systematic review with the goal of identifying the potential health hazards of exposure; the framework is presented in Appendix B. Based on the process, ATSDR has determined hazard identifications for the most relevant effects following exposure.

• Are the hazard identifications clear and justifiable based on ATSDR's SR process? (In other words, if you follow ATSDR's SR protocol from start to finish, would you come to the same hazard identification conclusions?) If not, discuss where in the process there was a deviation from the protocol.
• Do you agree with the selection of endpoints that was carried forward through the SR process? If not, please indicate which endpoints you think should or should not have been included and why.
• Do you agree with the SR framework as presented in Appendix B? Are there any steps that need to be revised? Please offer any suggestions to improve the utility, effectiveness, or clarity of the SR Framework.

Chapter 3. Toxicokinetics, Susceptible Populations, Biomarkers, Chemical Interactions

The intended audience for this chapter includes community-level public health officials, physicians, and concerned citizens. It is not intended to be a data review for toxicologists. Emphasis is placed on providing a summary evaluation of the weight of evidence, rather than on providing detailed descriptions of every relevant study.

Section 3.1 Toxicokinetics

This section, like all preceding sections, should provide a synthesis and a weight-of- evidence analysis of toxicokinetics without detailed descriptions of individual studies (unless they are key to understanding the data). Special attention should be focused on significant toxicokinetic differences between high- vs. low-level exposure and sex or species differences (especially between humans and animals) that might be relevant in extrapolation of animal toxicity data to humans. As in the discussion of toxicological effects, the section should be organized by human vs. animal studies and, within these, by duration of exposure where possible.

• Is there adequate discussion of absorption, distribution, metabolism, and excretion of the substance? If not, suggest ways to improve the text.
• Have the major organs, tissues, etc. in which the substance is stored been identified? If not, suggest ways to improve the text.
• Have all applicable metabolic parameters been presented? Have all available pharmacokinetic/pharmacodynamic models and supporting data been presented? If not, please explain.
• Is there adequate discussion of the differences in toxicokinetics between humans and animals? What other observations should be made?
• Is there an adequate discussion of the relevance of animal toxicokinetic information for humans? If not, please explain.
• If applicable, is there a discussion of the toxicokinetics of different forms of the substance (e.g., inorganic vs. organic mercury)?

Section 3.2 Children and Other Populations That are Unusually Susceptible

This section begins with standard language (in bold) and identifies known or potential unusually-susceptible populations, including a discussion about whether children are more susceptible than the general adult population

Please answer the following questions in your review:

• Are there any data relevant to child health and developmental effects that have not been discussed in the profile and should be?
• Are there any general issues relevant to child health that have not been discussed in the profile and should be?
• If you answer yes to either of the above questions, please provide any relevant references.
• Is there a discussion of populations at higher risk because of biological differences that make them more susceptible? Do you agree with the choices of populations? Why or why not? Are you aware of additional studies in this area?

Section 3.3 Biomarkers of Exposure and Effect

This section begins with standard language (in bold).

• Are the biomarkers of exposure specific for the substance or are they for a class of substances? If they are not specific, how would you change the text?
• Are there valid tests to measure the biomarker of exposure? Is this consistent with statements made in other sections of the text? If not, please indicate where inconsistencies exist.
• Are the biomarkers of effect specific for the substance or are they for a class of substances? If they are not specific, how would you change the text?
• Are there valid tests to measure the biomarker of effect? Is this consistent with statements made in other sections of the text? If not, please indicate where inconsistencies exist.

Section 3.4 Interactions With Other Chemicals

Discuss the influence of other substances on the toxicity of the substance.

• Is there adequate discussion of the interactive effects with other substances? Does the discussion concentrate on those effects that might occur at hazardous waste sites? If not, please clarify and add additional references.
• If interactive effects with other substances are known, does the text discuss the mechanisms of these interactions? If not, please clarify and provide any appropriate references.

Chapter 4. Chemical and Physical Information

This chapter should contain very little text. Most of the information should be presented in tabular form.

• Are you aware of any information or values that are wrong or missing in the chemical and physical properties tables? Please provide appropriate references for your additions or changes.

Is information provided on the various forms of the substance? If not, please explain.

Chapter 5. Potential for Human Exposure

This chapter includes general statements describing the ways in which substance releases are modified by time and environmental fate processes and the potential for human exposure to the substance via the different pathways.

Section 5.2 Production, Import/Export, Use, and Disposal

The level of detail in this section should be appropriate to an overview.

• Are you aware of any information that is wrong or missing? If so, please provide copies of the references and indicate where (in the text) the references should be included.

Sections 5.3-5.7

• Has the text appropriately traced the substance from its point of release to the environment until it reaches the receptor population? Does the text provide sufficient and technically sound information regarding the extent of occurrence at NPL sites? Do you know of other relevant information? Please provide references for added information.
• Does the text cover pertinent information relative to transport, partitioning, transformation, and degradation of the substance in all media? Do you know of other relevant information? Please provide references for added information.
• Does the text provide information on levels monitored or estimated in the environment, including background levels? Are proper units used for each medium? Does the information include the form of the substance measured? Is there an adequate discussion of the quality of the information? Do you know of other relevant information? Please provide references for added information
• Does the text describe sources and pathways of exposure for the general population and occupations involved in the handling of the substance, as well as populations with potentially high exposures? Do you agree with the selection of these populations? If not, why? Which additional populations should be included in this section?

Chapter 6. Adequacy of the Database

This chapter begins with standard ATSDR language (in bold). “Data needs” are defined as substance-specific informational needs that, if met, would reduce or eliminate the uncertainties of human health assessment. This definition should not be interpreted to mean that all data needs discussed in this section must be filled. In the future, the identified data needs will be evaluated and prioritized and a substance-specific research agenda will be proposed.

Section 6.1 Information On Health Effects

Figure 6-1 “Summary of Health Effects Studies on [Substance X]” is provided to illustrate that the evaluation of post-exposure health effects data exist. There is standard language (in bold) in the text. The bars in the figure do not imply anything about the quality of the study or studies. Gaps in this figure should not be interpreted as “data needs” information.

• Do you know of other studies that may fill a data gap? If so, please provide the reference.

Section 6.2 Identification of Data Needs

Carefully consider the data needs because they will serve as the basis for establishing a substance-specific research agenda. The following questions also pertain to both of those sections.

Chapter 7. Regulations and Guidelines

This chapter should present most information in tabular form. Information that is relevant but does not fit conveniently into the tabular format may be described in a brief paragraph.

• Are you aware of other regulations or guidelines that may be appropriate for the table? If so, please provide a copy of the reference.

Chapter 8. References

The intent of this section is to provide a reasonably complete list of references, whether cited in the text or not. Every reference cited in the text should appear with an asterisk in the bibliography.

• Are there additional references that provide new data or are there better studies than those already in the text? If so, please provide a copy of each additional reference.

Unpublished Studies (If Applicable to Review)

See previously stated criteria for evaluating the quality of human and animal studies.

• For each of the unpublished studies included with the profile, prepare a brief evaluation that includes your assessment of the:
  • Adequacy of design, methodology, and reporting;
  • Validity of results and author's conclusions; and
  • Study inadequacies or confounding factors.
• Provide a summary of your conclusions? Do you agree or disagree with those of the author? If not please explain why.