Attachment D2 - SAMHSA's Response to the Report

Comments of the Department of Health and Human Services on the U.S. General Accounting Office Report, ASynar Amendment Implementation: Quality of State Data on Reducing Youth Access to Tobacco Could Be Improved@

Following a detailed review, the Department of Health and Human Services finds the General Accounting Office=s (GAO) report to provide some useful guidance for future changes in the direction of the Synar program.

The Department=s Substance Abuse and Mental Health Services Administration (SAMHSA) appreciates GAO=s efforts to incorporate Department comments on the draft of this report into the final report. The recommendations in particular reflect the Department=s comments. SAMHSA did note some technical errors remaining in the final report, and provides a list of these technical errors in Appendix A of this response. Although correction of these errors would not alter the conclusions of the report, the GAO may be interested in further discussing these issues with SAMHSA. In the narrative below, SAMHSA responds to GAO=s recommendations for the Synar program.

GAO Recommendation:

Help States improve the validity of their samples by working more closely with them in developing ways to increase the accuracy and completeness of the lists of tobacco outlets from which they draw random samples for inspections.

Department Comment:

SAMHSA presumes that in order to obtain the most accurate retailer violation rate, each State must maintain a complete and accurate list of tobacco outlets from which to draw a random sample for inspections. However, while some States can easily obtain a list of tobacco outlets from retail licensing lists, other States do not have laws requiring tobacco retailer licensing, nor do they maintain a central registry of outlets that sell tobacco. Furthermore, the Synar Amendment does not require States to enact such licensing laws, and SAMHSA does not have the authority to mandate that autonomous States license their tobacco retailers. Instead, SAMHSA has chosen to work with States in other meaningful ways to improve their lists, while continuing to recommend mandatory State licensing as the most reliable method for developing sample lists. Some of the ways SAMHSA currently works with States to improve their tobacco outlet lists include:

41848. Requiring States to report and justify the accuracy and completeness of their tobacco outlet lists;

41849. Helping to assess the accuracy and completeness of tobacco outlet lists through site visits to States;

41850. Including the improvement of sample lists as a topic at multi-State technical assistance workshops as well as at each National Synar Workshop;

41851. Offering expert statistical technical assistance to States on sample design issues;

41852. Conducting sample design reviews of all State Synar Reports annually; and

41853. Conducting in-depth statistical reviews of selected States.

Additionally, SAMHSA plans to do the following to further strengthen the lists from which States draw their samples of outlets:

1. Strengthen its recommendation that States use a statistician to design their Synar samples.

2. Offer States the opportunity to individually discuss sample design concerns with a statistician at the upcoming Sixth National Synar Workshop;

3. Publish a revised version of the SAMHSA/CSAP Synar Sampling Guidance Document;

Although individual States have unique circumstances, the efforts of SAMHSA and the States have resulted in continuous improvement of the quality of States= Synar samples.

GAO Recommendation:

Revise the inspection protocol guideline to better reflect research results, particularly regarding the ages of minor inspectors, and work with States to develop a more standardized inspection protocol consistent with State law, and more uniform implementation across States.

Department Comment:

SAMHSA presumes, in accord with the GAO recommendation, that SAMHSA guidance on inspection protocols should reflect research results. Whether a revision of the inspection protocol guideline is appropriate would depend on the results of a review of the current research. SAMHSA believes that the effects on inspection outcomes of inspector demographic factors such as gender and race should be studied along with the effects of age. The issue of retailer adaptation to inspection protocols must also be studied, since retailers have access to every state=s protocol through the Freedom of Information Act and may be able to recognize when a compliance check is in progress.

The GAO report discusses the issue of accurate measurement of State success in reducing retailer violations of tobacco access laws for youths. SAMHSA believes that before discussing accurate measurement it is necessary to define the standard of performance being measured. With regard to the GAO Report=s discussion of the age of inspectors as an influence on inspection outcomes, two possible standards of measurement are: 1) to find the largest possible number of violations of tobacco sales to minors laws, or 2) to measure the likelihood that a typical underage smoker will be able to access tobacco through retail sources.

The GAO Report appears to focus on the first standard. However, SAMHSA believes that in order to obtain the most accurate retailer violation rate, the mix of youth inspectors used to conduct each State=s Synar survey should reflect the demographic mix of actual youth who attempt to purchase tobacco products in the State. For example, although minors over the age of 16 are more likely than younger minors to successfully purchase tobacco from retail sources, youth under the age of 16 both smoke and attempt to purchase cigarettes. Since SAMHSA=s intent is to measure youth tobacco access as it occurs under real world circumstances, SAMHSA allows State to use a range of inspector ages.

With regard to the GAO=s concern about standardization of inspection protocols, SAMHSA currently works with States to improve protocol standardization in the following ways:

47128. Publishing a youth inspector training manual and video;

47129. Recommending that inspectors aged 15 and 16 be used in every State; and

47130. Requiring States to describe and gain SAMHSA approval for any changes to their inspection protocols prior to their implementation.

In response to GAO=s recommendation regarding research, SAMHSA plans to:

4. Convene a workgroup of youth tobacco access control researchers and practitioners to discuss and reconcile issues related to SAMHSA=s inspection protocol guidelines;

5. Compile and publish the recommendations of this workgroup;

6. Make the workgroup=s report available to Single State Agency (SSA) Directors and State Synar Coordinators; and

4. Present the recommendations to the Department for possible further action.

As the GAO recommends, SAMHSA plans to continue to improve within-State standardization by:

48160. Working more closely with each State to develop a more standardized inspection protocol (regarding such variables as youth demographics and training protocols) that is consistent with State law; and

48161. Requiring that States use a more consistent inspection protocol each year in order to enhance the comparability of each State=s Synar survey results from year to year.

GAO Recommendation:

Ensure that all States= retailer violation rates exclude invalid inspections, particularly those in which the ages of minors and outcomes of inspections are unknown.

Department Comment:

SAMHSA knows of no circumstances during the history of the Synar program in which a State=s retailer violation rate was approved despite invalid inspections that would have put the State out of compliance with Synar requirements. In fact, the GAO Report acknowledges this fact. In recent years, as SAMHSA has become aware of discrepancies in reporting, we have instituted protocols that require States to exclude inspections that do not include the following information:

1. Age of minor inspector;

2. Gender of minor inspector; and

3. Outcome of inspection.

SAMHSA plans to continue to enforce this requirement in future years.

Appendix A:

Technical Comments of the Department of Health and Human Services on the U.S. General Accounting Office Report Titled: Synar Amendment Implementation: Quality of State Data on Reducing Youth Access to Tobacco Could Be Improved (GAO-01-1003)

1. On Page 5, first paragraph, second sentence from the bottom under the heading Background, the Report states, ABeginning in fiscal year 1997 for most States and in subsequent years for all States, the Secretary can withhold 40 percent of a State=s Substance Abuse Prevention and Treatment (SAPT) block grant if it does not comply with the rate reduction requirements.@ In fact, Section 1926(c) of the PHS Act states that AIf the Secretary determines that a State has not maintained such compliance, the Secretary is required to decrease the Block Grant...by 40 percent.@ However, Section 1926 also says that if a State does not comply with the Synar requirements, Athe Secretary, in extraordinary circumstances, may consider a number of other factors such as a scientifically sound survey indicating that the State is making significant progress toward reducing use of tobacco products by minors.@

2. On Page 12, Figure 1: AStates with the Highest Percentage of Inspections Conducted by 14- and 15-Year-Olds for Fiscal Year 1999,@ the inspector age breakdowns listed for the State of Georgia are inaccurate. In Fiscal Year 1998, 56% of Georgia=s inspections were conducted by 14- and 15-year-olds and 44% were conducted by 16-and 17-year-olds. For Fiscal Year 1999, 26% of Georgia=s inspections in this year were conducted by 16- and 17-year-olds. The inaccurate figures also make the columns total to more than 100%.

3. On Page 14, first paragraph, third full sentence, the Report states, AHad the invalid inspections been excluded, the violation rates for Florida, Louisiana, and Minnesota would have been higher.@ In this sentence, AMinnesota@ should be replaced with AKansas@ to match the table.

4. On Page 18, first full paragraph, first sentence, the Report says, A...States are permitted to use SAPT block grant funds for enforcement activities only if a citation is issued for a violation at the time of the inspection.@ This statement should be modified to read, as the Regulation says, AStates may not use the Block Grant to fund the enforcement of their statute, except that they may expend funds from the primary prevention setaside of their Block Grant allotment for carrying out the administrative aspects of the requirements such as the development of the sample design and the conducting of the inspections.@ Thus, while SAPT Block Grant funds may be used only for the issuance of a citation in the course of an inspection for the Synar survey, they may not be used to pay for other enforcement costs related to that citation, such as follow-up inspections or court costs.