SAMHSA Response

1. What is the basis for the sample size?  Is this informed by precision estimates SAMHSA has made?  Please provide additional detail.

SAMSHA intends to use the National Outcome Measures (NOMs) for developing performance measures for managing programs and grantees within those programs. In addition, the intent is to seek OMB approval to use the NOMs to fulfill OMB’s GPRA reporting requirements. Given that the NOMs are designed as a performance management tool, SAMHSA’s preference is for grantees to collect the NOMs data on all consumers served.

However, SAMHSA has decided to allow for sampling, particularly for the grantees that serve a large number of consumers. The NOMs design allows for sampling only for grantees that serve more than 100 consumers per year. Sampling would then reduce the burden on these larger grantees.

The selection of this decision rule was based largely on SAMHSA’s intent, whenever feasible, to design the NOMs data collection protocol to be consistent with existing data gathering efforts. The one program that elected to sample consumers, the Children’s Mental Health Initiative (CMHI), will align the NOMs data collection with their OMB approved cross-site evaluation data gathering (0930-0280.) The CMHI sampling plan allows sampling only among grantees that serve more than 92 consumers.

Influence by existing CMHI procedures, the Program examined the level of precision achieved for proportions with a sample size of 100. Assuming an 80% response rate this would result in an achieved sample size of 80 for a grantee who is sampling consumers. The Program used proportions as the primary indicators will be binary, achieved or did not achieve desired outcomes. The figure below provides the level of precision for various proportions with 95% confidence intervals. The maximum confidence interval of plus or minus 10% was at the midpoint was deemed acceptable for measuring grantee performance.

 

2.  Re the race and ethnicity question - is this question administered over the phone? - in general "other" is not an acceptable category per OMB guidance - however if the interview is conducted over the phone the category may be left in provided it is not read to the respondent as a possible answer choice.

 

The “Other” category with race has been removed from the survey instruments. See Attachments 1, 2, and 3. The Program will also have the “Other” category from the screens removed.