GPRA Public Comment Response Matrix

	Agency Information Collection Activities: Proposed Collection; Comments Request 0930-XXXX Package Title (Package Abbreviated Title) XX FR XXXX (Month 05, 2021)
	Summary of Comments and SAMHSA’s Responses as of 09/30/2021
Comment Number	Date Received	Organizations	Summary of Comments	SAMHSA’s Response
1	9/7/2021	Alecia Smith, Plains Area Mental Health (IA)	1. Proposed changes are acceptable. 2. Would like SAMHSA to consider in the future a combined NOMS and TEDS tool. Co-occurring clients complete the TEDS in addition to the NOMS.	SAMHSA appreciates your feedback and positive comments about the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. CMHS will consider aligning TEDS with the NOMS data collection tool in the future.
2	9/7/2021	Ann Bicego, Macomb County Community Mental Health (GA)	1. Merging the child client-level measures with the adult client-level measures will eliminte error and streamline our process. 2. The NOMS is a lengthy document and to eliminate any portion that is not serving a useful aim is welcome. 3. Shifting the questions from a five-point psychometric response scale will indeed significantly reduce the burden on staff and individuals served. 4. Modification for expansion of diagnoses and adding Z-codes will help with accurate, specific data and ease the burden on data entry teams to find the most closely aligned diagnosis. 5. Shifting NOMS reporting to baseline, 3- or 6-month reassessment, and clinical discharge makes sense to us clinically. 6. Reducing reporting frequency of physical health indicators to three points in time would ease the burden of data entry.	SAMHSA greatly appreciates your feedback and positive comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools.
3	9/10/2021	Melissa King, Kent School of Social Work (KY)	1. Combing the adult and child tools are more burdensome since they only serve children; many of the questions simply do not apply. 2. Shifting from a 5-point response to Yes/No will mask improvements many of our clients make during treatment, potentially harming SAMHSA's abilit to show program effectiveness. Particularly when I think about the quality of life questions and social support questions. 3. Having only three NOMS points for our projects would be a large savings for us, since the nature of our projects means that sometimes a single client is in treatment for 12-18 months.	SAMHSA does not believe that the merging of the child- and adult-level NOMS tools will add burden to the grantee data collection process. The language has been modified so questions can either be asked of the adult, the child/youth if they have the capacity to respond, or the caregiver of child. SAMHSA appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners are may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts.
4	9/7/2021	Karen Cellarius, Portland State University (OR)	I approve of all of the requested revisions except for one: I object to the proposal to shift questions from a 5-point psychometric scale to a two-point yes/no question. "This change would make it harder to assess change over time for certain conditions, such as mental health symptoms or substance use."	SAMHSA appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners are may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts.
5	9/9/2021	Brooke Felger, West Michigan Community Mental Health (MI)	"While I find the tools absolutely necessary and extremely valuable, the administrative burden and perceived hassle to the consumer can sometimes get in the way of said value".	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) changes in the data collection tools. We agree that there is value on reducing the burden of data collection for bith the grantee and its clients clients. CMHS has proposed changes to the NOMS data collection tool that reduces grantee burden by 50 percent.
6	9/8/2021	Karen Guan, Uplift Family Services (CA)	"It does not take much time to read the five responses as most consumers get the hang of them after a question or two." 2. I hope that existing grants will be able to keep the same reporting requirements as they had when the started. We have already developed robust data collection procedures and to change them part-way through the grant would be additional burden".	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. There was great variability in the current NOMS 5-point psychometric scale in that response options for different sections of the NOMS tool were not uniform. For example, response options for Section F (Perception of Care) ranged from strongly disagree to strongly agree. In comparison, response options for Section B (Functioning) ranged from never to daily/almost daily. SAMHSA believes that this variability in response options actually increased grantee burden. SAMHSA acknowledges the concern about exising grants and their current reporting requirements. However, OMB does not allow for a previous tool to be used when a new tool has been approved. SAMHSA will work with both the SPARS contractor and grantees to ensure that there is a sufficient time and training for an overlap between the use of the current and new tools and reports.
7	9/2/2021	Jayne Ragland, Gary Bess Associates (CA)	"I would like to write of full support for (1) merging the adult and child NOMS forms; (2) shifting the reporting of NOMS data to baseline, 3-month, or 6-month assessment and a final clinical discharge assessment; (3) reducing the number of physical health indicators and reporting frequency from quarterly to three points in time (baseline, 3- or 6-month reassessment, and clinical discharge)"	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) changes in the data collection tools.
8	9/7/2021	Sabrina Ullah, Healthright 360 (CA)	1. Great idea to merge child- and adult tools. (2) Supports deleting questions not being used for monitoring and QI. (3) Does not see how grantee burden is reduced by shifting questions from a 5-point sale to a yes/no response. (4) Supports the expansion of ICD-10. (5) supports reducing # of physical health indicators.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) changes in the data collection tools. We agree that there is value on reducing the burden of data collection for both grantees and their clients. SAMHSA appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed at the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts.
9	9/2/2021	Jetta Meadows, Gary Bess Associates (CA)	I would like to write of full support of (1) merging the adult and child NOMS forms; (2) shifting the reporting of NOMS data to baseline assessment, 3-month, or 6-month reassessment, and a final clinical discharge assessment; and (3) reducing the number of physical health indicators and reporting frequency from quarterly to three points in time (baseline, 3- or 6-month reassessment, clinical discharge assessment).	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes in the GPA data collection tools.
10	9/23/2021	Dan Olshansky, Behavioral Health Network (MA)	The revisions noted in this project reflect an intent to reduce the data set to fields that reflect outcome measurement practices, are in the service of showing the efficacy of SAMHSA programs, and reduce the administrative burden of not-for-profit agencies. One additional request is to create an electronic upload process of this data so that we may collect this information in our electronic health records and then securely send the data to SAMHSA.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed revisions of GPRA data collection tools. SAMHSA does anticipate that an electronic batch upload of data will become available within a year after the proposed data collection instruments are approved by OMB.
11	9/26/2021	Paul Frankel, Centerstone Research institute (TN)	1. Reduce substance use questions for programs that are not solely focused exclusively on substance use. 2. Eliminate or greatly shorten the military family deployement section. 3. We do not believe that the demographic questions about gender are adequately descriptive and reflect the spirit of the time for gender-related identities. For example, male and female are indicators of biological sex, not gender. Biological sex at birth should be separated from gender identity. Transgender can have a multitude of variations to encompass all clients such as non-cisgender and transitioning. 4. We are not in favor of a blanket change to all Likert scale items to a "yes" and "no" format. 5. With respect to trauma-related questions: * there are no questions regarding trauma in the child NOMS, even when a grant project is specfically focused on trauma (NCTSI). * Section H2 on the child NOMS asks "as a result of treatment ...." These questions are asked at baseline and this does not make logical sense since treatment has not started. * In the violence and trauma section for adults, the sub-questions about traumatic experiences are double-barreled or compound questions that are not easily answered such as "have nightmares about it or thougt about it", "tried hard not to think about it..." * We believe that the trauma-related question "have you ever experienced violence or trauma in any setting"should be asked at every interview. * The trauma-related question "...have you been kicked, slapped, or otherwise physically hurt...." presumes that the client has been physically hurt and .... creates a negative and defensive reaction in clients. 6. With respect to employment status, we believe that the questions regarding "...minimum wage", "...paid directly to you by your employer..." and "could anyone have appliced for this job" are not sufficiently clear for clients to understand the meaning of these items. Further we do not believe that these questions provide meaningful insight into the employment situation of our clients.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to data collection tools. Responses to your comments are as follows: 1. CMHS does not have any grant programs focused only on substance use. However, since there are several CMHS programs that serve clients with co-occurring disorders, ICD-10 data on substance use is collected. The proposed CMHS tool contains no additional questions on substance use. 2. SAMHSA has greatly reduced the number of questions related to military family deployment. 3. The wording of questions related to demographic data have been modified to be more rather than less inclusive. For example, a consumer will be asked "What do you consider yourself to be?", and the responses are male; female; transgender man/transman/female-to-man; transgender woman/transwoman/male to female; genderqueer/gender non-conforming; neither exclusively male or female; additional gender category - specify; refused. 5. With respect to trauma-related comments, SAMHSA responses are as follows: * Two program-specific questions related to ltrauma have been added for NCTSI grantee collection of data at reassessment and clinical discharge, not at baseline: "As a result of treatment and services received, my [my child's] trauma and/or loss experiences were identified and addressed; and "As a result of treatment and services received for trauam and/or loss experiences, my [my child's] problem behaviors/symptoms have decreased. * SAMHSA has removed all client/consumer-specific questions about trauma and violence. However, CMHS has proposed in the revised tool that a licensed clinician/mental health professional complete the Behavioral Health diagnosis(es) section which includes questions about screening/assessment for trauma-related experiences and suicidality. We removed questions directly asked of a client regarding past experiences of violence or trauma due to concerns of possible retraumatization. 6. With respect to employment status, CMHS has greatly reduced and revised the response options to more clearly identify a client's employment status. 7. Regarding IPP data collection, CMHS eliminated indicators which were not being used by any CMHS programs. CMHS also strongly encourages staff to only require the collection of IPP data that is used to monitor the progress of a grant program and improved the quality of services provided by the program. SAMHSA appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is superfluous at the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts.
12	9/24/21	Lorenza Daniel, Valle del Sol Community Health (AZ)	1. The non-digitized tool has created multitude of challenges … namely that the document can be easily manipulated, staff often make errors using the paper form, since the data has to be entered into the SPARS portal, room for human error and an additonal time-consuming step. Suggest that there is a portal where staff can administer the NOMS directly. 2. The data collection tool - in agreement with proposed revision. Simplify the form to only what is needed. 3. The data collection timeline - in agreement with the proposed revision. 4. The health data collection: allow patients to opt out of health data collection. Also, offer a wider window for health data collection and noting whether the data was through face-to-face collection or telehealth. 5. Limited training: training is left to the individual organization; this puts a burden on staff. Recommend that you offer comprehensive training that is easily accessible to all staff in an organization; 6. Reporting and tracking: Many of the SPARS reports are either ineffective or inoperable, leaving the tracking to the individual organization. Offer breakdown reporting for quarterly and 6-month reassesment needs. 7. Proposed changes to the IPP data collection of removing superfluous data collection, expansion of the diagnosis codes, and using a simplified answer section would be highly beneficial in reducing grantee burden.	SAMHSA appreciates your feedback and positive comments regarding many of the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. A new function for SPARS will be a portal where staff can administer the NOMS directly, thereby minimizing human errors and reducing the time currently needed to collect data. A grantee's client can opt out of current physical health indicators data collection. CMHS also believes that by reducing the number of physical health indicators, collecting data at only three points of time, and the grantee having ability to collect physical health data either face-to-face or by telehealth means will reduce grantee burden and increase the current collection rate for these data. Lastly, SPARS will begin offering training on the SPARS tools once the proposed changes are approved by OMB.
13	10/1/2021	Lori Holbrook, Avita Partners (GA)	I have reviewed the recommended revisions to this process and support the 7 that have been made. We submit to the SPARS system, in our organization, and I believe these changes would have a positive impact on the time spent gathering and inputting the data we collect.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed revisions of GPRA data collection tools.
14	9/27/2021	Tamara El-Amour, Comprehensive Healthcare (WA)	1. Make the document shorter and have it in Spanish and other languages. 2. In the NOMS record management section, add "left against treatment advice". 3. If you list more than three diagnoses, you cannot submit the NOMS since you are only allowed three, 4. Reword several questions in the functioning adult tabe, violence and trauma, stability in housing, crime and criminal justice, program specific questions for adults, and services received.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Heatlh Services (CMHS) proposed changes to the GPRA data collection tools. CMHS has significantly shortened the National Outcome Measures data collection tool and made numerous revisions to ensure that is less burdensome to both grantees and consumers. You are not limited to only three diagnoses and will no longer have to identify any as primary, secondary, and tertiary. SAMHSA believes that the other changes you requested in your comments have been addressed,inlcuidng offering the data collection tools in Spanish.
15	9/27/2021	Lisa Larson et al; IMPACT Inc	• Removal of substance use questions does not fit with population served or project outcomes. o Clients served by our projects often have co-occurring mental health and substance use disorders. o Our projects have goals and objectives related to substance use, with outcomes documented through NOMS questions. o The removal of questions related to violence and trauma may present similar challenges for other projects. • Combining adult and child measures seems incomplete, and does not accommodate interviews regarding young children. o The instructions are not always clear which items are for adults vs. children vs. everyone. o Very few items are relevant for young children served by projects. E.g., the Education question has no options for grade completion less than 12th grade (Section C, item #2, page 12). o Some items don’t seem appropriate for very young children, and the instructions as drafted don’t indicate a skip pattern. E.g., sexual orientation. • How will we be able to combine and compare data collected using the previous form with data collected from the new form? We’re working with projects that have collected baseline and outcome data for several years using the existing forms, with goals/objectives and outcome tracking built around data gathered from those forms. o The proposed changes to items and to rating scales will invalidate any comparisons of data collected over the life of the grant.  With the exception of the Perception of Care section, virtually none of the items and sections are directly comparable across versions of the previous tool and the draft of the new tool. o Will there be a crosswalk issued to map “old” data to “new” data that ensures the validity of item-to-item comparisons? oWill the NOMS outcome reports be adjusted in SPARS, and how will those reports combine “old” and “new” data (while maintaining validity)? o While one of the goals of the revision is to “reduce grantee burden”, the extent of the proposed changes will create substantial burden for existing grantees and the clients served (e.g., with the addition of “supplemental questions” and data entry applications to capture data no longer being collected). • The instructions suggest that demographic data would be expected to be collected regardless of whether an interview is conducted (i.e., the instruction at the bottom of page 3). While some of this data may be available from provider admission information, most of the data is not available (or should not be assumed) without interviewing the client. • Should the proposed draft tool be approved, we strongly urge that it be edited for clarity of instructions, numbering, etc. E.g., which staff are expected/required to complete which items, recognizing that programs vary widely in what category of staff complete the NOMS (e.g., therapists, case managers, peers, data collection specialists, etc.). E.g., What guidance will be available to non-clinical staff who may be obtaining diagnoses from electronic health records, and find more than two diagnoses listed? o E.g., Interviewers are instructed on page 8 to go to Section I. However, there does not appear to be a Section I in the draft. CORRECTED	SAMHSA appreciates your feedback and concerns regarding the Center for Mental Health Services (CMHS) proposed revisions to the GPRA data collection tools. The purpose of collecting and reporting GPRA data are: (1) to ensure SAMHSA meets its obligations under the GPRA Modernization Act of 2010; and (2) to allow SAMHSA to report to Congress on key outcome measures that relate to specific grant programs. GPRA is not intended to be a substitute for individual grant performance evaluations that (1) assess the achievements of the project in relationship to the stated goals; and (2) assess individual grantee improvement or lack thereof. SAMHSA offers the following in response to specific concerns: 1. Grantees will still be able to enter substance use diagnoses, thereby addressing issues of clients with co-occurring disorders. 2. Many of the questions related to violence and trauma were removed because of the trauma that could be inherently reactivated by asking these questions. Instead, mental health clinicians/licensed professionals are asked at the onset of the basline interview to (1) provide diagnoses; and (2) assess if the client/consumer has been exposed to trauama and/or consisdered suicide. 3. SAMHSA will ensure that the final tool has been sufficiently reviewed and revised so that the instructions are clear. 4. SAMHSA acknowledges the concern about exising grants and their current reporting requirements. However, OMB does not allow for a previous tool to be used when a new tool has been approved. SAMHSA will work with both the SPARS contractor and grantees to ensure that there is a sufficient time and training for an overlap between the use of the current and new tools and reports.
15A	10/12/2021	Lisa Larson et al; IMPACT Inc	1. S1: no information is provided in the columns for "proposed IPP indicators" or "notes". We are unclear whether this indicator will be retained with no change, adjusted, or deleted. 2. Screening for trauma and S3, Screening for suicidal ideation – Will SAMHSA be providing a list of acceptable screening tools, or will each grantee define for themselves how they will conduct screening for these issues? 3. 7, Died by suicide – The language indicates that grantees will record the number of individuals who “died by suicide while in the grant program.” We assume the focus is on the number of deaths that occurred while the person was actively enrolled in the grant (as opposed to deaths among individuals who were previously enrolled but discharged) Generally, grantees are required to set “annual goals” for each IPP indicator and report on progress towards those goals. Will we be expected to set a target for deaths by suicide, or will this be assumed to be zero for all grantees? 4. Suicide attempts – Is the expectation that grantees will ask consumers directly about any suicide attempts made? If so, what is the expectation for how frequently this will be asked of consumers? Again, will grantees be expected to set a target for suicide attempts, or will this be assumed to be zero for all grantees? 5. DEI training – This new measure is designed to document the “number of individuals trained in diversity, equity, and inclusion”. However, it’s unclear who the target population is for this training: consumers, staff, or both. The bolded language that follows from the “Notes” section seems to imply that the target population is consumers (“Measure added to understand the number of individuals being trained in DEI measures while enrolled in the grant program.”).	SAMHSA appreciates your feedback and numerous positive comments regarding the Center for Mental Health Services revised IPP indicators. IPP indicator S1 (# of individuals screened for mental health or related interventions" will be retained with no changes. IPP indicator S1 (the number of individuals screened for mental health or related interventions) will be retained without revision. SAMHSA/CMHS will provide a listing of screening tools but grantees will be allowed to propose to their assigned GPO alternative screening tools. SAMHSA/CMHS did revise the IPP measures related to suicide (T7 and T8). T7 will now ask grantees to “reduce the number of individuals who died by suicide” and T8 has been revised to ask grantees to “reduce the number of individuals who attempted suicide”. Applicants for grant funding will need to provide data on the number of suicides and number of suicide attempts for their population of focus in the selected geographic catchment area. Grantees will then be required to set IPP annual goals linked to these baseline numbers. The IPP indicator “the number of individuals trained in diversity, equity, and inclusion as a result of the grant” remains unchanged but additional language has been added to provide more clarity, i.e., “this measure has been added to understand the number of grant project staff trained in diversity, equity, and inclusion”.
16	9/28/2021	Sarah Van Hala, (1) Westbrook Health Services (2) Southern Highlands (3) Seneca (4) FMRS	1.We appreciate the language questions. We were already asking these as additional questions. 2.We appreciate the updated Armed Forces questions (Demographics-7 & Demographics-8). We were not using the army branch questions or the deployment questions. We were using the “Is anyone in your family or someone close to you currently serving on active duty or retired/separated from the Armed Forces, the 3.We have no problem with the removal of the 12 month+ interviews. 4.We appreciate that follow-up questions were added for Transgender clients. 5.We appreciate the added sexual orientation question options. 6.We appreciate the additional race options. 7. Reserves, or the National Guard?” to identify veteran families. It may be good to ask this question of children so that we can identify children with veteran families which could be a disparity group. 8.In the State of West Virginia, diagnosis and mental status exams can be provided by individuals other than a physician, licensed psychologist, licensed clinical social worker, licensed marriage and family therapist, or nurse practitioner with a Psychiatric Certification. We have case managers and master’s level staff that are not licensed or license eligible conducting the mental status exam and assigning the diagnosis. The vast majority of our NOMs are collected by staff without license/certification nor do they require a co-signature to submit. In addition, the NOMs by itself does not provide enough clinical support for a diagnosis. Other services in addition to the NOMs (ie. Clinical and/or Psychiatric Intake) would need to be provided to offer a reliable diagnosis and may take multiple sessions. The client’s diagnosis must be consistent with the presenting problems, history, mental health status exam and/or other clinical data, including any current medical diagnosis. 9.Functioning A-1 “How would you rate your [your child’s] overall mental health right now?” is okay. Asking about overall health instead of mental health may be more useful for integrated care programs. 10.We prefer recording the actual number of days for Question B-1 which asks about homeless days, hospital nights, ER visits etc. as this allows us to do more powerful statistical tests for improvement and to calculate cost savings. For example, of the 399 clients with 6-month follow-ups, there were 72 ER visits at intake and 9 at follow-up. This results in an average costs savings of about $71,001 ($1,127/visit). Yes/No responses render this data less useful to our project team as we will no longer be able to calculate system cost savings. 11.The Substance Use questions from the previous NOMS are useful to our team. We think that it is important to know what substances clients are using to best help to improve client mental health as many of our clients have co-occurring disorder. Measuring client frequency of use helps our team to measure client improvement. Also collecting data on substance use assists our agency in identifying which substance use programs are needed and for which groups of clients. A description of what we have been able to learn from the current NOMs question is provided below. (as with the yes/no change from a five question requirement, the program can continue to collect this level of data - SAMHSA however will not require it. 12.We have no problem with the removal of The following questions ask about how you have been feeling during the last 4 weeks: a.how would you rate your quality of life? b.do you have enough energy for everyday life? c.how satisfied are you with your ability to perform your daily living activities? d.how satisfied are you with your health? e.how satisfied are you with yourself? f.how satisfied are you with your personal relationships? 13.We are using the violence/trauma question: Have you ever experienced violence or trauma in any setting (including community or school violence; domestic violence; physical, psychological, or sexual maltreatment/assault within or outside of the family; natural disaster; terrorism; neglect; or traumatic grief)? And the follow-up questions: Did any of these experiences feel so frightening, horrible, or upsetting that in the past and/or the present you: a.Have had nightmares about it or thought about it when you did not want to? b.Tried hard not to think about it or went out of your way to avoid situations that remind you of it? c.Were constantly on guard, watchful, or easily startled? d.Felt numb and detached from others, activities, or your surroundings? We found gender and sexual orientation (Figure 5) disparities in client responses which we were able to report back to program staff and which impacted program services 14. We would prefer to keep the physically hurt question, as this question gives our clients another avenue to report injury. The wording allows clients to report injury without accusing theri abuser and can start a conversation about abuse and safety.	SAMHSA appreciates your positive feedback and comments regarding many of the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. The behavioral and substance use diagnoses sections do not have to be completed by a licensed clinician and can instead be completed by a member of the grantee's program staff. SAMHSA's responsibility to manage grants does not entail the level of clinical detail seen in the current NOMS tool. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed at the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners are may and do use more specific and granular assessment information in their patient/organizational management. SAMHSA's goal in proposing changes to the current tools was only to collect information that is necesary to meet the GPRA requirements, thereby greatly descreasing grantee burden.
17	9/30/2021	Martha Callow-Rucker, Community Mental Health Authorrity of Clinton, Eaton, Ingham Counties	Change the wording for some of the questions “were not binge drinking” to “no binge drinking in past 30 days” “were never using illegal substances” to “no illegal substance use in past 30 days” “were not using tobacco products” to “no tobacco product use in past 30 days” · Add a slot for “Not Applicable” Yes? under Perception of Care for questions E, H, and J. At Charter House (Clubhouse), we do not provide treatment information, talk about medications or give information on mental illness. Some CCBHC consumers receive their Mental Health Treatment in the community and would like to keep that private and separate from Charter House. · Change the answer options so they are uniform throughout. For example, either 1) all “strongly agree,” “agree,” “undecided,” “disagree,” “strongly disagree” questions, or (2) “all of the time,” “some of the time,” “none of the time,” “a little of time.” It is difficult when the answer options change from one question to the next, causing the interviewer to have to read the new answer options to the consumer. for almost every question, which takes more time. Consider changing for consistance? · Minimize the options from five Likert scale options to three. For example, change them from “strongly agree,” “agree,” “undecided,” “disagree,” “strongly agree” to “agree,” “undecided,” and “disagree” (or something along those lines). · Remove the two “yes” or “no” questions before the Services section on the paper form of the NOMs. · Create an automated way (such as a data download) to load the NOMS scores into SPARS instead of having to hand enter each one. Consider. · Have some of the NOMS questions and answer options match BHTEDS, since they collect similar information. The NOMS question and answer options of “What is your gender?” matches the BHTEDS Gender field ID A023. The answer options in the NOMS for when a person answers “Yes” to being Hispanic or Latino could match the BHTEDS answer options for the Hispanic or Latino Ethnicity Field IS A027. The Youth and Adult NOMS answer options for the question on education could also match up with the BHTEDS Education Field ID A029. Lastly, the NOMS answer options for the question of “In the past 30 days, how many times have you been arrested?” could be matched with the BHTEDS “Number of Arrests in Past 30 Days” Field ID A054. IPP Feedback · While we no longer report on the Outreach indicator, this was a very confusing one to figure out how to collect, what counted and didn’t count, etc. It would be helpful to have more clarity around each indicator. We can add clarity in the instructions? · It would also be helpful if the IPPs had an option for automated data download rather than having to hand enter each one. To be considered?	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. The purpose of collecting and reporting GPRA data are: (1) to ensure SAMHSA meets its obligations under the GPRA Modernization Act of 2010; and (2) to allow SAMHSA to report to Congress on key outcome measures that relate to specific grant programs. GPRA is not intended to be a substitute for individual grant performance evaluations that (1) assess the achievements of the project in relationship to the stated goals; and (2) assess individual grantee improvement or lack thereof. CMHS will consider aligning the NOMS and TEDS data collection tools in the future. SAMHSA appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts. Lastly, batch uploading of data may be available within one year of revised tools being programmed in SPARS.
18	9/30/2021	Bill Russell, Coleman Health Services	Firstly, I do want to thank SAMHSA for the work that you do, and assistance they provide to agencies like mine in Ohio who do non-profit behavioral health work. I also want to thank SAMSHA for suggesting revisions to the NOMs documents in advance of the rule expiration, and the opportunity for us to provide our added voice! The proposed changes are a welcome reduction, though I fear they are not nearly enough of a reduction to enhance the care we are able to offer those we serve. The NOMs and GPRA data collection we have to do on multiple grants is lengthy and intrusive enough as it is to cause clients to leave and abandon their care early in the course. They need help, and they need it when they come to us, not at their 2nd, 3rd, or 4th session after we’ve been able to get all of the ‘obligatory information gathering’. Much of the NOMs data being left here is still gathering through already required assessment procedures (section B jumps out most at me on this). This is redundant and costly work. The financial impact demonstrated in this source document relates to lost workforce hours. It does not reflect the further costs to treatment agencies for losing clients to attrition of paperwork. The estimated time impacts I would argue are underestimated. My staff, skilled clinicians and case managers alike, spend easily an hour doing an initial NOMs for our clients, sometimes more. Regardless of reading level, it is to long and intensive for clients to sit through these. Many of our clients are unable to complete these independently, and required significant assistance. I am fortunate to work at a large agency, with some administrative overhead to commit to supporting the NOMs process. Many agencies in my state are not. In the world of managed care, small agencies will rely more and more on funding like this to continue to provide care, yet will be unable to maintain adherence to rigorous measures added on top of what already rigorous documentation and info-gathering is required for our certifications and accreditations to state and national bodies to even provide the service. Good data gathering is not accomplished by added questionnaires and documentation, rather drawn from the documentation that is actually being done as part of care. The NOMs can be part of care, but it is not enough to substantiate a diagnosis or treatment plan, where as an actual clinical interview and assessment is. We are tired, we are buried in paperwork, and our industry is looking into the largest workforce shortage in our history. This is an opportunity to help address one of these things.	SAMHSA appreciates your feedback and comments regarding the proposed changes to the Center for Mental Health Services (CMHS) NOMS Data Collection tool. Significant revisions have been made to the proposed NOMS data collection tool to ensure that SAMHSA is only asking for GPRA data that will be used to monitor the progress of discretionary grants, serve as a decision-making tool, and improve the quality of program services
19	9/29/2021	Sydney Upham Soelter, Peninsula Behavioral Health	At Peninsula Behavioral Health (PBH) in Port Angeles, Washington, we are in the second year of a CCBHC-E grant and also are a sub-recipient of a CSAT Emergency Covid grant through Washington State. These projects require us to conduct the NOMS and the GPRA on a regular basis. The proposed improvements that are detailed on the federal register will be welcomed changes. I would like to add general feedback for consideration. Like many other State licensed behavioral health agencies, PBH is subject to a number of lengthy reporting requirements. We perform a comprehensive psychosocial intake, gathering information on every aspect of the patient’s mental and physical health, social supports, as well as their legal, developmental and trauma history. We utilize several standardized clinical tools to screen and measure functional impairment and outcomes, including DLA 20, GAIN-SS, PHQ9, GAD 7, CANS and PCL5, as appropriate to the patient. These tools are incorporated into our clinical workflows and yield information that helps inform treatment planning. The downside is the intake process (without the NOMS or GPRA) takes about 2 hours to complete, just for someone to qualify for services. When we add the NOMS or GPRA on top of that, it becomes unreasonably long. It is also often frustrating for the patient as they are answering questions that they have already answered. The GPRA is especially cumbersome (and for some, it is blatantly offensive) because of the requirement to ask specifically about every substance used, every sexual encounter, and every law violated over the past 30 days. Many clients are reticent to share this level of information right at the onset of treatment. This is often experienced by clinicians as the antithesis of trauma-informed care. Although the option to refuse the survey is available, many clients experience the survey as an obstacle to care, as it delays their ability to talk about why they are coming here today. The NOMS and GPRA are required to be completed within 7 days of enrollment, and they contribute to a patient experience that is already burdened with extensive data collection requirements. I am asking that SAMHSA consider alternate sources for the data they are interested in. If SAMHSA would allow grantees to utilize measures that have established clinical application rather than asking so much of the same information in slightly different ways, the data would be more easily incorporated into treatment planning and would create a better patient experience. Further, if the interviewer were allowed to complete the survey based upon answers provided elsewhere in the intake, rather than being required to follow the script, the patient would not be subject to sometimes painful redundancies.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. SAMSHA/CMHS acknowledges the burden for both grantees and clients have with NOMS data collection. We agree that there is value on reducing the burden of data collection for bith the grantee and its clients clients. CMHS has proposed changes to the NOMS data collection tool that reduces grantee burden by 50 percent. Further, The purpose of collecting and reporting GPRA data are: (1) to ensure SAMHSA meets its obligations under the GPRA Modernization Act of 2010; and (2) to allow SAMHSA to report to Congress on key outcome measures that relate to specific grant programs. GPRA is not intended to be a substitute for individual grant performance evaluations that (1) assess the achievements of the project in relationship to the stated goals; and (2) assess individual grantee improvement or lack thereof.
20	9/29/2021	Precia Stuby, Hancock County ADAMHS Board	Satisfied with changes to the NOMS tool and will decrease grantee burden. Commenter suggested the following changes 1. Option to collect demographics information (e.g., gender, DOB, race/ethnicity, level of education) through the behavioral health organization's electronic health record, if available. Currently, the NOMS process requires these questions to be asked directly to the client during the baseline interview. 2. Survey questions on the intruments should not end with potentially triggering questions (i.e., suicidal ideation and attempts). We assume that all grantee staff are trained appropriately to handle relevant issues, but the field should be trauma-informed. 3. Continue to allow flexibility in how the data are collected. Hisotrically, CMHS requires NOMs survey interviews to be conducted in person. Currently, client NOMS surveys can be collected by phone due to the pandemic. 4. Batch upload for NOMS data into SPARS. 5. Supportive of IPP indicator changes.	SAMHSA appreciates your feedback and positive comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. SAMHSA will be working with the SPARS contractor to make available an electronic batch upload function within one year of the revised tools being programmed into SPARS. Most of the demographic information can be collected through the EHR and there will be flexibility in data being collected face-to-face or via telehealth. SAMHSA has removed all client/consumer-specific questions about trauma and violence. However, CMHS has proposed in the revised tool that a licensed clinician/mental health professional complete the Behavioral Health diagnosis(es) section which includes questions about screening/assessment for trauma-related experiences and suicidality. We removed questions directly asked of a client regarding past experiences of violence or trauma.
21	9/30/2021	Billy Green, CASES, The Center for Alternative Sentencing and Employment Services	Our recommendations on Revision of Mental Health Client/Participant Outcome Measures and Infrastructure, Prevention, and Mental Health Promotion Indicators are as follows: 1. Not to merge the CMHS NOMS Child Client-level Measures for Discretionary Programs data collection instrument with the current CMHS NOMS Adult Client-level Measures for Discretionary Programs data collection instrument. 2. No need to delete questions for data not being utilized for program monitoring and quality improvement 3. Maintain five-point Likert psychometric response scales despite proposal to shift questions to “Yes”/ “No”, “No Response,” or “Not Applicable” responses to reduce grantee burden. 4. Expansion of the ICD-10 diagnoses to expand the F40-48, F60-63, and F90-99 codes to allow for more specificity.5. Addition of ICD-10 “Z” codes to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client/consumer mental disorder. 6. Shift reporting NOMS data to baseline assessment, 6-month assessment, and a final clinical discharge assessment. 7. Reduce the reporting frequency from quarterly to three points in time (baseline, 6-month reassessment, clinical discharge) to further reduce grantee burden, but maintain the number of physical health indicators. 8. Elimination of only 8 out of 10 of the Infrastructure, Prevention, and Mental Health Promotion indicators not being used by any SAMHSA programs (A3, A6, F1, F2, F3, O2, T4, WD1, WD3, and WD4). 9. Revision of two Infrastructure, Prevention, and Mental Health Promotion indicators (A1 and A5) to provide more clarity. 10. No Comment on Addition of 10 Infrastructure, Prevention, and Mental Health Promotion indicators to reflect program developments during the past three years (R2, S2, S3, T5, T6, T7, T8, TR2, TR3, and TR4). We believe these changes will lessen grantee burden with data collection and improve capacity to report qualitative performance and quantitative outcomes for all discretionary grant programs without sacrificing clinical nuance, rigor or empirical integrity of the data collection tools, including: Demographic characteristics of clients served, clinical characteristics of clients served before, during, and after receipt of services, numbers of clients served, and characteristics of services and activities provided to clients. Further details on each of these recommendations are provided in the PDF letter.	SAMHSA appreciates your feedback and comments regarding many of the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. SAMHSA believes that the proposed changes to the data collection tools will greatly reduce both grantee and client burden. SAMHSA appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant-funded programs. Individual programs and practitioners may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts. This also applies to the request to further expand of the ICD-10 "Z” codes, and expansion of ICD-10 diagnoses.
22	9/30/2021	Tammie Colon ,Mental Health and Recovery Services Board of Allen, Auglaize, and Hardin Counties	The NOMs surveys are required to be completed in-person (with the exception during the pandemic that allows interviews to be conducted by phone) at multiple time points. For example, CMHS programs require their grantees to ask the client a set of questions every 6-months while they are receiving services through the grant. The client level surveys must be asked to each client in a standardized method meaning that questions are asked in the order of the survey and in verbatim. However, there are certain questions in the NOMs survey that clients already provide during their intake process (e.g., demographic information, housing, employment, and current ATOD use). One of the key take-aways from our self-reported families’ experiences with services and providers was the unnecessary burden of having to respond to the same or similar questions repeatedly. While organizations may attempt to restructure the methods of collecting certain information from clients to reduce redundant questions, this may not be feasible to implement at an organizational level. Another issue is that for some programs, the NOMs survey ends with questions about suicide attempts and suicide ideation. These are not only an awkward way to end the interview but are potentially triggering questions. While all program staff are trained to conduct the NOMs survey with fidelity and cultural sensitivity, the rigorous standards of administering the NOMs surveys can often be the antithesis to the principles of trauma informed care. For small communities, there is also a concern that clients and caregivers who live in the same household and receive services from more than 1 grant project are being burdened with multiple requests to complete the NOMs survey The client level NOMs data are supposed to be anonymous for reporting purposes to SAMHSA. This requires maintenance of survey specific identification codes for each project which do not correspond to the client’s unique ID in the EHR system. These separate project and survey IDs do not correspond to a patient’s unique ID in the EHR system. Currently, linking the separate client IDs is not an automated process and staff cannot easily determine which client or household members are receiving services through specific grant projects. It is not always clear when a client who is receiving services through more than one grant and this creates unnecessary care coordination and workflow interruptions. Small rural counties are often times working with a limited number of stakeholders and contracting with a limited number of agencies. We have experienced both stakeholders and providers rejecting the opportunity to partner and develop shared services due to the requirements of this data collection. As I write this I have a local hospital refusing to partner for a position I want to have placed in their local emergency department because they do not want the client in their ED for that length of time, they do not want their staff to have to collect this data, nor do they want their ED staff to be responsible to try to collect the 6 month follow up data if they are receiving their outpatient services by an organization that is not contracted by the MHRSB and cannot be forced to do the GPRA.	SAMHSA appreciates your feedback and numerous positive comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. We agree that there is value on reducing the burden of data collection for both grantees and their clients/consumers and believe that the proposed changes to the NOMS data collection tool will reduce grantee burden by at least 50 percent. Demographic and diagnostic data collected during the intake process can be used to populate the Background section of the tool. Many of the questions related to violence, trauma, and suicide have been removed because of the trauma that could be inherently reactivated by asking these questions. Instead, mental health clinicians/licensed professionals are asked at the onset of the baseline interview to (1) provide diagnoses; and (2) assess if the client/consumer has been exposed to trauma and/or considered suicide.
23	9/30/2021	Cheri Walter, Ohio Association of County Behavioral Health Authorities (OACBHA)	Changes to the EHR system require expertise, time, and resources. For certain projects that have relatively small number of expected NOMs survey participants or when the completed surveys are provided to evaluators, the costs associated with making changes to the EHR system are not feasible to absorb. Overall, organizations continue to face workforce shortages which contribute to a slower start-up phase and limit the organization’s capacity to collect additional client level data beyond information obtained for claims data. Additionally, questions on the NOMs survey may require staff to change their own attitudes about certain topics (e.g., sexuality, suicide ideation, substance use). While the collections of GPRA data is a federal requirement, it is our working assumption that SAMHSA has discretion in the tools, process, and amount of data collected. With that assumption in mind, we offer the following for consideration. 1. Since the GPRA Modernization Act of 2010, and as a result of the Affordable Care Act, the majority of service providers have implemented electronic health records. As a result, the potential to collect and report data electronically is a reality. The current NOMS collection is predominately dependent on face-to-face paper/pencil collection. NOMs surveys are required to be administered through in-person interviews with the exception that during the pandemic situation interviews are allowed by phone. One potential method that may assist communities to collect NOMs data is to allow grantees to use an online survey portal for clients to complete on their own. 2. Critical information related to the success/failure of a funded projected can be retrieved from an electronic health record. Examples include but are not limited to: demographics; treatment retention; length of time between services; mix and intensity of services provided and diagnosis. 3. As we continue down the path toward population health, it is imperative that we begin to understand and apply “big data” concepts such as predictive analytics; data clustering, through the use of existing electronic data bases will be critical. Questions we could be asking are: What is the relationship of the NOMS to these concepts? Given all the years of collection, do we have predictive analytics that can be shared with the field? If so, can we track and report the data that we know to be predictive vs. comprehensive data collection that doesn’t have predictive value? 3. In addition to the “big data” concepts, measurement-based care with the use of repeated and validated measures provides the opportunity to establish benchmarks at the individual and population level. This would allow us to capture person centered functionality as well as recovery measures. Data collection in this matter provides measurement tools that can be therapeutically shared with an individual/family similar to reviewing the results of bloodwork. Individuals could identify changes while treatment staff could help them to interpret the results.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools and and process. SAMHSA will be working with the SPARS contractor have an electronic batch upload be available within one year of OMB approval of the revised data collection tools. SAMHSA will also continue to allow grantees to collect NOMS data virtually (e.g., telephone). A new function for SPARS will be a portal where staff can administer the NOMS directly, thereby minimizing human errors and reducing the time currently needed to collect data. Information that can be gathered from the EHR can be used. Lastly, the purpose of collecting and reporting GPRA data are: (1) to ensure SAMHSA meets its obligations under the GPRA Modernization Act of 2010; and (2) to allow SAMHSA to report to Congress on key outcome measures that relate to specific grant programs. GPRA is not intended to be a substitute for individual grant performance evaluations that (1) assess the achievements of the project in relationship to the stated goals; and (2) assess individual grantee improvement or lack thereof. SAMHSA's aggregates responses from data collected to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners may and do use more specific and granular assessment information in their patient/organizational management.
24	9/30/2021	Michele Guzman, TriWest Group	1. Merge the CMHS NOMS Child Client-level Measures for Discretionary Programs data collection instrument with the current CMHS NOMS Adult Client-level Measures for Discretionary Programs data collection instrument. We support this change and think it will streamline the process for grantees. 2. Shift reporting NOMS data to baseline assessment, 3-month or 6-month reassessment, and a final clinical discharge assessment. Reduce the number of physical health indictors and reporting frequency from quarterly to three points in time (baseline, 3- or 6-month reassessment, clinical discharge) to further reduce grantee burden. We support both of these changes. In addition to reducing grantee burden, the 3-month reassessment may better align with project goals related to client improvement, as 6-months is somewhat far out to obtain initial feedback about changes in clients symptoms and functioning. Additionally, with a three-month assessment there is a greater likelihood that clients may still be receiving services and therefore, be more accessible for reassessment. 3. Reduce grantee burden by shifting questions for a five-point psychometric response scale to “Yes”, “No”, “No response”, or “Not applicable” responses. It was difficult to identify which five-point psychometric responses were being referred to. There are a number of different types of responses in the NOMS. If this change is referring to items such as the ones I’ve pasted in below, then we do not support this change as greatly restricts the variability of client responses and will result in a loss of data. It also results in the loss of meaningful clinical information, as there is a significant difference between “yes” and “always”, “usually”, or “sometimes”. Likewise, “yes” or “no” responses also remove the variability that is captured by the Strongly Agree to Strongly Disagree response scale.	SAMHSA appreciates your feedback and comments about the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. SAMHSA also appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners are may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts. SAMHSA offers technical assistance to grantee organizations in the use, application and analysis of various clinical assessment tools.
25	9/29/2021	Sarah Van Hala, Southern Highlands	1. We have no problem with the removal of the 12 month+ interviews. 2. We appreciate that follow-up questions were added for Transgender client 3. We appreciate the added sexual orientation question options. 4. We appreciate the additional race options. 5. We appreciate the language questions. We were asking these as additional questions. 6. We appreciate the updated Armed Forces questions (Demographics-7 & Demographics-8). We were not using the army branch questions or the deployment questions. 7. We were using the “Is anyone in your family or someone close to you currently serving on active duty or retired/separated from the Armed Forces, the Reserves, or the National Guard?” to identify veteran families. It may be good to ask this question of children so that we can identify children with veteran families which could be a disparity group.(this question about kids was brought up elsewhere - could we add this for children, or is it not seen as necessary/relevant?) 8. Behavioral Health Diagnosis: It would be very difficult for us to have a licensed clinician fill out this section given workforce and billing demands. 9. Diagnosis Options: Can we have a PTSD option and an NA option for the Secondary Diagnosis? Also can we have an “Other, specify” option for both diagnosis? (is PTSD excluded from our revisions? i dont think we need them to include n/a - we would not use it. 10. Functioning A-1 “How would you rate your [your child’s] overall mental health right now?” is okay. This question might be more useful split into two questions: 1) “How would you rate your [your child’s] overall mental health right now?” 2) “How would you rate your [your child’s] overall physical health right now?” We appreciate that the 5-point scale was retained here. This question will be useful to our team. 11. Functioning A-2 - A-3: Lack of a 5-point response scale reduces sensitivity to detect statistically significant improvements. The 2-point scale will result in reduced sensitivity to improvement. Also there are two answer options tor A3-3c.   12. We prefer recording the actual number of days/days for Question B-1 which asks about homeless days, hospital nights, ER visits etc. as this allows us to do more powerful statistical tests for improvement and to calculate cost savings. For example, of the 397 clients with 6-month follow-ups in our PBHCI grant, there were 16 ER visits at intake and 4 at follow-up. This results in an average costs savings of about $13,524 ($1,127/visit). Yes/No responses render this data less useful to our project team as we will no longer be able to calculate system cost savings. Also we would prefer B1-1:4 to be with regard to “nights in facility” so that we can capture cost savings. Asking the question as written in the proposed NOMs will force us to re-ask these questions in a way that will be useful to measuring our outcomes. They can continue to collect this for their own project managment, we do not require it. same for the rest of these comments. 13. Question B-1-6 “living condition satisfaction” would be more useful on a 5-point scale. Yes/No responses will not be as sensitive to improvement. 14. Question C-4 “Enough money to meet needs” would be more useful on a 5-point scale. Yes/No responses will not be as sensitive to improvement. 15. Question E-1 “Perception of Care” The Yes or No scale works here because clients are either satisfied with services or not satisfied. 16. Question F-1 “Social Connectedness” would be more useful on a 5-point scale. Yes/No responses will not be as sensitive to improvement. 17. The Substance Use questions from the previous NOMS were useful to our team. We think that it is important to know what substances clients are using to best help to improve client mental health. A description of what we wereWe would also like to be able to look at the number of drugs used by each client to identify polysubstance use. 18. We have no problem with the removal of The following questions ask about how you have been feeling during the last 4 weeks: a. how would you rate your quality of life? b. do you have enough energy for everyday life? c. how satisfied are you with your ability to perform your daily living activities? d. how satisfied are you with your health? e. how satisfied are you with yourself? f. how satisfied are you with your personal relationships? 19. We are using the violence/trauma question: Have you ever experienced violence or trauma in any setting (including community or school violence; domestic violence; physical, psychological, or sexual maltreatment/assault within or outside of the family; natural disaster; terrorism; neglect; or traumatic grief)? And the follow-up questions: Did any of these experiences feel so frightening, horrible, or upsetting that in the past and/or the present you: a. Have had nightmares about it or thought about it when you did not want to? b. Tried hard not to think about it or went out of your way to avoid situations that remind you of it? c. Were constantly on guard, watchful, or easily startled? d. Felt numb and detached from others, activities, or your surroundings? able to learn from the substance use NOMs questions during our PBHCI grant are provided below. Many of our clients suffer from co-occurring disorder. 20. We would like to keep the physically hurt question to give our clients another opportunity to share their experiences. 21. We were under the impression that there would be no quarterly H-data collection. Did you mean to take out “Report Quarterly” under G-8?	SAMHSA appreciates your feedback and numerous positive comments about the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. SAMHSA also appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners are may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts. Also, many of the questions related to violence and trauma were removed because of the trauma that could be inherently reactivated by asking these questions. Instead, mental health clinicians/licensed professionals are asked at the onset of the basline interview to (1) provide diagnoses; and (2) assess if the client/consumer has been exposed to trauama and/or consisdered suicide. It appears that other recommended changes refer to the current NOMS data collection tool, not the proposed revised tool.
26	10/1/2021	Lauren Titsworth, Denton MHMR	Get rid of duplicated and rephrased questions or statements. for these two concerns, I would just say, we will consider changes in wording. The question about “in the last thirty days have you felt… “restless or fidgety” confuses some people, particularly our bilingual clients and sometimes the translators too. Use more wording that is easily translated into other languages. The question “Could anyone have applied for this job?” seems to belittle some people’s work experience and the job that they were able to obtain. We would recommend rewording this question. Use a standard scaling rather than changing it for each set of questions. It takes a significant amount of time to continue explaining the changes in scaling to clients.	SAMHSA appreciates your feedback and numerous positive comments about the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. We agree that there is value on reducing the burden of data collection for bith the grantee and its clients clients and believe that the proposed changes to the NOMS data collection tools will reduce grantee burden by at least 50 percent. We have standardized the responses to "yes", "no", "refused/did not answer", or "not applicable".
27	10/1/2021	Stephanie Collingwood/Aaron McHone, UnityPoint Health	UnityPoint Health appreciates the time and effort of SAMHSA in developing the summary of information collection request and respectfully offers the following comments: In an effort to lessen grantee burden, SAMHSA is requesting a number of revisions to te NOMS measures as well as requesting revisions to the OPP indicators. COMMENT: UnityPoint Health finds the overall revisions to be positive ...but do have the following recommendations for consideration: 1. SAMHSA utilize 6-month reassessment data to calculate outcome measures, as opposed to utilizing 3-month reassessment data. 2. SAMHSA disclose these (IPP) measure specifications prior to implementation through the draft rule process in order to provide public comment.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. During the NOMS data collection review process, CMHS staff saw that over 90 percent of its programs discharged clients prior to the 6-month reassessment requirement and neither reassessment nor clinical discharge assessment data was collected. Therfore, for programs with an average of a less than 6-month treatment period, the Notice of Funding Opportunity annoucement will indicate if grantees are to collect 3- or 6-month reassessment date, followed by a clinical discharge assessment. SAMHSA/CMHS believes that this change will signficantly increase the reassessment and discharge assessment rates, thereby providing needed program effectiveness data.
28	10/1/2021	Charles Ingoglia, National Council for Behavioral Health	The National Outcome Measures (NOMS) Mental Health Client/ Participant Outcome Measures (1) Merge the Center for Mental Health Services (CMHS) NOMS Child Client-level Measures for Discretionary Programs data collection instrument with the current CMHS NOMS Adult Client-level Measures for Discretionary Programs data collection instrument. The National Council supports SAMHSA’s merging the Child and Adult data collection instruments. Given the additional data collection, reporting and analytical demands that have been placed on mental health and substance use providers during the last decade, any instrument streamlining data collection will create efficiencies and relieve administrative burden for providers. Although the Federal Register included the statement that SAMHSA was requesting approval to modify its Center for Substance Abuse Treatment (CSAT) Client-Level Instrument by removing forty items and adding forty-one, we were unable to locate those changes and compare them to the Mental Health changes. Given the large number of behavioral health providers who offer both mental health and substance use treatment services, we would recommend that SAMHSA synchronize the review of both documents to incorporate consistency whenever it is appropriate. (2) Delete questions for data not being utilized for program monitoring and quality improvement. While the National Council has not been able to locate the questions SAMHSA is proposing to eliminate, if the information is not being used for program monitoring and quality improvement purposes, there is no reason for providers to collect and report it. (3) Reduce grantee burden by shifting questions for a five-point psychometric response scale to ‘‘Yes’’, ‘‘No’’, ‘‘No response’’, or ‘‘Not applicable’’ response scale. The National Council supports this revision since the NOMS outcome measures are not collected as part of an ongoing, real-time clinical evaluation process; given this, there is no need for a five-point psychometric response for data that is used within a performance management framework, not a clinical improvement process. (4) Modify IDC–10 diagnoses to expand the F40–48, F60–63, and F90–99 codes to allow for more specificity. Adding additional diagnostic codes will improve NOMS data to reflect more specific information about the clinical conditions of individuals served by behavioral health providers. (5) Add ICD–10 ‘‘Z’’ codes to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client/consumer mental disorder. Given current understanding about the relationship between social determinants of health (SDOH) and health outcomes, adding these codes to the NOMS client-level data collection instruments is a good long-term strategy because if allows standardized coding of SDOHs. However, in the short run it will result in significant additional administrative burden for providers, most of whom are not routinely utilizing Z codes. This will be particularly true if the Z codes must be manually loaded in the current reporting platform. In addition, SAMHSA will need to provide training and technical assistance to achieve consistent interpretation in assigning the individual Z codes since ICD provides no interpretive guidance. (6) Shift reporting NOMS data to baseline assessment, three-month or six-month reassessment, and a final clinical discharge assessment. Consistent with our comment above on administrative streamlining and the purpose for which NOMS data is collected, the National Council only supports providers conducting baseline assessments and annual reassessments. (7) Reduce the number of physical health indicators and reporting frequency from quarterly to three points in time (baseline, three- or six-month reassessment, clinical discharge) to further reduce grantee burden. Since we were unable to locate the specific proposed reductions in the number of physical health indicators, the National Council is unable to comment on SAMHSA’s proposal. However, given the current focus on whole person health care, we would hope that some key physical health metrics (e.g. A1c, total cholesterol) would be included in NOMS. Regarding the reporting frequency, we do not support the proposed requirement for a three- or six-month reassessment and think that baseline and annual reassessment provide sufficient data points. The Infrastructure Development, Prevention, and Mental Health Promotion (IPP) Indicators (1) Delete ten indicators not used by SAMSHA programs (A3, A6, F1, F2, F3, O2, T4, WD1, WD3, and WD4) Since the IPP indicators currently number thirty-one, the National Council supports deleting ten indicators that are not used by any SAMHSA program. We also suggest a review of the remaining indicators in terms of their validity as measures of grant outcomes, their strength in detecting positive results, and the measurement burden they present for providers, e.g., determining “the number of organizations or communities that demonstrate improved readiness to change their systems in order to implement mental health-related practices that are consistent with the goals of the grant” (PD2.). (2) Revise two indicators to provide more clarity (A1 and A5) Since we were unable to locate these proposed changes, the National Council has no comment. (3) Add ten indicators to reflect program developments during the past three years (R2, S2, S3, T5, T6, T7, T8, TR2, TR3, and TR4). Since we were unable to locate these proposed changes, the National Council has no specific comments. However, given our concern about the IPP indicators’ validity, strength and measurement burden noted above, the National Council does not support additions to the IPPs.	SAMHSA appreciates your feedback and numerous positive comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. We agree that there is value on reducing the burden of data collection for bith the grantee and its clients clients. CMHS has proposed changes to the NOMS data collection tool that reduces grantee burden by 50 percent. During the NOMS data collection and tools review process, CMHS staff saw that over 90 percent of its programs discharged clients prior to the 6-month reassessment requirement and neither reassessment nor clinical discharge assessment data was collected. Therefore, for programs with an average of a less than 6-month treatment period, the Notice of Funding Opportunity annoucement will indicate if grantees are to collect 3- or 6-month reassessment date, followed by a clinical discharge assessment. SAMHSA/CMHS believes that this change will signficantly increase the reassessment and discharge assessment rates, thereby providing needed program effectiveness data. The option of providing a Z code was added to the NOMS data collection tool in the event that the grantee/provider had not yet made a diagnosis. A grantee/provider will not be required to provide a Z code unless it is applicable to the client. A majority of the IPP indicators collect output data rather than outcome data. IPP measures are selected for CMHS programs that are focused in infrastructure, prevention, and/or promotion, not direct services
29	9/30/2021	Phyllis C. Panzano, Decision Supports Services, Inc.	(1) Merge the CMHS NOMS Child Client-level Measures for Discretionary Programs data collection instrument with the current CMHS NOMS Adult Client-level Measures for Discretionary Programs data collection instrument. I am strongly opposed to this revision. I am very familiar with the Adult NOMs but not the Child NOMs. I am quite familiar with both populations of clients. I am concerned that merging the two forms will constrain/ limit the questions that remain on the NOMs UNLESS sophisticated SKIP patterns are built in to allow for differentiation between questions that are geared to Adult versus Child behavior. The upkeep for maintaining and updating branching patterns can be significant and branched interviews add burden to the user. Adult and child versions may also require different phrasing (e.g., when an adult is the informant on behalf of a child versus adult self-report) for instructions etc. I am concerned that merging the forms will create more problems than it solves. Moreover, I am not clear about whose burden (e.g., SPARS subcontractor, grantees) this proposed modification will reduce. (2) delete questions for data not being utilized for program monitoring and quality improvement. This is too broad and vague of a statement to allow for an informed reaction. SAMHSA needs to specify exactly which items are targeted for deletion and to delineate whether they are currently being utilized for program monitoring or quality improvement. (3) reduce grantee burden by shifting questions for a five-point psychometric response scale to “Yes”, “No”, “No response”, or “Not applicable” responses. I am strongly opposed to this modification from both the perspective of reducing burden and loss of discriminatory information for analytic purposes. • Burden reduction: It is quite possible that a shift from a 5-point scale to dichotomous Yes-NO scale will not substantially reduce burden and may even increase it as a result of forcing respondents to commit to a YES versus a NO response. In fact, participants may tend to perseverate more to arrive at response and/or may to turn to the interviewer for advice if their specific circumstances make it difficult to commit to a “Yes” or “No” response. • Loss of discriminatory information: A shift from a 5-pt to a dichotomous response scale will eliminate important information about extent/degree to which an issue exists and also may reduce the opportunity to gauge incremental change. For example, if a respondent must select between two options (Y or N) regarding depression (i.e., “Yes, I have…” versus “No I have not… “ been depressed in the past 30 days)” there will be no way to discern between folks who had had a down day or two out of the past 30 from those who are severely depressed every day”. Yes/no options are neither preferred from a clinical or an analytic perspective. 4) modify IDC-10 diagnoses to expand the F40-48, F60-63, and F90-99 codes to allow for more specificity. Also, add ICD-10 “Z” codes to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client/consumer mental disorder. I am in favor of this proposed change. In contrast to #3 above, it adds rather than subtracts information that may be valuable from an analytic and clinical perspective. (6) shift reporting NOMS data to baseline assessment, 3-month or 6-month reassessment, and a final clinical discharge assessment. I am absolutely and strongly opposed to this proposed change. This proposed modification is not sensitive to important differences in enrollment timelines (e.g., CMHS PIPBHC grant enrollment periods can span up to five years whereas CSAT MAT grant enrollments may end after 6 months). It may make sense to conduct a baseline, 6 month and discharge NOMs for a grant with a relatively brief enrollment timeline (e.g., CSAT MAT grant) but it makes little sense, clinically or from an engagement or analytic standpoint to have the same requirement for grants with multi-year enrollment periods because important trend information is likely to be lost. I strongly recommend aligning NOMs interview timeline requirements (can this be done program by program, or given the way SPARS works, do all timeframes need to be consistant across grant programs?) with the particular grant initiative, and for those with multi-year periods to require full NOMs at least every 6 months. (7) reduce the number of physical health indictors and reporting frequency from quarterly to three points in time (baseline, 3- or 6-month reassessment, clinical discharge) to further reduce grantee burden. I am strongly opposed to this proposed change. This proposed change also is made in the abstract without consideration of differences in grant initiative enrollment timelines (e.g., 6 months versus 5 years) or of the nature of the grant (e.g., housing versus integrated healthcare). For integrated healthcare grants such as the PIPBHC and CCBHC initiative, mechanical measures should be required at least every 6 months and lab measures should be required at least annually (ideally every 6 months). Requiring PHIs to be measured in conjunction with NOMs interviews will ensure that measures taken coincide with NOMs self-report interview data and will facilitate analyses of covariance between objective and subjective measure. In addition, coordinating measurement of PHIs with NOMs interviews conveys the important message to enrolled clients and to clinical team members that behavioral health and physical health are important to track in a coordinated way. In addition, both types of PHI measures (Mechanical and Labs) should be included in the NOMs SPARS interface (Section H) for integrated healthcare projects regardless of whether they are required for a given grant initiative. That is, PHI measures can be treated as optional for some NOMs interview periods. This would give grantees who have grant goals linked to PHIs a vehicle for coordinating PHI data gathering with NOMs interview timelines to support the examination of covariance between self-report measures on the NOMs interview and objective PHIs. IPPs SAMHSA also requests the following revisions to the Infrastructure, Prevention, and Mental Health Promotion indicators: (1) Delete ten indicators not used by any SAMSHA programs (A3, A6, F1, F2, F3, O2, T4, WD1, WD3, and WD4). I support this proposed modification (2) revise two indicators to provide more clarity (A1 and A5); That seems reasonable. (3) add ten indicators to reflect program developments during the past three years (R2, S2, S3, T5, T6, T7, T8, TR2, TR3, and TR4). • This proposed change is ambiguous. What are these indicators? Without definitions is not possible to react to these proposed modifications.	SAMHSA appreciates your feedback and comments about the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. The intent with a majority of the proposed revisions was to reduce both grantee and client burdern. SAMHSA also appreciates your feedback regarding the change from a five-point Likert scale to a response of simply "yes" or "no". SAMHSA's responsibility to manage grants does not entail the level of clinical detail reflected in the current five-point scale. Rather, it aggregates responses to develop more general outcomes of grant programs. While SAMHSA recognizes the utility of more refined assessment scales in both individual clinical work and grantee level organizational management, it is not needed for the level of analysis required within SAMHSA's management of grant funded programs. Individual programs and practitioners are may and do use more specific and granular assessment information in their patient/organizational management and SAMHSA's decision to reduce the scales to "yes" and "no" responses should in no way impede those efforts. In response to the concern of shifting the reporting of NOMS data to baseline, 3- or 6-month reassessment, and discharge, a review of CMHS programs collecting NOMS data showed that the length of stay for over 85 percent of our programs was less than six months. As such, programs will now have the opportunity to select a reassessment time period that more closely aligns to average length of stay. Regarding the number of physical health indicators and reporting frequency, a majority of grantees reported that the current requirements were burdensome and insiginificant for the purposes of GPRA reporting. Data for the proposed measures can be collected virtually if necessary and align with the assessment/reassessment data collection time periods.
30	10/1/21	Scott Lloyd, MTM Services	While we are excited that SAMSHA is taking the time to look at the NOMs tool to adjust it, we respectfully pose via our comments below the more significant overarching question of if the tool should be replaced instead of adjusted based upon the negative impacts it generates in the areas of consumer engagement and the unjustifiable stress that it puts on the organizations who are being required to collect the NOMs data. With all of the challenges we see for teams utilizing the NOMs on a daily basis, we cannot see how any small changes will rectify that and again ask the question, “Is the NOMs the correct tool going forward for what you are wanting to accomplish?” For us that answer is obviously no.	SAMHSA appreciates your feedback, comments, and support for the many of the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. CMHS believes that the changes made to the NOMS Data Collection Tool are significant and not solely an adjustment of making small changes. We believe that the NOMS data collection tool, as proposed, can meet our GPRA requirements while also greatly reducing both grantee and client burden.
31	10/1/21	Edward Carlson, Treatment Communities of America	1. Merge the CMHS NOMS Child Client-Level Measures for Discretionary Program data collection instrument with the current CMHS NOMS Adult Client Measures for Discretionary Programs data collection instruments. Feedback: Adolescents services are altogether unique and tailored to the stages of development. Therefore, it is TCA’s recommendation that this information remain separate from the collection instrument used for Adults. 2. Delete questions for data not being utilized for program monitoring and quality improvement. Feedback: No comment can be provided by TCA without an itemized list of the proposed deleted questions. 3. Reduce grantee burden by shifting questions for a five-point psychometric response scale to “Yes”, “No”, “No response”, or “Not applicable” responses. Feedback: TCA is in support of this revision. 4. Modify ICD-10 diagnoses to expand the F40-48, F60-63, and F90-99 codes to allow for more specificity. Feedback: TCA is in support of this revision. Although, it is to be noted that substance use disorder (SUD) treatment providers are more likely to utilize DSM criteria. 5. Also, add ICD-10 “Z” codes to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client/consumer mental disorder. Feedback: TCA is in support of this revision. 6. Shift NOMS data to baseline assessment, 3-months or 6-months reassessment and a final clinical discharge. Feedback: TCA cannot provide comment without clarification on the client population this data collection tool is required for. 7. Reduce the number of physical health indicators and reporting frequency from quarterly to three points in time (baseline, 3- or 6-month reassessment, clinical discharge) to further reduce grantee burden. Feedback: Again, TCA cannot provide comment without clarification on the client population this data collection tool is required for. Additional Comments: The specific client population that has been requested to provide feedback for the National Outcomes Measures (NOMS) Mental Health Client/Participation Outcome Measures tool remains vague and unclear. Are these new reporting requirements intended to track all individuals in recovery, or just individuals receiving services from SAMHSA grant awardees? This data collection tool can capture useful outcome data; however, can be taxing and pose a burdensome time constraint for both providers and organizations. Additionally, obtaining a 3- or 6-month reassessment interview poses another challenge based on the client population we serve due to the numerous variables involved in substance abuse recovery. We would require additional staffing specific to the need to ensure proper and ethical collection and recollection of this data therefore adding additional overhead to the organization. Increased or reappropriated grant funding for the administration and collection of this data would be beneficial and would ensure the accuracy of the data being collected in order to further prove the efficacy and further comprehension of increased clinical outcomes. At this time, additional requirements or assessments without needed funding would result in an unfunded mandate across the substance use disorder (SUD) treatment field.	SAMHSA greatly appreciates your feedback and positive comments regarding many of the the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. The purpose of collecting and reporting GPRA data are: (1) to ensure SAMHSA meets its obligations under the GPRA Modernization Act of 2010; and (2) to allow SAMHSA to report to Congress on key outcome measures that relate to specific grant programs. GPRA is not intended to be a substitute for individual grant performance evaluations that (1) assess the achievements of the project in relationship to the stated goals; and (2) assess individual grantee improvement or lack thereof. The intent of the proposed reporting requirements and revised data collection tools is to track clients/consumers receiving services from CMHS grantees. SAMHSACMHS found that over 85 percent of the questions in the current Adult- and Child NOMS Client-level data collection instrument were identical and not linked to developmental stages of a child/youth. SAMHSA's responsibility to manage grants does not entail collection of detailed clinical data; rather, it aggregates responses to develop more general outcomes of grant programs. Individual programs and practitioners may and do use more specific and granular assessment information in their patient/organizational management.
32	10/1/21	William Martyn, Coalition of LA Addiction Service and Prevention Providers	1. Merge the CMHS NOMS Child Client-Level Measures for Discretionary Program data collection instrument with the current CMHS NOMS Adult Client Measures for Discretionary Programs data collection instruments. Feedback: Adolescents services are altogether unique and tailored to the stages of development. Therefore, it is CLASPP’s recommendation that this information remain separate from the collection instrument used for Adults. 2. Delete questions for data not being utilized for program monitoring and quality improvement. Feedback: No comment can be provided by CLASPP without an itemized list of the proposed deleted questions. 3. Reduce grantee burden by shifting questions for a five-point psychometric response scale to “Yes”, “No”, “No response”, or “Not applicable” responses. Feedback: CLASPP is in support of this revision. 4. Modify ICD-10 diagnoses to expand the F40-48, F60-63, and F90-99 codes to allow for more specificity. Feedback: CLASPP is in support of this revision. Although, it is to be noted that substance use disorder (SUD) treatment providers are more likely to utilize DSM criteria. 5. Also, add ICD-10 “Z” codes to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client/consumer mental disorder. Feedback: CLASPP is in support of this revision. 6. Shift NOMS data to baseline assessment, 3-months or 6-months reassessment and a final clinical discharge. Feedback: CLASPP cannot provide comment without clarification on the client population this data collection tool is required for. 7. Reduce the number of physical health indicators and reporting frequency from quarterly to three points in time (baseline, 3- or 6-month reassessment, clinical discharge) to further reduce grantee burden. Feedback: Again, CLASPP cannot provide comment without clarification on the client population this data collection tool is required for. Additional Comments: The specific client population that has been requested to provide feedback for the National Outcomes Measures (NOMS) Mental Health Client/Participation Outcome Measures tool remains vague and unclear. Are these new reporting requirements intended to track all individuals in recovery, or just individuals receiving services from SAMHSA grant awardees? This data collection tool can capture useful outcome data; however, can be taxing and pose a burdensome time constraint for both providers and organizations. Additionally, obtaining a 3- or 6-month reassessment interview poses another challenge based on the client population we serve due to the numerous variables involved in substance abuse recovery. We would require additional staffing specific to the need to ensure proper and ethical collection and recollection of this data therefore adding additional overhead to the organization. Increased or reappropriated grant funding for the administration and collection of this data would be beneficial and would ensure the accuracy of the data being collected in order to further prove the efficacy and further comprehension of increased clinical outcomes. At this time, additional requirements or assessments without needed funding would result in an unfunded mandate across the substance use disorder (SUD) treatment field.	SAMHSA greatly appreciates your feedback and positive comments regarding many of the the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. The purpose of collecting and reporting GPRA data are: (1) to ensure SAMHSA meets its obligations under the GPRA Modernization Act of 2010; and (2) to allow SAMHSA to report to Congress on key outcome measures that relate to specific grant programs. GPRA is not intended to be a substitute for individual grant performance evaluations that (1) assess the achievements of the project in relationship to the stated goals; and (2) assess individual grantee improvement or lack thereof. The intent of the proposed reporting requirements and revised data collection tools is to track clients/consumers receiving services from CMHS grantees. SAMHSA does not believe that the merging of the child- and adult-level NOMS tools will add burden to the grantee data collection process. Over 85 percent of the questions in the current Adult- and Child-NOMS data collection tool were identical The language has been modified in the proposed tool so questions can either be asked of the adult or a child's caregiver. There are a minimal number of questions that are only asked of either a child or adult.
33	10/1/21	Sara Reid, MHP of Colorado	We support SAMHSA’s efforts to continuously improve the GPRA data collection process and reduce burden. For the changes proposed in the 8/2/2021 comment notice, MHP would like to inform you of the following questions and comments for each proposed change. It is also important to note that MHP has been working for several years with our electronic health records vendor, Streamline Health, to build the NOMS tool into the EHR, thus changes to the tool at this point would significantly add to our burden as a grantee.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. SAMHSA will work with the SPARS contractor to minimize data collection disruptions created by the proposed new NOMS tools.
34	10/1/21	Matt Mikaelian, MHA Westchester (NY)	MHA Westerchester wishes to express their support to the proposed changes to data collection for CCBHCs. Specifically we wish to note that the reduction in reporting events and the combining of forms help to alleviate the burden on both staff and clients. This change helps to approproately moderate the demand on technology and training to perform as a CCBHC. We are hopeful that the acceptance of additional codes and modified answer options are also accepted to better reflect the client lived experience.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools.
35	10/1/21	Ute Gazioch, FL Behavioral Health Association	The Florida Behavioral Health Association would like to offer its support to SAMHSA in revising the National Outcome Measures (NOMS) Mental Health Client/Participant Outcome measures to: 1. Merge the CMHS NOMS Child Client level Measures for Discretionary Programs data collection instrument with the current CMHS NOMS Adult Client‐level Measures for Discretionary Programs data collection instrument;   2. Delete questions for data not being utilized for program monitoring and quality improvement;   3. Shift questions for a fivepoint psychometric response scale to ‘‘Yes’’, ‘‘No’’, ‘‘No response’’, or ‘‘Not applicable’’ responses;   4. Modify IDC–10 diagnoses to to allow for more specificity and add ICD–10 ‘‘Z’’ codes;   5. Shift reporting NOMS data to baseline assessment, 3‐month or 6‐month reassessment, and a final clinical discharge assessment;   6. Reduce the number of physical health indictors and reporting frequency from quarterly to three points in time (baseline, 3‐ or 6‐ month reassessment, clinical discharge). We further offer one recommendation to expand the criminal justice indicator to include individuals having been incarcerated in the last 30 days in addition to those arrested in the last 30 days to better reflect the range of interactions patients may have with the criminal justice system.	SAMHSA appreciates your feedback and and numerous positive comments regarding the Center for Mental Health Services (CMHS) changes in the data collection tools. We agree that there is value on reducing the burden of data collection for both grantees and the grantee clients. CMHS has proposed changes to the NOMS data collection tool that reduces grantee burden by 50 percent. The criminal justice indicator in the proposed data collection tool does ask if a client has been arrested in the past 30 days and if the client has spent time in jail, a correctional facility, or on probation in the past 30 days.
36	10/1/21	Marie Williams, TN Dept of MH and SA	In general, TDMHSAS is supportive of SAMHSA’s efforts to reduce the administrative burden associated with the collection of data and completion of NOMS instruments for SAMHSA discretionary grants … our provider partners and TDMHSAS rely on the data collected via the NOMS to measure the effectiveness and efficiency of those same federal discretionary grants and therefor advocate for NOMS that mine critical and timely demographic and clinical data that assists in providing high quality services to the individuals we serve. Additionally, changes to the NOMs instruments must be made in a manner so as not to impact the ability of grant administrators to measure trends over time. 1. “Don’t Know” and “Refused” options: Sometimes both of these options are used, sometimes just “refused” is used. To simplify, please consider the single option of “Did not answer” throughout. • Language other than English spoken at home: We support the addition of this question as it may lead to gaining information that could increase the effectiveness of services associated with a discretionary grant. • Education: Since the new NOMS instrument will be used to collect information for both children and adults, please consider adding the options: Student less than grade 5”; “Student grades 5-8”; and “Student grades 9-12.” Also, for children, we recommend retaining questions D1 and D1a on the current children’s NOMs form about absenteeism. • Employment: The proposed NOMs form includes these two employment options: “Unemployed, but looking for work” and “Not employed, NOT looking for work.” For consistency, please consider using the same terminology, either “Not employed” or “Unemployed” throughout. Ex. “Not employed, but looking for work” and “Not employed, not looking for work” OR “Unemployed, but looking for work” and “Unemployed, not looking for work.” • Military experience: TDMHSAS supports asking this in the demographic section at baseline. It seems that all other questions about military experience were removed from the proposed NOMs form. Please consider retaining questions B.7b and B.8 on the current Adult NOMs form. • Trauma questions: TDMHSAS supports the addition of the trauma screening question in the diagnoses section. It appears this will replace questions B.9, B.10 and B.11on the current Adult NOMs form. If a consumer is not screened for trauma on the new form, however, it does not appear that information about their trauma experience is gathered elsewhere. Please consider retaining questions like B.9, B.10 and B.11 from the current Adult NOMS form about trauma. Also, please consider asking these questions to adults and children old enough to answer for themselves at baseline, reassessment, and discharge as new traumas may occur following the start of treatment. Lastly, please consider incorporating section G9 on the new form into the core questions so the impact of services on trauma experiences can be routinely evaluated. About 80% of our consumers report trauma experiences, making insights gained especially informative.• Substance Use: All substance use questions were removed from the newly proposed NOMs form. TDMHSAS asks that a question or two re: substance use be kept in the NOMS in order to allow insight into co-occurring disorders. As is done currently, these questions should only be answered by adults or children old enough to respond for themselves. • Stability in Housing: Question B1 on the proposed new form: Please consider adding options for spending time in the hospital overnight and emergency room use for physical health concerns to standard choices as it would provide a more complete understanding of the consumer’s situation. o Question B2 on the proposed new form: Some of the options could be phrased with more clarity. Please consider using these options: ▪ Own or rent a room, apartment, or house (include college dormitory)▪ Someone else’s room, apartment, or house ▪ Physical health care setting (hospital, nursing home, veteran’s home, assisted living) ▪ Mental Health Center/Hospital ▪ Substance Use Treatment Center ▪ Transitional/supportive housing ▪ Foster care ▪ Jail/correctional facility ▪ Homeless/shelter ▪ Other (SPECIFY) ▪ Did not answer • Question G4.1a on the proposed new form: Please consider adding a question about whether the client has already tested positive for HIV. This would provide a better understanding about why an HIV test was not completed. This same principle would also apply to testing questions about hepatitis and tuberculosis. • Discharge Status: Please consider adding an option about successfully completing treatment. For example, instead of the option “Withdrew from/refused treatment” consider the options “left with satisfactory progress” and “left without satisfactory progress” . 2. Modify ICD-10 diagnoses to expand the F40-48, F60-63, and F90-99 codes to allow for more specificity. Also, add ICD-10 “Z” codes to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client/consumer mental disorder. TDMHSAS supports these proposed revisions. 3. Shift reporting NOMS data to baseline assessment, 3-month or 6-month reassessment, and a final clinical discharge assessment. Information gathered at the 6-month reassessment often is ore informative than information gathered at the 3-month reassessment (if applicable, some discretionary grants do not conduct a 3-month reassessment). Secondly, if there are no additional assessments between 6-months and discharge (i.e., 12- month, 18-month, etc.), assessing progress and maintaining contact with consumers may be difficult, and in turn, make it difficult to obtain meaningful discharge assessment information.	SAMHSA/CMHS appreciates your feedback and numerous positive comments regarding the Center for Mental Health Services proposed changes to the GPRA data collection tools and believe that many of your suggested changes were included in the revised NOMS data collection tool. The purpose of collecting and reporting GPRA data are: (1) to ensure SAMHSA meets its obligations under the GPRA Modernization Act of 2010; and (2) to allow SAMHSA to report to Congress on key outcome measures that relate to specific grant programs. GPRA is not intended to be a substitute for individual grant performance evaluations that (1) assess the achievements of the project in relationship to the stated goals; and (2) assess individual grantee improvement or lack thereof. We agree there is value in reducing the burden of data collection for both grantees and their clients/consumers. The proposed changes to the GPRA NOMS data collection tool will reduce grantee burden by at least fifty percent. The response options for all questions are consistently "yes", "no", "no response", or "not applicable". We have removed "not appliable" as a response option. Many of the questions related to violence and trauma were removed because of the trauma that could be inherently reactivated by asking these questions. Instead, at the baseline interview mental health clinicians/licensed professionals are asked to indicate if the client/consumer has been screened for exposure to trauma and/or considered suicide. Lastly, during the NOMS data collection tool review process, CMHS found that over 90 percent of its programs discharged clients prior to the 6-month reassessment and that neither reassessment nor clinical discharge data were collected. Therefore, for programs with an average length of staff of less than six months, the Notice of Funding Opportunity (NOFO) will indicate if grantees are to conduct a 3- or 6-month reassessment, followed by a final assessment when the client is discharged. SAMHSA believes that this change will significantly increase the reassessment and clinical discharge rates, thereby providing needed programmatic data.
37	10/1/21	Bryan Hardy, LA Department of Health	1. Merge the CMHS NOMS Child client instrument with the CMHS NOMS Adult Client-Level measures for discretionary grant programs data collection instrument: We request SAMHSA clarify the intent of "merging" questionnaires as it applies to NOMS measure specficifcations and reporting. Will the actual NOMS measure specfications for children and adults remaing as previously defined if the questions are merged? Or will the merging of client questionnaires for both populations result in identifical measure specifications for NOMS measures for children and adults? We suggest population specific measures for children and adults. 2. Delete questions fo data not being utilized for program monitoring and quality improvement. The intent of this recommendation is to create efficiencies. We agree that if tjere are indicators/measures that are "empty" and does not measure increased or decreased quality of care, those measures should be deleted. 3. Reduce grantee burden by shifting questions from a 5-point pychometric scale to "Yes", "No", "No response, or "Not applicable" responses. Reducing the response to yes or no metric will quickly identify if the individual has an issue in the area or not. However, the impact of creating this efficiency will create a programmatic impact because it diminishes the variable of determining severity of the issue. 4. Modify ICD– 10 diagnoses to expand the F40–48, F60–63, and F90–99 codes to allow for more specificity. Also, add ICD–10 ‘‘Z’’ codes to allow for a focus on social determinants of health that may affect the diagnosis, course, prognosis, or treatment of a client/consumer mental disorder. Response: Expansion of the ICD-10 codes will allow more specificity, which will enhance the ability to treat individuals based on a more accurate diagnosis vs a broad spectrum. The ICD-10 Z codes focusing on social determinants of health will influence health status and contact with health services, which will help to monitor the whole person versus just their BH conditions. 6. Shift reporting NOMS data to baseline assessment, 3-month or 6- month reassessment, and a final clinical discharge assessment. Response: This has been a longstanding recommendation by the field of BH and is supported by OBH. Reduction of an reassessment period will not impact overall health outcomes or quality of care. This reduction will also reduce the burden of data collection on the client, clinician and overall organization. 7. ) Reduce the number of physical health indictors and reporting frequency from quarterly to three p oints in time (baseline, 3- or 6- month reassessment, clinical discharge) to further reduce grantee burden. Response: This has been a longstanding recommendation by the field of BH and is supported by OBH. Reduction of an reassessment period will not impact overall health outcomes or quality of care. This reduction will also reduce the burden of data collection on the client, clinician and overall organization. 8. SAMHSA also requests the following revisions to the Infrastructure, Prevention, and Mental Health Promotion indicators: Delete ten indicators not used by any SAMSHA programs (A3, A6, F1, F2, F3, O2, T4, WD1, WD3, and WD4); Response: Omitting indicators that are not utilized for client outcomes will reduce the burden on programs and clinicians, including creating more efficiencies.	SAMHSA appreciates your feedback and numerous positive comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools and believe that these changes will greatly reduce grantee burden by requiring grantees to only collect and report data needed for the purposes of GPRA reporting. SAMHSA does not believe that the merging of the Child- and Adult-level NOMS data collection tools will During the review process, SAMHSA/CMHS found that over 85 percent of the questions in the current Adult- and Child NOMS Client-level data collection instrument were identical and not linked to developmental stages of a child/youth. The proposed merger of these two data collection tools have identical questions that can be asked of an adult, a youth who is capable of providing responses, or a caregiver of a child who cannot answer the questions themselves. SAMHSA's responsibility to manage grants does not entail collection of detailed clinical data; rather, it aggregates responses to develop more general outcomes of grant programs. Individual programs and practitioners may and do use more specific and granular assessment information in their patient/organizational management.
38	10/2/21	Tara Saylor, Family and Childrens' Services	* Revise two indicators to provide more clarity (A1 and A5); Response: We support this recommendation to clarify gender and veteran status. * Add ten indicators to reflect program developments during the past three years (R2, S2, S3, T5, T6, T7, T8, TR2, TR3, and TR4). Response: The 10 new indicators referenced to be included were not located for comments.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools. A table of the proposed IPP indicators (including proposed deletions, additions, and revisions) was sent to all commenters who expressed concern about the inability to view the proposed change.
39	10/1/21	Andrew Jimmie, Alaska Native Health Board	1. SAMHSA GPRA reporting measures have been a massive burden to providers. Tribes and tribal organizations frequently find that reporting requirements use more resources than the SAMHSA funding allows. ... for large grants, 15-20% of funding can be a substantial portion, all of which would be better spent on supporting patient services. 2. The cost of data collection and reporting hinders the Tribes’ ability to apply for funding. We have also found this to be true for large grants, such as the SAMHSA COVID-19 funding. We are grateful for SAMHSA’s expeditious delivery of COVID-19 response funds. The GPRA reporting requirements for these grants, however, took more time than the delivery of services, and the cost for gathering and reporting the GPRA grant information cost more in staffing and resources than the grant itself provided. In Alaska, we know of large THOs which have had to give up their SAMHSA COVID-19 grants due to these GPRA grant requirement burdens. 3. “SAMHSA is requesting approval to modify its existing CSAT Client-level GPRA instrument by removing 40 questions and adding 41 questions to its existing CSAT Client-level GPRA instrument resulting in a net addition of 1 question.” Not only is this a replacement of existing burden, plus one, it also does not reference any of the questions being changed, nor did it include culturally-informed development through the Tribal Advisory Committee or Tribal consultation. 4. We recommend Tribes be exempt from GPRA reporting requirements, so more resources could go directly to services instead of being redirected to culturally oppressive data collection, data entry, and data reporting. 5. ANHB further recommends, related to the Proposed Project on GPRA Client/Participant Outcomes Measure (OMB No. 0930–0208), that SAMHSA should align CSAT questions and measures with existing SAMHSA National Outcomes Measures/Treatment Episode Data Set questions. More broadly, ANHB continues our recommendation that HHS agencies, including SAMHSA, align reporting and improve consistency across measures with HRSA and IHS GPRA reporting measures to reduce question burden on providers, clients, and patients. 6. GPRA reporting requirements for brief intervention and/or crisis intervention should not be longer than the clinical intervention. Currently, when providing brief intervention services, completing the GPRA reporting requirements takes more time than that clinical intervention itself. 7. ANHB recommends that SAMHSA maintain the ability to collect GPRA data via telephone and other electronic means (e.g., web-based surveys). These questionnaire completion modalities have created easier means for patients and clients to provide information, boosting response rates, and likely improving data reliability overall.	CMHS believes that many of the comments from this organization refer to the proposed CSAT data collection tool, not the CMHS tools. For example, the commenter referred to GPRA reporting requirements for brief intervention and/or crisis intervention and CMHS does not have any grant programs for this purpose. CMHS clearly recognized that the burden to grantees of collecting and reporting GPRA data, particularly the National Outcome Measures, was significant. CMHS has greatly reduced the time needed to collect and report GPRA data in several ways: merging the Adult and Child NOMS tool; deleting NOMS questions for data not being used for program monitoring and quality improvement; shift questions from a 5-point psychometric response scale to a "yes", "no", "no reponse", or "not applicable response; shift reporting of NOMS data to only three points in time; and reduce the number of physical health indicators and reporting frequency from quarterly to three points in time. SAMHSA/CMHS cannot recommend that a particular entity (i.e., Tribes) be exempt form GPRA reporting requirements since SAMHSA is required by the Government and Performance and Results Modernization Act of 2010 to collect and report accurate and timely data to stakeholders and Congress. CMHS will continue to review grantee program data collection requirements to ensure that the data collected and reported is needed to monitor the progress of SAMHSA's discretionary grants, serve as a decision-making tool on funding, and improve the quality of servvices provided through the programs.
40	9/30/21	Emily Snow, Community Counseling of Bristol County (MA)	Our organization has multiple CCBHC grants at this time, and would find the proposed changes to the NOMS quite helpful. More specifically, a reduction in reporting requirements from continuous reassessments during a client’s entire course of treatment, to having only a baseline, 1 reassessment, and a discharge, with health measures following a similar pattern, would substantial reduce the administrative burden being placed on a workforce that has already experienced substantial stress and strain while working through the pandemic. Staff burnout has resulted in staffing problems throughout our industry, and reducing the already-substantial paperwork burden could be helpful in minimizing further burnout and attrition. We would additionally approve of removal of unnecessary questions and of simplifying the various scales used for client responses, and these would make the NOMS easier to administer. Merging the adult and child NOMS instruments would remove the issues of 1. How to define where the split in age ought to be (not necessarily obvious; as an example, we are in a state where other child vs. adult paperwork changes at age 21) and 2. the potential errors around of enrolling transitional-age clients who would otherwise age out of youth paperwork. Modifying the available codes to better match the full range of our diagnosing system including SDOH’s, would improve the quality of data we are collecting, which benefits both us and SAMHSA. We cannot speak to the changes proposed to the IPP measures simple because we do not have enough information about the specifics of these changes.	SAMHSA appreciates your feedback and comments regarding the Center for Mental Health Services (CMHS) proposed changes to the GPRA data collection tools and believe that these changes will greatly reduce grantee burden and by only requiring grantees to collect and report data that is needed for the purposes of GPRA reporting.