Social Network Analysis of a Service System for Transition Aged Youth
Supporting Statement
1. Circumstances of Information Collection.
The Center for Mental Health Services (CMHS) of the Substance Abuse and Mental Health Services Administration (SAMHSA) is requesting OMB approval of 1 new questionnaire; The Social Network Analysis Questionnaire. This questionnaire obtains information about the relationships between a wide array of organizations that provide services and include individuals with serious mental health conditions between the ages of 16 and 25 in their clientele. The objective of this questionnaire is to assess changes in the relationships between these organizations in one community, Clark County Washington, as a result of a Center for Mental Health Services funded grant initiative, the Partnership for Youth Transition Program (PYT).
Under Section 561 of the Public Health Service (PHS) Act (42 USC 290ff)(Appendix A), SAMHSA’s Center for Mental Health Services is responsible for the Comprehensive Community Mental Health Services for Children with Serious Emotional Disturbances program. Grant recipients must establish and operate one or more systems of care for making a range of specified mental health services available to each child. One of the required services is assisting the child in making the difficult transition from the services received as a child to the services to be received as an adult. Such programs can help reduce the crime, homelessness and underemployment that are rampant in this population in adulthood (Davis & Vander Stoep, 1997).
The PYT grant program aims to remediate some of the most difficult system barriers that interfere with building better supports for youth in transition by providing community leaders and advocates funding for direct services and infrastructure building, technical assistance to help shape the vision, and time to establish programs and interagency relationships. Since no single site in the country has ever successfully built a comprehensive transition support system SAMHSA does not know whether combining the resources of this grant, with the resources of the community are sufficient to make significant strides in transition system building. It is imperative to answer this question systematically and rigorously in order to guide future efforts.
System characteristics that indicate successful transition system building are the presence of services that, taken together, address the wide array of service needs of this population, that allow continuity of services over time and across ages (Clark et al., 2000), and that share clients and resources and communicate with each other. Comprehensive care continuity means allowing therapeutic relationships and beneficial services to continue as long as it is needed, and to coordinate efforts across the various services that, taken together, address comprehensive needs. Because of the dichotomy of adult and child mental health systems, care continuity is typically disrupted when youth reach the age that only adult mental health serves. Thus, it is common that upon their 18th or 21st birthday there is a change in case manager, or change in program, that results solely from a change in age and for no therapeutic reasons. In order to provide care continuity across this age threshold, agencies have to overcome tremendous system barriers. The collection of social network data will help to determine the relative success of, and conditions that encourage the integration of services between adult and child mental health, human service, education and justice systems. This information collection is authorized by Section 501(d)(4) of the PHS Act (42 USC 290aa) (Appendix A).
2. Purpose and Use of Information
To date, there is not a single site in the country that has achieved an integrated system with comprehensive and continuous services for youth with serious mental health conditions as they mature into adulthood. The purpose of this study is to determine if it is possible to achieve this goal using the resources provided through a federal grant program (Partnerships for Youth Transition (PYT), Center for Mental Health Services, of the Substance Abuse and Mental Health Services Agency). In order to examine this question, a social network analysis will be conducted in one local community. Because network analysis is a costly and significantly burdensome procedure it is important to approach the question by obtaining sufficient information while not adding undue burden. Detailed examination of one site can answer the essential question, if the site is well chosen. Of the five sites that have received a PYT grant, the site deemed most likely to succeed in transition-system building was chosen to assess network functioning. Information from this one site can answer the question “Can the resources of the PYT grant lead to establishment of a continuous and comprehensive transition system?”. It can also provide some information for future investigators about characteristics of organizations that do and do not improve their interorganizational relationships along transition system dimensions. Multiple sites would be needed to examine such questions as how many sites achieved the goal, the characteristics of sites that do and don’t, or to evaluate the relative success of the program overall. Once it is determined that this type of approach CAN result in a continuous and comprehensive transition system, these more refined questions can be addressed in future endeavors.
The PYT Program in Clark County Washington, named the Options program, is the one site to be studied. The transition network in Clark County consists of approximately 110 organizations. Each of the five funded sites received four years of funding to build comprehensive supports that help adolescents with serious emotional disturbance and their families make the difficult transition from adolescent to adult functioning through the age of 25.
This project will conduct a Social Network Analysis in Clark County, Washington in order to describe the current state of a social service system for youth who are transitioning from children’s mental health services into adulthood and compare those findings to that of a Social Network Analysis conducted at baseline. Briefly, Social Network Analysis is a methodology that describes which organizations are in a network, characteristics of each network organization, the strength and direction of each organization’s relationship to the other organizations in the network, members of sub-networks, their characteristics, and their relationships with other sub-networks (Van de Ven & Ferry, 1980; Morrissey et al., 1994). The data for Social Network Analysis is collected through interviews with knowledgeable leaders in each organization (i.e. program directors, agency administrators etc.). The strength of interoganizational relationships are described along three dimensions; sharing of client referrals and information, sharing of staff, and sharing of resources. Using this information, interorganizational networks can be described in terms of their size, the degree to which they are centralized, their density, and their subsystems. Characteristics of organizations at the center or periphery can be described (i.e. ages served, type of service offered, program size, adherence to transition model guidelines).
The grant is in its final year, so the current system will reflect changes that have accrued from the onset of the grant. The results of this network analysis will be compared to the network analysis conducted at baseline to determine system change. The standardized Social Network Analysis interview will be used and has been augmented with questions specific to accepted guidelines for transition services. One hundred and ten agencies have been identified in Clark County that could potentially serve youth or young adults with serious mental, emotional and behavioral disorders. This study will conduct network analysis by interviewing one key informant from each of these programs about their organization’s professional relationship with other network organizations.
The research answers the following questions.
Does the system, at the end of the grant,
1. Provide adequate interagency linkages of services within and across the child and adult systems?
2) Provide adequate opportunities for continuity of services from adolescence into adulthood?
3) What types of organizations were most and least likely to achieve the above characteristics?
3. Use of Information Technology
The informed consent form will be mailed to the respondent in advance so that they are prepared for the interview (see Appendix B), along with a cover letter (Appendix B_2). The principal investigator’s (PI) experience has demonstrated that the most efficient and valid approaches to gathering social network data from administrators is through face to face interviews in which concepts and questions can be clarified and participants encouraged to complete the activity. Because the transition issues are relatively new to mental health systems, there is a lot of confusion about the terminology that typically needs to be clarified for both the respondent and the interviewer, thus, a face to face interview is the best method for achieving high quality information in section II, and section IIIA.
Survey Monkey will be used to collect data on sections I, and IIIB, IIIC, & IIID. Survey Monkey is an easy-to-use tool for the creation of online surveys. Its primary strength is its intuitive Web interface, which makes it easy for non-technical individuals to create surveys and export collected data. It has advanced features, like the ability to branch questions based on response and exporting to different formats, including HTML, CVS and SQL. Through Survey Monkey respondents will be able to reduce their burden, by answering at their leisure when they can obtain the needed information, for example, about the number of clients served in their organization. They will be able to submit their responses electronically.
4. Efforts to Identify Duplication
Literature searches, conversations with colleagues, attending conferences in which these kinds of findings are reported have all universally revealed no other network analysis related to youth in transition. There is a small literature base regarding the functioning of service networks for adults with long term mental illness and these articles have been used to inform the development of this study. A thorough literature search for this knowledge within other systems that have a child and adult counterpart; such as developmental disability services, revealed no relevant literature.
5. Involvement of Small Entities
This project will not have a significant impact on small businesses or entities.
6. Consequences If Information Collected Less Frequently
Each respondent will be asked to respond only once to each section of the instrument. Thus, less frequent collection would mean not collecting the information at all. Without the collection of this information, new programs that are likely to develop will most likely be offered only within child or within adult services, and won’t allow for age continuity. For those who do attempt care continuity there will be no federal leadership about how this can be accomplished, and little information in the literature about what leads to integration of services across age boundaries. Mental health is an increasingly important issue for our nation, and these types of data do not currently exist, it is important that this study be conducted.
7. Consistency With the Guidelines in 5 CFR 1320.5(d)(2)
This information collection fully complies with 5 CFR 1320.5(d)(2).
8. Consultation Outside the Agency
The notice required in 5 CFR 1320.8(d) was published in the Federal Register on February 6, 2006. See Attachment D. No comments were received.
Several consultants have reviewed and made suggestions about this instrument. These include experts in Social Network Analysis, individuals who work at county and program levels in Clark County and members of families who care for youth with serious mental health disorders. Each provided comments about the clarity, understandability, coherent, and unambiguous terminology used in the instruments. The researchers have also collected data using this instrument previously and the feedback from the interviewers have been incorporated.
Steven Banks |
Cheri Dolezal |
DeDe Sieler |
Professor Center for Mental Health Services Research Department of Psychiatry University of Massachusetts Medical School 55 Lake Ave Worcester, MA 01655 508 856-8738 |
Deputy Director, Department of Community Services 1601 E. Fourth Plain Blvd Vancouver, Washington 360-397-2130 |
Program Specialist, Department of Community Services 1601 E. Fourth Plain Blvd Vancouver, Washington 360-397-2130 |
9. Payment to Respondents
No direct remuneration will be offered to the interviewees. However, to increase compliance, the researchers will send all individuals with whom an interview is requested a $10 Starbucks gift certificate with their informed consent form, which they can use regardless of participation. This token has been demonstrated to increase study participation (Dillman, 2000). In addition, all will be given the option to receive a copy of the final report. Typically managers have little time to spare, and network analysis requires that a very high response rate to be valid. Therefore it is critically important to enlist staff members from as many of these programs as possible.
10. Assurance of Confidentiality
All data related to this study will be kept in a protected environment in accordance with the procedures outlined in the IRB application on file at Portland State University. Paper records of interviews will be stored in a locked filing cabinet at the Regional Research Institute for Human Services at Portland State University. Access to this cabinet will be limited to the co-principal investigators and lead research assistant employed by the project. Electronic files will be kept on a secure network managed by Portland State University. Access to the files is by password only and only the co-principal investigators and lead research assistant will have such access. All interviewers will be trained in confidentiality and in discussing the risks of the participation with the interviewees.
This project has been reviewed and approved by the Institutional Review Boards of the Portland State University and the University of Massachusetts Medical School (PSU FWA#: 00000091; UMASS FWA#:00004009). See Appendix C_1 and C_2 for the letters of approval.
A description of the purpose of the project, the government’s involvement, the voluntary nature, confidentiality considerations are described to respondents in the informed consent form (see Appendix B). The researchers will offer to protect each individual’s identification in the publication of the report. Participants will be asked to sign that they either grant or waive protection of identification in their consent form. Respondents are informed that their participation is voluntary.
11. Questions of a Sensitive Nature
There are no questions of a sensitive nature.
12. Estimates of Annualized Hour Burden
The following table summarizes the estimated response burden for this project.
Respondent |
Number of Respondents |
Responses/ Respondent |
Total Responses |
Hours per Response |
Total Hour Burden |
Wage Rate |
Total Cost |
Key informants from social services in Clark County |
110 |
1 |
110 |
1.25 |
137.5 |
$30 |
$4,125 |
CMHS estimates that one individual will be interviewed regarding each organization, and that up to 110 programs will be identified. Thus, the estimation is that 110 individuals will be interviewed. Each will be interviewed once for about 45 minutes, and complete the online survey in about 30 minutes (total burden=137.5 hours) at an estimated wage of $30/hour for a total of $4,125 in cost.
13. Estimates of Annualized Cost Burden to Respondents
There are no costs to respondents associated with capital and start up efforts or operation and maintenance of services for this project.
14. Estimates of Annualized Cost to the Government
The cost to CMHS of the contract to the PI to do this research is $100,000.
The cost of the time of the project officer is $1,378 (40 hours @ grade GS-14), for a total cost to the Government of $101,378.
15. Changes in Burden
This is a new project.
16. Time Schedule, Publication and Analysis Plans
The start of the data collection is anticipated for August of 2006, and will end by November 31, 2006. This is dependent on receipt of OMB approval. Since the PYT grant funding for this program ends September 31, 2006, the rigor of the study will be compromised if the data are not collected by the end of September.
This report will be made available on the UMMS web site for the Center for Mental Health Services Research and the Portland State University web site for the Research and Training Center on Family Support and Children’s Mental Health. Publication date is expected to be early 2007.
Data Analysis Plan.
The research answers the following questions.
Does the system, at the end of the grant,
1) Provide adequate interagency linkages of services within and across the child and adult systems?
2) Provide adequate opportunities for continuity of services from adolescence into adulthood?
3) What types of organizations were most and least likely to achieve the above characteristics?
Question 1 will be answered through analyzing the organization-by-organization matrices generated by the data collected from each organization about their relationship with other organizations in the network. Respondents answer each question in section IIA using a five-point Likert-type scale (ranging from 1, Not at All to 5, Very Often). Answers to these four questions form the basis for describing four types of networks within the transition network; Client receive network, Client send network, Information exchange network, and Client planning network.
For each type of these four types of relations, the five possible Likert-type responses are dichotomized and arrayed in a 0-or-1 data matrix in which 1 represents the existence of a relationship between the two organizations, and 0 indicates no relationship. A summed 110x110 matrix, in which 110 denotes the number of agencies in the transition network, is created by adding the corresponding cell values for each of the 4 questions. For example, for organization A and B, organization A makes referrals to B (score 1), but B does not refer to A (score 0), they meet for client planning purposes (score 1), but not to discuss issues of mutual interest (score 0). The value for their summed cell would be 2 (1+0+1+0). Cell values in this matrix can range from 0 to 4, with higher numbers indicating stronger interagency linkage. Several measures can be derived from these matrices.
K-Cores are a useful technique to identify agencies that are in the core and agencies that are at the periphery of networks (Johnsen et al., 1996). Each K-core identifies a set of organizations with at least k relationships with other members of the set. Organizations in the most central core have the greatest number of ties with other organizations in the central core. Each core then has progressively fewer numbers of ties with other members in their cores, and is increasingly peripheral.
Block Modeling is a technique used to describe large systems with many cores. One way to simplify the relationships in a system is to look for organizations that are structurally equivalent: organizations that tend to relate to other organizations in a similar way and therefore play similar roles within the network in a particular dimension (i.e. meeting for client planning purposes). Organizations within a block do not necessarily interact with each other, the similarity is in the way they interact with other organizations, which may or may not include organizations within the block. This method simplifies a 103x103 matrix into a smaller matrix of 4x4, 8x8, or 16x16. The size of the best fitting matrix is determined by the degree of variance explained balanced by the size of the matrix. For example, if a network can be simplified to a 4x4 matrix, and explains 60% of the variance, it is a better simplification than an 8x8 matrix that explains 65% of the variance. Block modeling results in describing a system by its components. The components are groups of organizations that relate to other groups of organizations in similar ways. K-cores indicate which organizations are least and most connected to all other network organizations. Block Modeling shows with whom those relationships are and the direction of the relationships.
Each core and each “block” can be characterized by using the information respondents provide in questions in section I, IIB, and III, thus
K-Cores will be calculated using the UCINET program, a network software program, and block modeling will be calculated using a structural equivalence approach (CONCOR).
The number and strength of the linkages in the K-Cores and Block Models will be compared to those in other studies of enhanced systems to determine the adequacy of their connections within child and adult systems and across the two to answer #1.
Question 2 will be addressed by analyzing the data from section IIB of the questionnaire. Descriptive statistics will be used to describe how many of each type of service provides the opportunity for continuity of care across the adult age threshold. Previous work has shown that the vast majority of services serve either only youth (up to age 18/21) or only adults (those 18/21 and older). “Adequate opportunity for care continuity” will be defined as 50% of services having the capacity to allow individuals to remain within their service without having to change staff or programs from before age 18 until after age 21.
Question 3 will be addressed by using multiple regression modeling of the K-cores and Block Model groupings, and using discriminant analysis to describe organizations that offer services that are continuous across the transition ages (as opposed to youth- or adult-only services).
Schedule of analysis: Complete data entry by December, 2006. Complete K-Core and Block Modeling analysis by February, 2007, Complete service continuity analysis by March 2007, and complete multiple regression and discriminant analysis by June 2007. Report complete by August, 2007.
17. Display of Expiration Date
Expiration date will be displayed.
18. Exceptions to Certification Statement
This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. Certifications are included in the submission.
Social Network Analysis (Social Network Analysis) is a methodology that describes which organizations are in a network or system, the characteristics of each of those organizations, the strength and direction of each organization’s relationship to the other organizations in the network, and sub systems of the network (Van de Ven & Ferry, 1980; Morrissey et al., 1994). By the term organization, this study is referring to the organization at a program level. For example, a mental health agency may offer 5 programs that serve adolescents or young adults; each of those programs would be considered an organization.
Briefly, Social Network Analysis methodology consists of 1 informant within each network organization completing a structured interview (Van de Ven & Ferry, 1980). Providers’ self-report on interorganizational networks has been shown to be valid and reliable (Calloway et al., 1993). The interview consists of three sections. The first section asks about organizational attributes (i.e. size, budget). The second section asks about the network of ties in the organizational environment, specifically about their referral and information exchange about clients, and their sharing of staff. The second section also asks about the specific services available in their organization and the ages that are served continuously in each service. The third section asks about organizational performance (availability, accessibility, and quality of care). See Appendix D for a copy of the instrument.
1. Respondent Universe and Sampling Methods
One of the most critical aspects of Social Network Analysis methodology is called “bounding the system” or identifying network members. Bounding is achieved through talking with those who centrally work with adolescents and adults with psychiatric disabilities in public sectors to find out all of the potential organizations and programs that might be utilized in a transition system. Some organizations will serve only adolescents, and others will serve only adults, and a smaller number will serve individuals across the adolescent/adult age barrier. Since a model transition support network must provide comprehensive supports several informants will be queried to help bound the system (i.e. public vocational rehabilitation agency, mental health agency, juvenile justice agency, child welfare agency, substance abuse agency, homeless services agency representatives). These individuals are found on the interagency advisory panel to the PYT grant agency and have volunteered time to help bound the system. Once the network is bounded the research team will identify a key informant within each organization and schedule the interviews. That key informant will be an organizational “boundary spanner” who has both extensive knowledge of the organization and global knowledge of interorganizational relationships between that organization and organizations in the area.
2. Information Collection Procedures
Identification and Contacting of Stakeholders.
The lead administrator of the agency receives a letter, fax or e-mail from RRI/PSU researchers describing the study and asking them to identify the best person to interview (see letter in Appendix B1). Researchers at RRI/PSU will follow-up by phone or email to obtain the name, address, and email address of the nominated informant.
The introductory letter (Appendix B-2) and gift certificate is sent to the person(s) nominated as informants.
A member of the research team calls the potential interviewee to schedule an interview.
If the individual agrees to be interviewed their name and email address is submitted electronically and then Survey Monkey contacts the informant with the web-based survey.
An interviewer comes to the work site for the interview at the scheduled time.
The interviewer provides the participant with the another copy of the letter, reviews the details of the study and asks if the participant has any questions before starting.
The interviewer conducts the interview.
Any individuals who were interviewed who did not complete the web survey will be telephoned to verify the email address and to encouraged participants to complete the survey.
Survey Monkey will provide the cover letter (Appendix B_2) and the consent form consent form (see Appendix B). Interviewers will also provide the letter and consent form to informants at the interview and will review this information with the interviewee prior to beginning the interview. The research contains no risk for subjects. Professionals will be providing non-sensitive information in their professional capacity. It will be clear that their answers are not confidential, and that the questions are not about sensitive issues. It will also be clear that their names will not be published unless they give the researchers specific permission to do so.
The researchers estimate that there will be 110 organizations/programs that will comprise the potential and eventual transition system and that interviews will be conducted with 110 organizational informants. Social Network Analysis calls for interviewing a representative of every organization/program in the network, thus this is not a sample, but the actual population.
Conducting Interviews
Data for network analysis is collected by interviewing agency directors and program managers about their organization’s professional relationship with other social services. Questions focus on aspects of professional relationship such as how often clients are referred to another agency and how often referrals are received from another agency. The complete survey is attached in Appendix D, face to face interviews will only collect responses on sections II A&B and IIIA. Interviewers will be conducted by graduate students in the Graduate School of Social Work at Portland State University. They will be trained by researchers from UMMS. On site supervision will be provided by an experienced research manager employed by the Regional Research Institute for Human Services at Portland State University.
3. Methods to Maximize Response Rates
Each organizational administrator will initially be contacted through the introductory letter which will be sent via email, fax, or post. They will then be contacted directly by phone. During the phone conversation the relative uniqueness of the community effort and the importance of understanding how youth in transition can best be supported will be highlighted. The value of the time of the informant will be acknowledged. Similarly, the initial contact with the informant, which will occur by post, will be followed by telephone contact and the relative uniqueness of the community effort and the importance of understanding how youth in transition can best be supported will be highlighted. The value of their time will be acknowledged. A symbolic acknowledgement of our appreciation of their considering participating in the study will be offered through the gift certificate (which they are sent regardless of eventual participation). As long as a potential informant has not declined participation interviewers will alter forms of contact until they either participate or decline. Forms of contact will include emailing, telephoning, and mailing.
4. Tests of Procedures
Social Network Analysis data collection methodology was established for mental health organizational systems by Morrissey, Calloway, and colleagues (1994 & 1997). It has been used to successfully assess the contribution of service integration to client outcomes for a variety of populations including homeless adults with mental illness (Rosenheck, Morrissey, et al.) and children with serious emotional disturbance (Johnsen et al.). The procedures are similar to ones conducted previously by the co-PI’s (Davis and Koroloff, Davis et al., 2005). The only major change in approach is utilizing Survey Monkey to obtain information on the portions of the questionnaire that don’t require clarification.
5. Statistical Consultants.
Project Officer: Diane Sondheimer, SAMHSA, (240) 276-1922.
Co-Principal Investigator: Maryann Davis, Ph.D., University of Massachusetts Medical School, (508) 856-8718
Co-Principal Investigator: Nancy Koroloff, Ph.D. Portland State University, (503) 725-4157
Analyst, Matt Johnson, Ph.D.
Consultant, Steve Banks, Ph.D. University of Massachusetts Medical School
Responsible for receiving; Brigitte Manteuffel, Ph.D., MACRO International, Phone: 404-321-3211
List of Attachments
Appendix A Legislation
Appendix B Consent Letter
Appendix B_1 Consent Letter
Appendix B_2 Consent Letter
Appendix C_1 IRB – University of Massachusetts
Appendix C_2 IRB - Portland State University
Appendix D Instrument
Appendix E FR Notice
| File Type | application/msword |
| File Title | Circumstances of Information Collection |
| Author | davism |
| Last Modified By | SKING |
| File Modified | 2006-07-19 |
| File Created | 2006-07-19 |