WIRED Partner Survey Part B 6.25.09

 

B.  COLLECTION OF INFORMATION INVOLVING STATISTICAL METHODS

1.  Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used.  Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample.  Indicate expected response rates for the collection as a whole.  If the collection had been conducted previously, include the actual response rate achieved during the last collection.

 

The potential respondent universe includes the full range of regional partners and collaborators identified through reviewing grantee materials and conducting site visit interviews, as well as the universe of local workforce investment boards, local economic development agencies, and community colleges within each region. Survey data collection will consist of surveying the total universe of potential respondents with a goal of achieving an 80% response rate.

2.  Describe the procedures for the collection of information including:

Because the survey sample will include the total respondent universe, no sampling methods are needed. With efforts to maximize response rate and an anticipated minimum 80% response rate, other than an analysis of non-responses for potential response bias, no special statistical estimation procedures will be required.

  Statistical methodology for stratification and sample selection

Not applicable. No stratification or statistical sampling will be used.

Estimation procedure

Not applicable. No statistical estimation procedures will be required.

Degree of accuracy needed for the purpose described in the justification,

Not applicable.

Unusual problems requiring specialized sampling procedures, and

Not applicable.

Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Not applicable.

Statistical Methodology

 Not applicable.

Estimation Procedures

 Not applicable.

Precision of Estimates

Not applicable.

3.   Describe methods to maximize response rates and to deal with issues of non-response.  The accuracy and reliability of information collected must be shown to be adequate for intended uses.  For collections based on sampling, a special justification must be provided for any collection that will not yield reliable data that can be generalized to the universe studied.

 

1. Methods for Maximizing Response Rates

 Prior to distribution of the survey, members of the evaluation team will discuss the surveys and the importance of the Governor’s support with a member of the Governor’s staff in each state involved in a WIRED region. The team will provide each Governor with a draft letter that he can personalize and then sign. The signed letter will precede the survey as a mechanism to bring attention to the importance of the survey and help maximize the response rate. (Draft letter is attached.)

A signed letter from the regional leader expressing support for the data collection will also precede the survey as a mechanism to bring attention to the importance of the survey and help maximize the response rate. Also prior to distribution of the survey, the evaluation teams will create for each region a simple flyer describing the survey and its importance, asking lead organizations and their partner organizations to distribute it to their staff, and include the information as an article in their regional newsletters. (Draft flyer is attached.)

The evaluation’s primary method of distributing surveys will be via email. The survey email will link respondents to a web-based survey instrument, which will enable the separation of responses from individual email addresses, thus ensuring both confidentiality and a convenient response mechanism. In addition, the team will mail a hard copy of the survey to non-respondents. Finally, both the email and the mailed package will provide a telephone number that the respondent can call to complete the survey in person over the telephone.

The use of electronic, paper, and telephone survey formats is intended to maximize response rate by making survey completion easy for respondents with different preferred survey completion methods. In addition, we will make extensive telephone follow-up efforts and administer the survey by phone in order to achieve an acceptable response rate. Finally, in the unlikely event that a particular region’s response appears to be falling short, the evaluators may hire and train locally based staff on a temporary basis to locate and interview partners/stakeholders in-person to obtain the needed responses.

b. Addressing Nonresponse

Once the data collection is complete and all non-respondents have been contacted at least three times, the research team will conduct an analysis of the characteristics of non-respondents to identify potential response bias across dimensions such as region, type of organization, and respondent’s role or level within the organization. Any identified response bias will be taken into account during the analysis and reporting of survey results. Any significant deviations from random error will be accounted for by reporting results in terms of ranges and confidence intervals.

3. Reliability of Data Collection

The survey design addresses key evaluation questions by building on current research on social networks and stages of collaboration. Survey items were compiled from items that have been used before in other related surveys.^1,2,3,4

The use of the web-based survey tool Vovici to collect the survey data will help ensure the reliability of the data. The survey tool controls question branching or skip patterns, requires each item to be completed before going on to the next (eliminating item nonresponse due to respondent error), and controls the possible responses for closed-ended items with specific limited ranges and drop-down menus to help respondent select the appropriate response. Verifications and consistency checks are built into the system to standardize procedures. These procedures ensure the reliability of both the data collection method and the data collected through those methods.

4.  Describe any tests of procedures or methods to be undertaken.  Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility.  Tests must be approved if they call for answers to identical questions from 10 or more respondents.  A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

The survey was pilot-tested with five respondents. This included confirming that the categorization of types of organizations and roles/levels within organizations were clear, and also included setting up and testing the Vovici on-line data entry system that will be used for survey completion during the actual data collection process.

  

5.  Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Lead researchers for the Generation I evaluation have consulted on both the content and form of data collection with in-house (BPA and UCSD) experts in survey design and sampling, system collaboration and regional economic and workforce development policy, as well as with senior members of the evaluation team for the Generation II and III evaluation at Public Policy Associates and Upjohn Institute.

Name	Agency/Company/Organization	Telephone Number
Sherry Almandsmith, Project Director	Berkeley Policy Associates	510-465-7884
Linda Toms Barker, Survey Task Leader	Berkeley Policy Associates	808-934-9297
Mary Walshok, Ph.D Principal Investigator	University of California, San Diego, Extended Studies and Public Programs	858-534-3411
Pam Surko, Ph.D	University of California, San Diego, Extended Studies and Public Programs	858-822-4393
Nancy Hewat, Ph.D	Public Policy Associates	517-485-4477
Jeffrey Padden	Public Policy Associates	517-485-4477
Kevin Hollenbeck, Ph.D	W.E.Upjohn Institute For Employment Research	269-343-5541

Draft Governor’s Letter in Support of WIRED Partner Survey

Dear Colleague,

The front lines of global competition are in regions where companies, workers, researchers, entrepreneurs and governments come together to create a competitive advantage. Such collaborations can transform new ideas and new knowledge into advanced, high quality products or services – that is, innovate to generate growth and prosperity in the region’s economy.

The U.S. Department of Labor’s Workforce Innovation in Regional Economic Development (WIRED) Initiative is supporting such collaborative efforts to transform local regional economies across the country. The premise of the WIRED Initiative is that national competitiveness and regional prosperity are possible if communities learn to collaborate, link their knowledge resources with their business and innovation assets, and assure that their workforce has the skills required to work effectively in new and emerging industries. WIRED integrates economic, workforce, and talent development activities to drive economic transformation in regional economies across the U.S.

I am writing to urge you to participate in a brief survey on collaboration in the WIRED regions. Your response will contribute to important national research on collaborative efforts to transform local economies that will help all of us grow and prosper, and be more globally competitive. When you receive the “WIRED survey,” please take a few moments to respond.

I greatly appreciate your participation!

Sincerely,

Governor’s Signature

Be part of a regional and national social network study!

Regions where companies, workers, researchers, entrepreneurs, educators and governments come together to successfully create innovation networks can generate growth and prosperity in the region’s economy.

Please participate in a regional survey that will help us understand how collaboration is working in our region and across the country.

We need you to help - your input is important!

When you get an email asking you to complete the “WIRED survey,” please help with this important effort by taking a few minutes to complete it -- and pass along the invitation to others whose input would help give an accurate picture of how agencies are collaborating in regional transformation.

[GOVERNOR’S LETTERHEAD]

Dear Colleague,

The front lines of global competition are in regions where companies, workers, entrepreneurs, researchers, and governments come together to create a competitive advantage. Such collaborations can transform new ideas and new knowledge into advanced, high-quality products and services – that is, innovate to generate growth and prosperity in the region’s economy.

The U.S. Department of Labor’s Workforce Innovation in Regional Economic Development (WIRED) Initiative is supporting such collaborative efforts to transform regional economies across the country. The premise of WIRED is that national competitiveness and regional prosperity are possible if communities learn to collaborate, link their knowledge resources with their business and innovation assets, and assure that their workforce has the skills required to work effectively in new and emerging industries. WIRED integrates economic, workforce, and talent development activities to drive economic transformation in regional economies across the U.S.

I am writing to urge you to participate in a brief survey on collaboration in the WIRED regions. Your response will contribute to important national research on collaborative efforts to transform regional economies that will help all of us grow and prosper, and be more globally competitive. When you receive an email asking you to participate in the “WIRED survey,” please take a few moments to respond.

I greatly appreciate your participation!

Governor’s Signature

1 Collaboration: What makes it work? (second edition) A review of research literature on factors influencing successful collaborations, June 2001: Paul Mattessich, Barbara Monsey, and Marta Murray-Close.

2 Evaluating Community Collaborations: A Research Synthesis, The Lewin Group, April 3, 2000

3 Gray, Barbara “Conditions facilitating interorganizational collaboration” Human Relations 38(10), 911-936.

4 Public Policy Associates Skills Partnership Self Assessment Tool, based on PPA’s national evaluation of DOL demonstration projects addressing skills shortages.