Assessing Options to Evaluate Long-term Outcomes Using Administrative Data: Targets of Opportunity

Formative Data Collections for ACF Research

Appendix A_Evaluation Template_LTO 2018-01-17 ap ks rdh_resp to passback_clean

Assessing Options to Evaluate Long-term Outcomes Using Administrative Data: Targets of Opportunity

OMB: 0970-0356

Document [docx]
Download: docx | pdf

Appendix A

Evaluation Template



Assessing Options to Evaluate Long-Term Outcomes Using Administrative Data

Evaluation Template


This data collection effort is part of the Assessing Options to Evaluate Long-Term Outcomes Using Administrative Data study, which is funded by the Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF), U.S. Department of Health and Human Services.


This study is a new effort by OPRE to determine the feasibility of linking administrative data sets to program evaluation data. Data are being collected to assess the practical and legal feasibility for accomplishing the linkages, to assess potential costs, to determine data ownership, and to identify prior history of linkage, past findings, and current availability of relevant data and metadata. The information collected on each evaluation will be submitted to OPRE, along with a memo that summarizes the findings and identifies the most promising targets of opportunity.


Can you please review the information in the template, confirm or correct as needed, and fill in any missing information? We expect it will take you 1-2 hours to review and fill in this template, depending on the availability of information.


Please note that your participation in this data collection is voluntary and your information will be kept private to the extent permitted by law.




An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0356 and the expiration date is 3/31/2018.



Evaluation

Lead evaluator

Please verify the information that we prepopulated.

Funder(s)

Please verify the information that we prepopulated.

Federal project officer

Please indicate the project officer for the original evaluation.

Status

  • Ongoing or completed?

  • Start and end date

Please indicate the dates of the contract for the original evaluation.



Evaluation Dataset

Does the evaluation dataset still exist?

Yes/no

Data destruction

  • Timing

  • Data sources

Please indicate whether any data destruction has been conducted, what data were destroyed, and any other restrictions around who can access the data.

Location/ownership

  • Who has the dataset?

  • Who owns the dataset?

Please indicate the location and ownership of the dataset.

Was a public access or restricted use file created?

Yes/No. If yes, please indicate which type of file was created and where the file(s) are stored.

What personally identifying information is available for a potential match to an administrative data source?

For example, SSN, name, date of birth, contact information.

Administrative datasets linked to in original study

Please indicate which datasets were linked and their origin. For example, if you received UI wage data, was it state UI data or data from the National Directory of New Hires?

Measures and data sources) available in the evaluation dataset

Please indicate the broad domains of measures and/or data sources. For example, MIS data, SNAP benefits data, etc.

Data dictionaries/Codebook /instruments available

Please indicate whether record layouts, codebooks, data dictionaries and/or survey instruments are available and whether they can be shared externally. Please provide links as appropriate.

Data sharing agreements for sample roster/link file

Are there agreements in place that specifically pertain to the sample membership file? The sample membership file contains the PII needed for matching which is typically (but not always) collected at the time of enrollment.



Feasibility

Have any extended follow-up analyses been done?

  • Length of follow-up

  • Data sources used

Was any additional follow-up done after the original evaluation contract? If so, please indicate the length of common follow-up and which data sources were included.

Did the study collect informed consent? If not, please explain why and how data collected for the evaluation was obtained.

Yes/no. And (if possible) please indicate if consent was obtained on paper or electronically. If no, please explain why consent was not collected and how data were obtained for the evaluation.

Informed consent description

  • Are specific data sources mentioned?

  • Is a length of data collection mentioned?

  • Are there restrictions around who can access the data? Or when the data needs to be destroyed?

  • Is there any other information we should know about regarding permissions to use the data?

Please explain the language used in the consent. How precise was the language regarding which data sources will be collected, the length of data collection, restrictions around who can access the data, and when the data needs to be destroyed? Please provide a copy of the consent form(s) if convenient.

Linking to administrative data sources

  • Is it possible to link the evaluation dataset to other administrative data sources?

  • If not, why not?

Please indicate whether you think it’s possible to link the evaluation dataset to other administrative data. If not, please indicate the reason you do not think it is possible (e.g. consent form restrictions, previous IRB rulings, lack of personally identifying information, etc.).



Study and findings

Study design

Please indicate the research design used. For example, randomized control trial, matching design, etc.

Study enrollment period

Please indicate the range of dates when participants entered the study.

Number of sites

Please verify the information that we prepopulated.

Site location(s)

Please verify the information that we prepopulated.

Provider name(s)

Please verify the information that we prepopulated.

Sample size

  • Overall

  • By research group

Please verify the information that we prepopulated.

Sample characteristics

Please characterize the general characteristics of the sample. For example, their age and other demographics that are relevant to the intervention.

Intervention type and key services tested

For example, sectoral training or job search assistance.

Key outcomes in original study (e.g. primary outcomes)

Please specify the key/primary outcomes. And how these were measured. For example: employment status as defined as employment status 2 quarters after program exit.

Strength of findings

  • Size of impact

  • Consistency

  • Timing

  • Outcomes

Please characterize the overall pattern of impacts on key/primary outcomes only. Please specify the direction, size, and statistical significance of the key outcomes. The size of the impact should be indicated in percentage or dollar terms, or as an effect size.

Treatment contrast

  • Were there participation impacts?

  • On what services?

Please indicate whether there were participation impacts and if so, what services had impacts. Would you characterize these impacts as “large” or “small?” Please focus on the key services that were tested in this study.

Control group embargo

  • Length of embargo

  • Ineligible services/programs

Please indicate the length of the embargo and what services were covered by the embargo.

Follow-up in original study

  • Length of follow-up

  • Data sources used

Please indicate the common follow-up period – in other words, the amount of follow-up that the latest entrants have. Please also indicate the data sources that were used in the impact analysis.





File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorKelsey Schaberg
File Modified0000-00-00
File Created2021-01-14

© 2024 OMB.report | Privacy Policy