ETA SOLICITATION FOR GRANT APPLICATIONS: SGA/DFA PY 11-13
U.S. DEPARTMENT OF LABOR
Employment and Training Administration
Notice of Availability of Funds and Solicitation for Grant Applications for Pay for Success Pilot Projects
Announcement Type: Solicitation for Grant Applications (SGA)
Funding Opportunity Number: SGA/DFA PY 11-13
Catalog of Federal Domestic Assistance (CFDA) Number: 17.283
Key Dates: The closing date for receipt of applications under this announcement is [insert date 180 days after date of publication in Federal Register]. Applications must be received no later than 4:00 p.m. Eastern Time.
Addresses: Mailed applications must be addressed to the U.S. Department of Labor, Employment and Training Administration, Office of Grants Management, Attention: Eric Luetkenhaus, Grant Officer, Reference SGA/DFA PY 11-13, 200 Constitution Avenue, NW, Room N4716, Washington, DC 20210. For complete application and submission information, including online application instructions, please refer to Section IV.
Notice of Intent to Apply: We will be able to develop a more efficient process for reviewing grant applications if we understand the number of applicants that intend to apply for funding under this competition. Therefore, it is strongly encouraged for each potential applicant to notify ETA of the applicant's intent to submit an application for funding by sending a short e-mail message. This short e-mail should provide (1) the applicant organization's name and address, (2) a general overview of the Pay for Success project proposal, including the proposed workforce issue, target population to be addressed, and anticipated outcome(s) the project intends to achieve, (3) any preliminary information on the organizations you hope to partner with. It is requested that this e-mail be sent to [email protected] with ``Intent to Apply'' in the e-mail subject line by 8/31/12. This information will not be publically disclosed and final submitted applications are not required to align with the information in this Notice of Intent to Apply. Additionally, the information and preliminary project proposal will not have any bearing on the final evaluation of a submitted application. Applicants that do not provide this e-mail notification may still apply for funding.
Summary:
The Employment and Training Administration (ETA), U.S. Department of Labor (DOL, or the Department), announces the availability of approximately
$20 million in Pay for Success grants, funded out of the Workforce Innovation Fund in the Department of Labor Appropriations Act, 2012 (P.L. 112-74, Div. F, Tit. I). The Workforce Innovation Fund supports innovative approaches to the design and delivery of employment and training services that generate long-term improvements in the performance of the public workforce system, both in terms of positive results for job seekers and employers and cost-effectiveness. Grants awarded under this SGA will fund pilots of a Pay for Success model, an innovative funding strategy for achieving specific social service outcomes.
The Full-Year Continuing Appropriations Act of 2011 established the Workforce Innovation Fund (the Fund). The Fund invests in projects that demonstrate new, innovative strategies, or replicate effective evidence-based strategies which align and strengthen the employment outcomes for program beneficiaries. Additional funds were provided in the Department of Labor Appropriations Act, 2012 (P.L. 112-74). Out of the total FY 2011 and FY 2012 funds for the Workforce Innovation Fund, approximately $147 million will be awarded through the first Workforce Innovation Fund Solicitation for Grant Application (SGA) (SGA/DFA PY 11-05). Up to $20 million of the remaining money will be used for this second solicitation to fund grants that pilot a Pay for Success model, an innovative funding strategy for achieving specific social objectives. Grant awards under this solicitation must be made no later than September 30, 2013.
Background on the Pay for Success Model
The Pay for Success (PFS) model is a new way of financing social services to help governments target limited dollars to achieve a positive, measurable outcome. The PFS model offers a financing solution for preventative social services1, which are often the first services to get cut in austere budget times, even though in many cases they lead to long term benefits to the workforce system, such as reduced programmatic costs, more efficient public spending, and better social outcomes. At a time when all levels of government are facing cutbacks, Pay for Success offers a new approach to invest in services for vulnerable populations2 that need more – not less – support, while at the same time saving money for the public sector.
Under the Pay for Success financing model, a government agency commits funds as the financial funding agency to pay for specific target outcomes that are achieved within a given timeframe. A key feature of the Pay for Success concept is that the financial capital to cover the total operating costs of achieving the target outcome is provided by independent private, philanthropic, or other social investors for the entire period of performance of the project. The investors’ motivation for accepting the risks of funding the project is an expectation of a return on their investment. Payment of the committed funds by the government agency is contingent on achievement of results. Depending on the payment criteria and outcomes used, the investor may achieve a positive return in addition to repayment of the principal investment. Ideally, some or all of the financial return could be re-invested into further social capital initiatives. In this way, the model is different from how government agencies typically fund services; government funding is shifted from paying for specific processes and services to paying for specific outcomes.
The potential benefits of the Pay for Success model for the workforce investment system include:
Overcoming challenges of blending program- and population-specific government funds by providing flexible investor funding which could be used for integrated interventions for at-risk populations.
Creating incentives for social innovation, improved workforce outcomes, public sector cost savings, and efficiency gains.
Identifying effective preventative services that generate cost savings to the government. For instance, by preventing or shortening unemployment or under-employment, or addressing skills deficiencies, existing workforce system dollars can be spent more efficiently and reach more workers in need of employment services or training.
Allowing for more rapid learning about which methods and services work and which do not. Since the Pay for Success model allows for flexible management of service delivery strategies, providers will continually adapt their approach to improving results that are linked to payments. Through this mechanism, outcomes are incentivized in ways that encourage market-style efficiencies.
Providing a model for government investment in preventative and innovative service delivery models that transfer risks to the private sector.
Pay for Success Pilot funding design under this solicitation
In piloting the Pay for Success model, the Department will provide funding for one or more projects that will demonstrate the feasibility and viability of this funding model for providing positive workforce outcomes. The Department’s objectives in establishing this pilot are to:
Test a model for government investment in preventative and innovative service delivery models that transfers risk to the private sector.
Learn whether the Pay for Success concept is feasible in the workforce development policy arena.
Determine whether preventative social services complementing workforce development programming “pays off.”
In the long-term, those interventions that produce proven, positive results and cost efficiencies through the Pay for Success model may be scaled-up and replicated by Federal agencies and state/local/tribal partners. Furthermore, successful projects will demonstrate the feasibility of the Pay for Success financing model, providing the evidence necessary for state, local, and tribal governments to pursue the Pay for Success approach on their own in the future.
Given that this is a pilot testing the Pay for Success concept, a relatively short time horizon for grant activities is critical. For example, a project that intends to increase rates of high school graduation by providing intensive services and supports for high-risk high school students would be achievable within the grant period of performance, while a project with a much longer time horizon for measuring results (for example, early high school interventions with outcomes of post-secondary education credential attainment and employment), would not. The period of performance for these grants is described in Section II.B.
Eligible applicants under this solicitation are state, local, or tribal government entities. To apply, these entities must enter into an agreement with an intermediary organization for the achievement of specific workforce-related outcomes. On the basis of this agreement, the intermediary will raise capital from philanthropic, private sector, and/or other social investors, manage the delivery of services, and be responsible for achieving outcomes and overall cost savings to the public sector as negotiated with the applicant. The agreement between the state, local, or tribal government applicant and the intermediary organization must include all of the following:
A well-defined problem, and associated target population;
A preventative service delivery strategy that is managed, coordinated, and guided by the intermediary, is flexible and adaptive to the target problem and population, and has either an evidence-based history of success, or a justifiable level of confidence for success;
A commitment of funds from independent investors to cover all operating costs of the intervention, including administrative and overhead costs of the intermediary, if needed;
One or more well-defined, achievable outcome target(s) that are an improvement on the current condition of the target population and have been agreed to by all required project partners;
A financial model that shows public sector savings significant enough so that a return on investment (ROI) may be provided to investors and so that additional cost savings or efficiency gains are also realized by the public sector; and
A payment arrangement between the applicant and the intermediary, to be triggered by the verified achievement of the proposed outcome(s) within the grant period. The plan must describe a validation methodology and a payment plan that is derived from quantifiable data, measures outcome targets for the target population relative to a well-defined comparison population or control group, and credibly demonstrates that achievement of the outcome targets is due to the intervention and not due to random chance, general economic conditions, or participant selection. This payment arrangement must include a plan and timeline describing each payment point that the project partners have agreed to and the corresponding outcome targets. Funds for this payment arrangement will come from the grant, after the validation process. Pay for Success models are best suited to a very small number of payment points that occur after the achievement of the outcome can be determined. Throughout the period of performance, the intervention’s operating funds will be provided by the investors.
Partnership Structure
We will award grants only to applicants that have a fully-formed partnership in place. This partnership must include the roles played by the four entities described below. For the purposes of the SGA, the entities fulfilling these four roles will be referred to as the “partners” or “partnership.”
State/local/tribal government agency: The state, local, or tribal government agency is the grant applicant and is the entity testing the Pay for Success financing model. The agency identifies and enters into a contractual agreement with the intermediary through which payment is made for the achievement of specific target outcomes. The Federal government strongly prefers there be a single payment point that occurs toward the end of the grant’s period of performance. Additionally, as the grantee, the agency is responsible for submitting necessary reports to the Department according to final grant agreement provisions.
The intermediary: The intermediary coordinates the Pay for Success strategy. It is the organization that 1) enters into a contract with the state, local or tribal government 2) is responsible for achieving the negotiated outcome(s) for the target population by contracting with service delivery providers,3) raises funds from investors to cover the operating costs of achieving the outcomes; 4) has the flexibility to change or modify its service delivery methods and providers; and 5) collects and shares data with the state/local/tribal government grantee as necessary to fulfill the grant agreement.
The investor(s): The investors are entities that seek to invest in promising social service interventions. They may be not-for-profit or for-profit entities interested in social investment. They accept the inherent risk of total non-payment (writing off the operating funds as a loss) if the target outcome(s) are not achieved, but are willing to invest based on their confidence in the proposed intervention, and guarantee from the grantee that they will be paid for outcomes (i.e. earn their principal investment back plus a return on investment) should the target outcome(s) be met. These investors agree to fund all of the operational and service delivery costs of the intermediary and service provider(s) throughout the lifetime of the proposed project. The investors may also have a vested interest in the strategies that the intermediary chooses to implement.
Independent outcome validator: The independent validator is directly contracted by the state, local, or tribal government grantee, and is responsible for verifying and validating whether outcome target(s), for which the Department will pay the grantee under this pilot, are met. The validator informs and agrees to the validation methodology, monitors the outcome measures(s), determines whether the outcome targets have been met, and provides the documentation to trigger release of Pay for Success payments by the Department to the grantee. The validator is paid by the grantee from grant funds throughout this process.
Over the life of the project, at least two other types of entities will be involved, as described below:
Service provider(s): Service providers are contracted by the intermediary to provide services and/or administer the programs and interventions designed by the intermediary.
Department of Labor: In this Pay for Success pilot, the Department of Labor will play a key supporting role to the partnership, acting as the funding agency that releases the final payment(s) to the grantee upon successful achievement of the outcome target(s). Upon verification of the achievement of negotiated outcomes by the independent validator, the Department will confirm that the validation methodology was followed and make the appropriate payments to the state/local/tribal government grantee. The grantee (state, local, or tribal government entity) in turn will pay the intermediary and the investor according to the plan to which the partners have agreed. If the outcomes are not achieved, the Department does not release the funds. To support grantees’ success and knowledge about Pay for Success, the Department will provide technical assistance on the Pay for Success financing strategy. The Department will also sponsor a separate third-party national evaluation of the Pay for Success pilots to document the design and implementation of the funding mechanism.
For this pilot solicitation, it is acceptable for one organization to play the role of both investor and intermediary, or both intermediary and service provider. We will not fund proposals in which one organization is playing the role of investor, intermediary, and service provider, or proposals in which one organization is playing the role of investor and service provider. The independent validator cannot play any additional roles in the partnership and cannot be an affiliate or subsidiary of any of the other entities in the partnership.
Target population and social problem
Selecting a well-defined problem and associated target population is essential to applying the Pay for Success financing model. Pay for Success models are best suited to fund preventative interventions that address the needs of a target group for which the existing provision of services has been insufficient or inefficient in achieving desired outcomes. Pay for Success is designed to address the systemic problems facing target populations by providing targeted prevention approaches and innovative service models.
We are particularly interested in projects that focus on expanding the availability of social services to address difficult workforce system problems, such as strategies intended to eliminate significant barriers to employment faced by at-risk, disadvantaged, and hard-to-employ populations (e.g., high school dropouts, homeless individuals, long-term unemployed, former prisoners). As a result, it will be important to design a validation methodology that ensures that the outcome target is not achieved through serving only low-cost, easy to serve customers, or ignoring high-cost, hard-to-serve customers. More information on outcome and validation design can be found in Section I. F and Section I. G.
Developing the Intervention
It is important to have a robust understanding of the total costs of delivering the intervention, including infrastructure and overhead costs, so that an accurate budget and cost-benefit analysis for the proposed intervention may be developed. This budget will determine the level of funding that will need to be raised from investors by the intermediary. For the Pay for Success pilots, we expect that the proposed interventions will be improvements on, or different from, currently existing government-funded services – such as Workforce Investment Act (WIA) services – offered in the target area(s). The partnership may design new interventions, scale up promising, yet unfunded, preventative services, or complement existing programs in innovative ways.
Outcomes
Well-defined, achievable outcome target(s) form the foundation of the Pay for Success concept. While outcomes are the positive impacts on the target population that the program intends to achieve as a result of the intervention, outcome targets are the specific levels of achievement that the project intends to attain for a particular outcome.
All partners need to agree that the outcome targets are specific, measurable, achievable, realistic for the period of performance of the grant, and result in cost savings. As discussed in Section G. below, all partners must also agree to an outcome validation process that compares the impact the intervention has on the target population to a well-defined comparison population or control group, and credibly demonstrates that the outcome target is achieved due to the intervention and not due to random chance, selection, or other economic conditions. Whether suitable outcome targets can be identified and agreed upon by the partnership is a key determinant of whether Pay for Success is the appropriate instrument for addressing the identified social issue. Definitions for outcome and outcome validation terms may be found in Attachment B: Glossary.
Examples of outcomes appropriate for a Pay for Success project in the workforce sector include a reduction in duration of receipt of unemployment insurance, increased earnings, increased job retention rates, increased education or skills credential attainment rates, reduced juvenile incarceration rates, etc. Examples of appropriate outcome targets include reducing average duration of receipt of unemployment insurance by x weeks or increasing retention in unsubsidized employment six months after entry into the employment by x percent. Outcome targets must be defined relative to the comparison or control group.
Validation Methodology
Projects funded under this solicitation are required to contract with an independent validator to assess whether the project met its outcome targets. As part of the validation methodology, you must be able to credibly demonstrate that outcome targets are achieved due to the intervention and not due to random chance, general economic conditions, or participant selection by defining the outcome target relative to a well-defined comparison population or control group. This approach also protects the government from risks of inflating outcome targets by comparing a narrowly-defined treatment group against a broader, harder-to-serve comparison group. You have two options for meeting this requirement.
You can measure your outcomes for your treatment group, or a broader target population that includes both those you intend to treat and those you don’t, and define your outcome target(s) relative to a comparison group. This approach requires a credible quasi-experimental design that deals appropriately with other possible causes of the outcomes, such as selection, other policies, and economic conditions.
You can measure your outcomes for your treatment group and define your outcome target relative to a control group. This approach requires experimental design methodology that uses random assignment to create treatment and control groups.
For more information on quasi-experimental design and random assignment as it relates to selection of comparison and control groups, see Section V.A. Criterion 4 and Attachment B: Glossary.
Outcome Valuation
The outcome valuation is the public sector cost savings resulting from achieving the outcome target(s). The Pay for Success model works when the cost of achieving the outcome(s) and providing a return on investment to investors costs less than the public sector’s current cost of meeting the needs of the target population. The difference between the current costs and proposed costs of the intervention represent cost savings or efficiency gains for the public sector. Under the Pay for Success model, these cost savings must be tied to achieving each proposed outcome for the target population. You must describe your approach to valuing the savings expected from the project, and demonstrate that the investors’ proposed return on investment amount is reasonable compared to the expected overall savings. For the purposes of this solicitation, these savings may be defined narrowly in terms of the cost savings accruing to specific public sector budgets, such as the savings that result from shorter unemployment insurance duration, or reduced criminal recidivism, as well as broadly, in terms of efficiency gains, such as reduced workload for WIA core services, better allocation of resources, and/or more efficient use of resources. You may identify public sector savings at the Federal, state, local, and tribal levels; however, to support the Department’s goal of documenting the feasibility of this financing approach, some cost savings at the applicant’s level of government must be demonstrated.
Developing the Financial Model
You must describe a financial model and payment plan that links outcome targets, outcome valuation, and payment points. Your financial model must reflect the costs and savings associated with the specific application of the Pay for Success model and the return on investment payment associated with success. The financial model should include: 1) the current cost of the problem to the public sector, with a primary emphasis on costs at the applicant’s level of government – either state, local, or tribal; 2) the cost of the proposed intervention over the timeline necessary to achieve the outcome target(s), including overhead and other fixed costs; and 3) a cost-benefit analysis that justifies the intervention and the return on investment and illustrates the savings that will accrue to the public sector based on the successful achievement of the outcome target(s).
The Department strongly prefers projects structured such that outcome target achievement is determined toward the end of the grant’s timeline, and a single payment request is made. We will consider projects with multiple payment points, but only if those payment points are tied to the achievement of validated outcome targets that are independent of each other. If you are proposing multiple payment points, you should clearly show that the payment structure does not create potential high-stakes thresholds for your partners, such as those that that might incentivize early project shutdowns once particular outcome target levels are met. For example, you should not propose an early payment point that pays out a disproportionate amount of the grant, thereby making the remaining years of the project economically unattractive to complete. Conversely, while project partners may properly terminate a project that has become impossible to achieve and accept the financial losses, tying payment points to unreasonable or unattainable outcome targets would create instances of non-performance and non-payment, which are detrimental to both project partners and individuals in the target population. Final release of grant funds by the Department to the grantee will only be made upon proof of the timely achievement of the identified outcome target(s), as described in the application and finalized with the Department in the provisions of the grant agreement for those awarded funding under this solicitation.
Because Pay for Success is only intended to pay for interventions that achieve pre-stated outcome target(s), extensions to the period of performance will not be allowed, and as a result, you should design your payment points to be measured and validated on a fixed timeline. You should note that, while the Pay for Success model is intended to reimburse the investor for project costs plus a return on investment, Federal grant limitations require us to obligate a set amount of funds up front for the purpose of this award. Therefore, the amount to be paid, if outcome target(s) are achieved, by the Federal government to the grantee, will be indicated in the grant agreement, and once the grant is awarded, will not be tied to actual final costs of implementation.
Pay for Success models are intended to incentivize and reward higher performance and program efficiencies; therefore they can include a payment structure that allows for greater rewards for higher performance on the proposed outcome. Under this solicitation, you may propose a payment plan that ties a fixed payment to each outcome target achieved, or you may propose a payment plan that rewards your partners for significantly exceeding the outcome target(s), by proposing higher payments for higher outcome target levels. For example, such a model could include an estimated return on investment of 0 percent for achieving an outcome target (meaning investors would break even), and a 1 percentage point increase in the return on investment (up to some maximum) for every percentage point improvement above the outcome target level. If you propose this tiered return approach you must show the highest possible payment amount on your application and budget forms, as this funding will need to be obligated to the project by DOL upon award. Regardless of the proposed outcome target and return on investment, the maximum payment and return on investment must still meet the grant amount criteria defined in Section II.A.
In keeping with the Pay for Success model, grantees and their partners will have flexibility in how the project is funded operationally, and in how they disburse payments made by the Department for achievement of outcome targets. However, you are strongly encouraged to apply the same methodology and rigor in establishing your plan for paying the intermediary as the one proposed for receiving payment from the Department.
Up to $20 million is available under this competition, and the Department expects to fund approximately one to three grants. The maximum individual grant amount is $12 million. There is no minimum amount for any one grant award. Any grant application with a proposed total value (including all forms and categories of cost) greater than $12 million will be deemed non-responsive and will not be considered. In the event additional funds become available, the Department reserves the right to use such funds to select additional grantees from applications submitted in response to this solicitation. If there are no applications of merit, funds reserved for this solicitation will be used to award additional grants under SGA/DFA PY 11-05.
B. Period of Performance
The period of performance for grant awards will be up to 48 months from the date of grant award. This period includes up to 36 months for all necessary start-up and implementation activities, including award of contracts to the service providers; and delivery of the intervention. Six months are available for measurement, documentation, and validation of outcome results; and submission of payment requests to the Federal government. The final six months of the grant period of performance are available for the Department to approve and disburse payment. DOL will begin reviewing payment requests as soon as they are received. The period of performance for these grants will not be altered or extended; therefore, applicants must carefully construct their project timeline to allow sufficient time for all required activities.
III. Eligibility Information
A. Eligible Applicants
In order to be eligible for consideration under this Solicitation, you must be an eligible government or government agency, as defined in this section. You must specify your applicant type in the Abstract, described in Section IV.B Part III. If you do not meet eligibility requirements of the SGA, we will consider your application non-responsive and it will not be reviewed. Eligible government or government agencies are:
State Workforce Agencies eligible for assistance under Title I of the Workforce Investment Act (WIA) of 1998;
Local government agencies responsible for workforce programs under Title I of WIA; or
A Federally recognized Indian tribal government.
Cost sharing or matching funds are not required as a condition for application. The financial contribution from investors is not considered cost sharing or matching funds.
C. Other Eligibility Criteria
Applications that do not meet the following eligibility criteria will be found non-responsive and will not be reviewed.
You may submit no more than one application. It is our intent that no applicant will be funded more than once under this SGA.
In keeping with the overarching goals of the Workforce Innovation Fund as authorized by the Department of Labor Appropriations Act, 2012 (P.L. 112-74), the identified social problem must fall under the scope of the public workforce investment system, including, but not limited to, unemployment, underemployment, skills deficiencies, poverty, etc.
Applicants must have a fully formed partnership, as demonstrated by a signed partnership agreement, which includes the entities acting in the roles of the government agency, the intermediary, the investor(s), and the independent validator.
Applicants, in partnership with the intermediary, must have secured a commitment from independent investor(s) to cover the costs of administering the intervention.
D. Eligible Participants
Participants Eligible to Receive Services
Participants must meet the characteristics of the proposed target population. There are no restrictions on target population eligibility.
The Jobs for Veterans Act (Public Law 107-288) requires grantees to provide priority of service for veterans and spouses of certain veterans for the receipt of employment, training, and placement services in any job training program directly funded, in whole or in part, by DOL. The regulations implementing this priority of service can be found at 20 CFR Part 1010. In circumstances where a grant recipient must choose between two qualified candidates for a service, one of whom is a veteran or eligible spouse, the veterans priority of service provisions require that the grant recipient give the veteran or eligible spouse priority of service by first providing him or her that service. To obtain priority of service, a veteran or spouse must meet the program’s eligibility requirements. Grantees must comply with DOL guidance on veterans’ priority. ETA’s Training and Employment Guidance Letter (TEGL) No. 10-09 (issued November 10, 2009) provides guidance on implementing priority of service for veterans and eligible spouses in all qualified job training programs funded in whole or in part by DOL. TEGL No. 10-09 is available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2816.
Transparency
DOL is committed to conducting a transparent grant award process and publicizing information about program outcomes. Posting grant applications on public websites is a means of promoting and sharing innovative ideas. For this grant competition, we will publish the abstracts required by Section IV.B. Part III. (a), for all applications on the Department’s public website or similar publically accessible location. Additionally, we will publish a version of the Technical Proposal required by Section IV.B. Part II, for all those applications that are awarded grants, on the Department’s website or a similar location. No other parts of or attachments to the application will be published. The Technical Proposals and Abstracts will not be published until after the grants are announced. In addition, information about grant progress and results may also be made publicly available.
DOL recognizes that grant applications sometimes contain information that an applicant may consider proprietary or business confidential information, or may contain personally identifiable information (PII). Proprietary or confidential commercial/business information is information that is not usually disclosed outside your organization and the disclosure of which is likely to cause you substantial competitive harm. PII is any information that can be used to distinguish or trace an individual‘s identity, such as name, social security number, date and place of birth, mother’s maiden name, or biometric records, and any other information that is linked or linkable to an individual, such as medical, educational, financial, and employment information.3
Abstracts will be published in the form originally submitted, without any redactions. However, in order to ensure that PII and proprietary or confidential commercial/business information is properly protected from disclosure when DOL posts the winning Technical Proposals, applicants whose technical proposals will be posted will be asked to submit a second redacted version of their Technical Proposal, with any proprietary, confidential commercial/business, and PII redacted. All non-public information about the applicant’s staff should be removed as well. The Department will contact the applicants whose technical proposals will be published by letter or email, and provide further directions about how and when to submit the redacted version of the Technical Proposal. Submission of a redacted version of the Technical Proposal will constitute permission by the applicant for DOL to make the redacted version publically available. If an applicant fails to provide a redacted version of the Technical Proposal, DOL will publish the original Technical Proposal in full, after redacting PII. (Note that the original, unredacted version of the Technical Proposal will remain part of the complete application package, including an applicant’s proprietary and confidential information and any PII.)
Applicants are encouraged to maximize the grant application information that will be publicly disclosed, and to exercise restraint and redact only information that is clearly proprietary, confidential commercial/business information, or PII. The redaction of entire pages or sections of the Technical Proposal is not appropriate, and will not be allowed, unless the entire portion merits such protection. Should a dispute arise about whether redactions are appropriate, DOL will follow the procedures outlined in the Department’s Freedom of Information Act (FOIA) regulations (29 CFR Part 70).
Redacted information in grant applications will be protected by DOL from public disclosure in accordance with Federal law, including the Trade Secrets Act (18 U.S.C. § 1905), FOIA, and the Privacy Act (5 U.S.C. § 552a). If DOL receives a FOIA request for your application, the procedures in DOL’s FOIA regulations for responding to requests for commercial/business information submitted to the government will be followed, as well as all FOIA exemptions and procedures (29 CFR § 70.26). Consequently, it is possible that application of FOIA rules may result in release of information in response to a FOIA request that an applicant redacted in its “redacted copy.”
This SGA contains all of the information and links to forms needed to apply for grant funding.
B. Content and Form of Application Submission
Proposals submitted in response to this SGA must consist of three separate and distinct parts: (I) a cost proposal; (II) a technical proposal; and (III) attachments to the technical proposal. Applications that do not contain all of the three parts or that fail to adhere to the instructions in this section will be deemed non-responsive and will not be reviewed. It is the applicant’s responsibility to ensure that the funding amount requested is consistent across all parts and sub-parts of the application.
Part I. The Cost Proposal. The Cost Proposal must include the following items:
SF-424, “Application for Federal Assistance” (available at http://apply07.grants.gov/apply/FormLinks?family=15 ). The SF-424 must clearly identify the applicant and must be signed by an individual with authority to enter into a grant agreement. Upon confirmation of an award, the individual signing the SF-424 on behalf of the applicant shall be considered the authorized representative of the applicant. As stated in block 21 of the SF-424 form, the signature of the authorized representative on the SF-424 certifies that the organization is in compliance with the Assurances and Certifications form SF-424B (available at http://apply07.grants.gov/apply/FormLinks?family=15 ). The SF-424B is not required to be submitted with the application. All applicants for Federal grant and funding opportunities are required to have a Data Universal Numbering System (D-U-N-S®) number, and must supply their D-U-N-S® Number on the SF-424. The D-U-N-S® Number is a nine-digit identification number that uniquely identifies business entities. If you do not have a D-U-N-S® Number, you can get one for free through the D&B website: http://fedgov.dnb.com/webform/displayHomePage.do.
The SF-424A Budget Information Form (available at http://apply07.grants.gov/apply/FormLinks?family=15 ). In preparing the Budget Information Form, the applicant must provide a concise narrative explanation to support the budget request, explained in detail below.
Budget Narrative: The budget narrative must provide a description of costs associated with each line item on the SF-424A. It should also include a description of leveraged resources provided (as applicable) to support grant activities. You must detail the amount of funds budgeted for the following costs:
Your own program management and oversight responsibilities as the government applicant under the grant
The amount budgeted for the contract with the independent validator
The amount budgeted for the contract with the intermediary (which includes all costs related to the intervention, including all intermediary overhead and operating costs, and any return on investment). If you are using a payment structure that includes receiving greater payments for higher performance levels on the proposed outcome, you must include in your budget narrative and corresponding SF-424 application and SF-424A budget forms the full amount needed pay out at the highest possible level of proposed return on investment.
We expect that costs for (a) and (b) will be charged to the grant as these costs are incurred. Costs for (c) must be integrated into the Pay for Success financing model described in Section V.A. Criteria 5, and will only be paid by the Department once the outcome(s) are validated and submitted to DOL ETA.
Note that the entire Federal grant amount requested (not just one year) must be included on the SF-424 and SF-424A and budget narrative. No leveraged resources should be shown on the SF-424 and SF-424A. The amount listed on the SF-424, SF-424A and budget narrative must be the same. Please note, the funding amount included on the SF-424 will be considered the official funding amount requested if any inconsistencies are found. Applications that fail to provide an SF-424 including D-U-N-S® Number, SF-424A, and a budget narrative will be considered non-responsive and will not be reviewed.
Regardless of the method of application submission, all applicants must register with the Federal Central Contractor Registry (CCR) before submitting an application. Step-by-step instructions for registering with CCR can be found at http://www.grants.gov/applicants/org_step2.jsp. An awardee must maintain an active CCR registration with current information at all times during which it has an active Federal award or an application under consideration. To remain registered in the CCR database after the initial registration, the applicant is required to review and update on an annual basis from the date of initial registration or subsequent updates its information in the CCR database to ensure it is current, accurate and complete. For purposes of this paragraph, the applicant is the entity that meets the eligibility criteria and has the legal authority to apply and to receive the award. Failure to register with the CCR before application submission will result in your application being found non-responsive and will not be reviewed.
Part III. Attachments to the Technical Proposal. In addition to the Technical Proposal, the applicant must submit the following required attachments:
An abstract (See Attachment A for a suggested template), not to exceed three pages, which will serve as a summary of the proposal. The abstract will be shared publicly, and must include the following information: 1) applicant’s name and type; 2) applicant city/state; 3) geographic areas served by the intervention; 4) project name; 5) funding level requested; 6) names of entities serving as the intermediary, investor, and validator; 7) description of the problem and target population’s characteristics; 8) description of the outcome target(s) that will trigger government payment and the intervention that will produce the outcomes; 9) expected cost savings and/or efficiency gains; and 10) public contact information for the grant, which may be an email or website. If using Grants.gov for submission, this document must be attached under the Mandatory Other Attachment section and labeled “abstract.”
An SF-424, “Application for Federal Assistance” (available at http://apply07.grants.gov/apply/FormLinks?family=15). If using Grants.gov for submission, this form must be attached under the required forms section. Please note that this is a standard form used for many programs and has a check box for applying as an individual. Disregard this box on the form as individuals are not eligible to apply for this solicitation;
A letter of commitment from the investor(s) to provide working capital adequate to sustain the life of the project.
A partnership agreement signed by representatives of the state/local/tribal government applicant, intermediary, investor(s) and independent validator confirming that all partners are in agreement with the roles and responsibilities within the framework of the Pay for Success model and all components of the applicant’s grant proposal.
Applications that do not include the required attachments will be considered non-responsive and will not be reviewed.
Only those attachments listed above as required attachments will be excluded from the page limit. The required attachments must be affixed as separate, clearly identified appendices to the application. Additional materials such as résumés or general letters of support or commitment will not be considered.
The closing date for receipt of applications under this announcement is [insert date 180 days after date of publication in Federal Register]. Applications may be submitted electronically on http://www.grants.gov or in hard copy by mail or hand delivery (including overnight delivery). Hard copy applications must be received at the address below no later than 4:00 p.m. Eastern Time. Applications submitted on Grants.gov must also be successfully submitted (as described below) no later than 4:00 p.m. Eastern Time on the closing date. Applications sent by e-mail, telegram, or facsimile (FAX) will not be accepted.
Applicants submitting proposals in hard copy must submit an original signed application (including the SF-424) and one (1) ‘‘copy-ready’’ version free of bindings, staples or protruding tabs to ease in the reproduction of the proposal by DOL. Applicants submitting proposals in hard copy are also required to provide an identical electronic copy of the proposal on compact disc (CD). If discrepancies between the hard copy submission and CD copy are identified, the application on the CD will be considered the official applicant submission for evaluation purposes. Failure to provide identical applications in hardcopy and CD format may have an impact on the overall evaluation.
If an application is physically submitted by both hard copy and through http://www.grants.gov, a letter must accompany the hard-copy application stating which application to review. If no letter accompanies the hard copy, we will review the copy submitted through http://www.grants.gov.
Applications that do not meet the conditions set forth in this notice will be considered non-responsive. No exceptions to the mailing and delivery requirements set forth in this notice will be granted. Further, documents submitted separately from the application, before or after the deadline, will not be accepted as part of the application.
Mailed applications must be addressed to the U.S. Department of Labor, Employment and Training Administration, Office of Grants Management, Attention: Eric Luetkenhaus, Grant Officer, Reference SGA/DFA PY 11-13, 200 Constitution Avenue, NW, Room N4716, Washington, DC 20210. Applicants are advised that mail delivery in the Washington DC area may be delayed due to mail decontamination procedures. Hand-delivered proposals will be received at the above address. All overnight mail will be considered to be hand-delivered and must be received at the designated place by the specified closing date and time.
Applications that are submitted through Grants.gov must be successfully submitted at http://www.grants.gov no later than 4:00 p.m. Eastern Time on the closing date and then subsequently validated by Grants.gov. The submission and validation process is described in more detail below. The process can be complicated and time-consuming. Applicants are strongly advised to initiate the process as soon as possible and to plan for time to resolve technical problems if necessary.
The Department strongly recommends that before the applicant begins to write the proposal, applicants should immediately initiate and complete the “Get Registered” registration steps at http://www.grants.gov/applicants/get_registered.jsp. Applicants should read through the registration process carefully before registering. These steps may take as much as four weeks to complete, and this time should be factored into plans for electronic submission in order to avoid unexpected delays that could result in the rejection of an application. The site also contains registration checklists to help you walk through the process. The Department strongly recommends that applicants download the “Organization Registration Checklist” at http://www.grants.gov/assets/Organization_Steps_Complete_Registration.pdf and prepare the information requested before beginning the registration process. Reviewing and assembling required information before beginning the registration process will alleviate last minute searches for required information and save time.
As described above, applicants must have a D–U–N–S® Number and must register with the Federal Central Contractor Registry (CCR).
The next step in the registration process is creating a username and password with Grants.gov to become an Authorized Organizational Representative (AOR). AORs will need to know the D-U-N-S® Number of the organization for which they will be submitting applications to complete this process. To read more detailed instructions for creating a profile on Grants.gov visit: http://www.grants.gov/applicants/org_step3.jsp.
After creating a profile on Grants.gov, the E-Biz point of Contact (E-Biz POC) - a representative from your organization who is the contact listed for CCR – will receive an email to grant the AOR permission to submit applications on behalf of their organization. The E-Biz POC will then log in to Grants.gov and approve an applicant as the AOR, thereby giving him or her permission to submit applications. To learn more about AOR Authorization visit: http://www.grants.gov/applicants/org_step5.jsp, or to track AOR status visit: http://www.grants.gov/applicants/org_step6.jsp.
An application submitted through Grants.gov constitutes a submission as an electronically signed application. The registration and account creation with Grants.gov, with E-Biz POC approval, establishes an AOR. When you submit the application through Grants.gov, the name of your AOR on file will be inserted into the signature line of the application. Applicants must register the individual who is able to make legally binding commitments for the applicant organization as the AOR; this step is often missed and it is crucial for valid submissions.
When a registered applicant submits an application with Grants.gov, an electronic time stamp is generated within the system when the application is successfully received by Grants.gov. Within two business days of application submission, Grants.gov will send the applicant two email messages to provide the status of the application’s progress through the system. The first email, sent almost immediately, will contain a tracking number and will confirm receipt of the application by Grants.gov. The second email will indicate the application has either been successfully validated or has been rejected due to errors. Grants.gov will reject applications if the applicant’s CCR is expired. Only applications that have been successfully submitted by the deadline and subsequently successfully validated will be considered. It is the sole responsibility of the applicant to ensure a timely submission. While it is not required that an application be successfully validated before the deadline for submission, it is prudent to reserve time before the deadline in case it is necessary to resubmit an application that has not been successfully validated. Therefore, sufficient time should be allotted for submission (two business days) and, if applicable, additional time to address errors and receive validation upon resubmission (an additional two business days for each ensuing submission). It is important to note that if sufficient time is not allotted and a rejection notice is received after the due date and time, the application will not be considered.
To ensure consideration, the components of the application must be saved as .doc, .docx, .xls, .xlsx, .rtf or .pdf files. If submitted in any other format, the applicant bears the risk that compatibility or other issues will prevent us from considering the application. ETA will attempt to open the document but will not take any additional measures in the event of problems with opening. In such cases, the non-conforming application will not be considered for funding.
We strongly advise applicants to use the various tools and documents, including FAQs, which are available on the “Applicant Resources” page at http://www.grants.gov/applicants/app_help_reso.jsp.
ETA encourages new prospective applicants to view the online tutorial, “Grant Applications 101: A Plain English Guide to ETA Competitive Grants,” available through Workforce3One at: http://www.workforce3one.org/page/grants_toolkit.
To receive updated information about critical issues, new tips for users and other time sensitive updates as information is available, applicants may subscribe to “Grants.gov Updates” at http://www.grants.gov/applicants/email_subscription_signup.jsp.
If applicants encounter a problem with Grants.gov and do not find an answer in any of the other resources, call 1-800-518-4726 or 606-545-5035 to speak to a Customer Support Representative or email “[email protected]”. The Contact Center is open 24 hours a day, seven days a week. It is closed on Federal holidays.
Late Applications: For applications submitted on Grants.gov, only applications that have been successfully submitted no later than 4:00 p.m. Eastern Time on the closing date and then successfully validated will be considered. Applicants take a significant risk by waiting to the last day to submit by Grants.gov.
Any hard copy application received after the exact date and time specified for receipt at the office designated in this notice will not be considered, unless it is received before awards are made, it was properly addressed, and it was: (a) sent by U.S. Postal Service mail, postmarked not later than the fifth calendar day before the date specified for receipt of applications (e.g., an application required to be received by the 20th of the month must be postmarked by the 15th of that month); or (b) sent by professional overnight delivery service to the addressee not later than one working day before the date specified for receipt of applications. ‘‘Postmarked’’ means a printed, stamped or otherwise placed impression (exclusive of a postage meter machine impression) that is readily identifiable, without further action, as having been supplied or affixed on the date of mailing by an employee of the U.S. Postal Service. Therefore, applicants should request the postal clerk to place a legible hand cancellation ‘‘bull’s eye’’ postmark on both the receipt and the package. Failure to adhere to these instructions will be a basis for a determination that the application was not filed timely and will not be considered. Evidence of timely submission by a professional overnight delivery service must be demonstrated by equally reliable evidence created by the delivery service provider indicating the time and place of receipt.
This funding opportunity is not subject to Executive Order 12372, “Intergovernmental Review of Federal Programs.”
Determinations of allowable costs for the grantee (government entity) will be made in accordance with the applicable Federal cost principles. Disallowed costs are those charges to a grant that the grantor agency or its representative determines not to be allowed in accordance with the applicable Federal cost principles or other conditions contained in the grant. Note that the applicant’s relationship with the intermediary and validator must be through contracts and should follow the applicable Federal contract guidelines. All proposed project costs, grantee and vendor, must be necessary and reasonable and in accordance with Federal guidelines. [As contractors, the intermediary and validator will not follow Federal cost principles for grants or Uniform Administrative Requirements for grants; rather they will have to be in accordance with state law.] Neither applicants nor organizations awarded grants under this Solicitation will be entitled to reimbursement of pre-award costs.
4. Intellectual Property Rights
This section is applicable to activities of the direct grantee and the validator, and to the activities of the intermediary (including service provider activities) if outcomes are achieved and final payments are made.
The Federal government reserves a paid-up, nonexclusive and irrevocable license to reproduce, publish, or otherwise use, and to authorize others to use for Federal purposes: i) the copyright in all products developed under the grant, including a subgrant or contract under the grant or subgrant; and ii) any rights of copyright to which the grantee, subgrantee or a contractor purchases ownership under an award (including but not limited to curricula, training models, technical assistance products, and any related materials). Such uses include, but are not limited to, the right to modify and distribute such products worldwide by any means, electronically or otherwise. The grantee may not use Federal funds to pay any royalty or license fee for use of a copyrighted work, or the cost of acquiring by purchase a copyright in a work, where the Department has a license or rights of free use in such work. If revenues are generated through selling products developed with grant funds, including intellectual property, these revenues are program income. Program income is added to the grant and must be expended for allowable grant activities.
If applicable, the following needs to be on all products developed in whole or in part with grant funds:
“This workforce product was funded by a grant awarded by the U.S. Department of Labor’s Employment and Training Administration. The product was created by the grantee and does not necessarily reflect the official position of the U.S. Department of Labor. The U.S. Department of Labor makes no guarantees, warranties, or assurances of any kind, express or implied, with respect to such information, including any information on linked sites and including, but not limited to, accuracy of the information or its completeness, timeliness, usefulness, adequacy, continued availability, or ownership. This product is copyrighted by the institution that created it. Internal use by an organization and/or personal use by an individual for non-commercial purposes is permissible. All other uses require the prior authorization of the copyright owner.”
Withdrawal of Applications: Applications may be withdrawn by written notice to the Grant Officer at any time before an award is made.
This section identifies and describes the criteria that will be used for each category to evaluate grant proposals. The evaluation criteria are described below:
Criterion |
Points |
|
20 |
|
10 |
|
25 |
|
20 |
|
25 |
TOTAL |
100 |
|
|
Criterion 1: Pay for Success Partnership (20 points).
To be eligible for funding under this Solicitation, you must provide evidence of a fully-formed partnership consisting of entities filling the following required roles: the state/local/tribal government applicant; an intermediary; an investor(s); and an independent outcome validator. The partners must agree to 1) the problem(s) to be solved; 2) the outcome target(s) to be achieved; 3) the outcome measure(s), associated data, and methodology that will be used to verify achievement of the outcome target(s); and 4) the payment(s) to be made based on the achievement of the outcome target(s). In your proposal, you must identify the members of your partnership; describe their roles and responsibilities; and provide evidence that all partners are in agreement with the four project elements listed above. Both the intermediary organization and the independent validator are crucial to the ultimate success of the model. Therefore, you must also describe the qualifications of these two partners. You must also provide a detailed work plan and project management approach that demonstrates your ability get the project started on time. Points will be awarded based on the following factors:
Factor 1: Partnership Agreement (5 points). We will award points for this factor based on the extent to which you address the following:
You demonstrate that your partnership includes an intermediary, at least one investor (the intermediary and investor may be the same organization), and an independent validator.
You fully describe their roles and responsibilities of each major project partner.
You provide evidence through a signed partnership agreement that all partners are in agreement with all critical elements of the project utilizing the Pay for Success model, including but not limited to: 1) the problem(s) to be solved; 2) the outcome target (s) to be achieved; 3) outcome measure(s), associated data, and methodology to be used to verify achievement of those outcome targets(s); 4) the payment(s) to be made based on the achievement of the outcome targets(s); and 5) any other roles and responsibilities within the framework of the Pay for Success model as described in Sections I.B and I.C.
Factor 2: Partnership Qualifications (10 points). We will award points for this factor based on the extent to which you address the following:
You demonstrate that your intermediary partner has experience and qualifications for performing its three primary roles: 1) raising working capital funds; 2) administering complex outcome-based social service projects, including collecting outcome data, managing projects for continuous improvement, and adapting the intervention to achieve the outcome target(s); and 3) managing partnerships. Experience and qualifications may be demonstrated through examples of past work performed by the organization or its staff, or through other examples of documented competency.
You demonstrate that the entity you have selected to serve as the independent validator is capable of validating your project’s outcome measure(s) and outcome target(s). Experience and qualifications may be demonstrated by illustrating the entity’s prior success in validating outcomes and provide evidence of the quality of that work, as performed by the validator in the past five years.
You identify the professional qualifications you will require of your project manager and demonstrate that these qualifications are sufficient to ensure proper management of your Pay for Success project.
You, as the applicant, provide a brief description of your past experience and/or management capacity to work with Technical Assistance and Evaluation coordinators.
For your most recent grant or grants (formula or competitive), you demonstrate that you submitted reports (program and financial) on-time and describe the grant management practices you used to complete grant activities within the period of performance.
Factor 3: Work Plan for Timely Project Start Up (5 points): We will award points for this factor based on the extent to which you address the following:
You present a coherent set of milestones that demonstrate your complete understanding of all responsibilities required to start up your project. Your project startup milestones must indicate when the following will be completed: any necessary preliminary planning; the execution of the contract with the intermediary; the execution of the contract with the validator; the commitment of operational funds from the investor(s); the start date of a project manager; and the commencement of service delivery for the target population.
You present evidence of readiness to begin implementing the grant immediately upon grant award by including a feasible and reasonable timeframe for accomplishing all procurement and other necessary grant start-up strategies mentioned above within the first two months of the anticipated grant start date
Criterion 2: Description of the Problem and Target Population (10 points).
You must identify the target population and a specific employment and training related problem that will be the focus of your intervention strategy, and demonstrate that the chosen target population and problem represent opportunities for potential financial savings to existing Federal/state/local/tribal government-funded services through an intervention different from what the public sector is currently providing. We will award points for this criterion based on the extent to which you address the following:
You clearly define the identifying characteristics of the target population, for example, age, gender, ethnicity, or other demographics, such as census tract, etc. The target population may be an overall target population or an intent-to-treat group depending on how you structure your outcome target validation methodology.
You clearly identify the employment and training problem facing your target population using data and other relevant information.
You explain why you selected this problem and target population by describing the extent to which the problem(s) affects the target population; documenting the severity of the problem by citing appropriate research/data sources or logical constructs, and providing quantitative and/or qualitative evidence, as available; and clearly documenting the current cost of the problem for your target population to the public sector.
You identify the barriers that have prevented you from addressing the problem or realizing cost savings with currently available government resources.
Criterion 3: Outcome(s), Outcome Target(s), and Intervention Design (25 points).
You must clearly define the outcome target(s), developed and agreed to by the project partners and the service delivery strategy that will be used to achieve the outcome target(s). You must demonstrate that the outcome target(s) is reasonable, and that the intervention is likely to succeed and can be modified if necessary. We will award points for this criterion based on the extent to which you address the following factors:
Factor 1: Outcome Target(s) (15 points). Your outcome target(s) must be defined in terms of the specific measurable effect of the service delivery strategy on the target population, as expressed by your outcome measure(s), and must be clearly distinct from the project’s outputs (i.e. the number of participants enrolled, or trainings completed). The outcome target(s) must not be narrowly defined in terms of individual activities executed as part of the service delivery design (i.e. the individual results of participants). We will award points for this factor based on the extent to which you address the following:
You describe your outcome target(s), showing that it is specific, measurable, achievable, realistic, and can be achieved within the timeframe of the grant. Outcome targets can be a defined change of an outcome measure or a percentage improvement of that outcome measure when compared to a comparison or control group. Outcome targets must be defined relative to the comparison or control group (e.g. a reduction of, or improvement to, an outcome measure relative to a comparison or control group, as described in Criterion 4).
You describe the outcome measure(s) that you will use to ascertain the level of achievement the project attains. The outcome measure must be expressed as a formula that provides an assessment of a program’s impact and is applied to both target and comparison groups (e.g. an outcome measure could be expressed as an equation, clearly identifying the units of measurement used in the numerator and the denominator). The outcome measure is determined using relevant program data and has defined units of measurement by which the impact can be tracked.
You demonstrate how the proposed outcome target(s) is an improvement over the current state of the target population, through the use of historical data and/or other arguments as needed.
Factor 2: Intervention Design (10 points). You must present a narrative theory of change, supported by evidence as available, that: 1) explains how your proposed intervention will generate the outcome target(s) you identified in Factor 1; and 2) identifies a set of connected actions or steps that will generate the change necessary to realize the project’s outcome target(s). A compelling theory of change: 1) identifies key assumptions upon which the intervention is based; 2) provides a set of testable hypotheses that can be used to measure the effect of the proposed strategy; and 3) describes interim outputs that provide indications of your program’s progress toward the agreed-upon outcome target(s). For more information about logic models and developing theories of change, please visit http://www.doleta.gov/workforce_innovation. You must also show that the partnership is capable of adapting the intervention strategy as needed. We will award points for this factor based on the extent to which you address the following:
You clearly describe the intervention strategy the intermediary plans to use to achieve the negotiated target outcome(s) and provide a compelling theory of change.
You provide compelling evidence (quantitative, qualitative, and/or theoretical) that suggests that the intervention strategy is likely to be successful.
You describe the process the partnership will use to determine when changes in the intervention strategy are needed during the course of the project in order to achieve the outcome target(s).
Criterion 4: Independent Validation of Outcome Targets (20 points).
You must provide the process and mechanism agreed upon by the project partners that the validator will use to assess if the outcome target(s) has been met. As part of the validation methodology, you must be able to credibly demonstrate that your outcome target(s) will be achieved due to the intervention and not due to random chance or participant selection by defining the outcome target relative to a well-defined comparison population or control group, as described in Section I.G. You must provide dispute resolution procedures if all project partners cannot agree to the validator’s assessment of the achievement of the outcome target(s) at the end of the intervention period. We will award points for this criterion based on the extent to which you address the following factors:
Factor 1: Independent Validator Responsibilities (3 points). You must describe how the independent validator will perform the following the following tasks:
Designing and coding a management information system (MIS), as needed, that is tailored for research or evaluation, to track participants and obtain individual-level data.
Collecting or assessing individual-level data. The independent validator must work directly with the applicant and other organizations to enter into an agreement for the access and use of the data.
Obtaining other relevant datasets from various sources, for example, local workforce investment boards, state agencies, or other Federal agencies beyond DOL and ETA.
Verifying the data through site visits and other methods.
Factor 2: Validation Methodology (10 points). You must describe the methodology the independent validator will use to validate achievement of the outcome target(s). The methodology must be objective, transparent, and independently verifiable. As part of the validation methodology, you must be able to credibly demonstrate that outcome targets were achieved due to the intervention and not due to random chance, general economic conditions, or participant selection by defining the outcome target relative to a well-defined comparison population or control group. We will award points for this factor based on the following:
You must describe the methodology, including the data (as described in Criterion 4, Factor 3) and the outcome measure (as described in Criterion 3, Factor 1), that the independent validator will use to validate the final outcome target(s) results,. Your methodology must be objective, transparent, and independently verifiable. You must demonstrate how your proposed validation methodology is appropriate given your project goals, and provide your step-by-step approach to implementing that methodology.
You must describe which of the two options described in I.G. you will use to measure the achievement of the outcome target relative to a well-defined comparison or control group.
If you measure your outcomes for your treatment group, or a broader target population that includes both those you intend to treat and those you don’t, and define your outcome target(s) relative to a comparison group using a credible quasi-experimental design that deals appropriately with other possible causes of the outcomes, such as selection, other policies, and economic conditions, you must identify the method you will use, demonstrate that it is appropriate given your project goals, and provide your step-by-step methodology to implement the design.
If you measure your outcomes for a treatment group and define your outcome target relative to a control group using experimental design methodology that uses random assignment to create treatment and control groups, you must explain your randomization procedures and identify which data you will collect from both groups.
You must also show that both your target population group and comparison or control group are large enough to account for other external factors that could skew the outcome target(s) such as random chance, general economic conditions, or participant selection.
Factor 3: Data Administration and Management (5 points). You must show what data the validator will use to assess achievement of outcome target(s) and how the data will be captured (e.g. tools and systems) over the timeline of the project. We will award points for this factor based on the extent to which you address the following:
You must describe and indicate the quality of the data sources and systems you will use to track and extract participant and comparison data.
For existing data sources and systems, you demonstrate how and when project partners will collect and access the data.
If publicly accessible data sources and systems are not established, you demonstrate how consensus between the project partners has been reached around creating new, viable data sources at a reasonable cost, built into the operating costs financed by the investor(s).
If new data sources and systems are created, you demonstrate the capacity of the project partners to properly maintain and manage the data source, and demonstrate how and when project partners will collect and access the data.
You must provide assurance that all data sources and systems will be available to the independent validator and Federal Pay for Success evaluator described in Section VI.B.4.
Factor 4: Dispute Resolution (2 points). We will award points for this factor based on the extent to which you provide reasonable and timely dispute resolution procedures if all parties do not agree with the determination by the validator regarding whether or not the outcome target(s) was met. These dispute resolution procedures must be consistent with the grant implementation timeline described in Section II.B. All parties must have agreed to this methodology and dispute resolution procedures as part of their partnership agreement referenced in Section III. C. 3. and in Section IV. B. Part III. (d).
Criterion 5: Financing Model and Payout Structure (25 points).
You must describe the financing model for the project, including the demonstrated cost savings and/or efficiency gains to the public sector resulting from the project, and the commitments from investors to cover the full operating costs over the life of the project. You must also describe the payment arrangement between you (the applicant) and the Federal government, including the payment point(s), the validated outcome target(s) that triggers the payment(s), and the payment amount(s). Should multiple payment points be proposed, each must be tied to achievement of validated outcome target(s), and not be based on interim outputs. In addition, each payment amount must be justified by cost savings. We will award points for this criterion based on the extent to which you address the following factors:
Factor 1: Cost of the Outcome and Resulting Savings (10 points). You must provide a financial analysis that illustrates that the partnership is capable of achieving the outcome target(s) at an overall cost (including any arrangements for return on the investor(s)’ investment) that represents savings to the government applicant. We will award points for this factor based on the extent to which you address the following:
You must provide a financial analysis of the public sector’s current cost of addressing the problem and target population identified in Criterion 2.
You must provide a financial analysis that illustrates the anticipated total operating cost of achieving the outcome target(s) over the lifetime of the project. The total operating cost should include: 1) the cost of the contract with the intermediary, including any overhead or other costs that the intermediary plans to charge to the project; 2) the cost of the intervention strategy; 3) the cost of the contract with the independent validator; 4) any proposed return on investment for the investor(s); and 5) the applicant’s own program management and oversight costs.
You must demonstrate that the return on investment amount included in your total delivery cost is reasonable compared to the estimated overall savings generated by achieving the outcome target(s) (i.e. appropriate outcome valuation).
You must calculate the savings that will accrue to the public sector based on successful achievement of each outcome target through the intervention. Savings must be shown in dollars. If public sector savings accrue at multiple levels of government, you must break out the savings by each level of government. However, you must be able to demonstrate some savings at your level of government. Savings may accrue through preventative programs that reduce the customer burden on existing services. For example, a project that reduces unemployment insurance duration may yield savings at the Federal and state level based on reduced unemployment insurance payments, as well as savings at the local or tribal level. These savings for the local government could be based on reduced need for workforce system services by the target population, increased efficiency due to a decreased intensity of service delivery needs, or a better allocation of resources. Conversely, another way to generate savings may entail providing services that are more expensive per participant, but that produce significantly better outcomes. For example, if it can be shown that an existing training for Limited English Proficiency (LEP) individuals generally results in them getting lower paying jobs, and the proposed Pay for Success intervention results in LEP participants getting significantly higher paying jobs, there would be a significant benefit to society, and to the target population even if all the savings were not confined to the workforce system. Pay for Success models are best viewed from this cross-sector assessment; accordingly, you should describe any cross-sector savings or additional non-monetized benefits.
You must demonstrate that the cost savings are realistic and logically result from achievement of each outcome.
Factor 2: Investment (5 points). You must clearly identify the source of up-front operating capital and provide evidence of its availability upon grant award. You must identify the independent investor(s) and demonstrate that the quantity and availability of funds is adequate to sustain the life of the project, according to the anticipated total delivery cost of the project. We will award points for this factor based on the extent to which you provide, as an attachment, a specific letter of commitment from the investor(s) to provide working capital adequate to sustain the life of the project. The commitment of funding may be contingent on grant award.
Factor 3: Payments (10 points). You must provide a project timeline that indicates the following: 1) when the intervention will begin; 2) when the intervention will conclude; 3) when the period of validating the outcome target(s) will take place; and 4) when you will request the drawdown of funds from the grant. We strongly prefer there be a single payment point that occurs at the end of the grant’s period of performance. We will award points for this factor based on the extent to which you address the following:
You must provide a workable project timeline that defines when the intervention will begin; when the intervention will conclude; when the validation period will take place; and when you will request the drawdown of funds from the grant that coincide with the requirements in Section II.B.
You describe each payment point between the applicant and the Federal government that the project partners have agreed to, including the corresponding outcome target that must be achieved, the evidence that will validate the achievement of the outcome, and a justification for the level of payment requested, based on anticipated cost savings associated with each outcome target. Payment points cannot be linked to outputs. For example, in an intervention strategy focused on decreasing the unemployment rate of a certain target group, it is not appropriate to list a payment point for “enrollment of participants” or “graduation of participants” as these are examples of outputs, not outcomes.
You show that the payment structure does not create high-stakes thresholds that would incentivize early project shutdowns once particular outcome target levels or payment points are met.
Applications for grants under this Solicitation will be accepted after the publication of this announcement and until the specified time on the closing date. Following the closing date, the Department will begin a two-phase evaluation process. During the first phase, a technical review panel will evaluate all eligible applications against the selection criteria. These criteria are based on the policy goals, priorities, and emphases set forth in this SGA. Up to 100 points may be awarded to an application, depending on the quality of the responses to the required information described in Section V.A.
Once the eligible applications have been scored, the second phase of evaluation will begin. The ranked scores will serve as the primary basis for the Grant Officer’s tentative selection of applications for funding, in conjunction with other factors such as geographic balance; alignment with the goals of the workforce investment system; the availability of funds; and which proposals are most advantageous to the government. The panel results are advisory in nature and not binding on the Grant Officer. The Grant Officer may consider any information that comes to his/her attention.
The Grant Officer will then make contact with the tentatively selected applicants, providing any questions regarding the application that need clarification, and arrange a teleconference with the applicant and the required partners. The purpose of the teleconference is to discuss the applicant's proposed model, and allow the applicant and required partners to ask any follow up questions related to DOL’s clarifying questions. Questions from DOL may concern key proposal areas such as, but not limited to, target population selection, outcome metrics and valuation, and the outcome validation methodology. The tentatively selected applicants will then have 30 days to respond in writing to the questions from DOL. Applicants will also be expected to reconfirm investor commitments during this time. Note that this process does not relieve applicants from initially submitting a fully formed partnership and application. If after the review of the applicant’s response, DOL determines that the application is not fundable in its current state, the Grant Officer may decline to fund the applicant and tentatively select another applicant on the same basis for selection specified above.
The final grant award document will be based on the applicant’s signature on the SF-424, including electronic signature via E-Authentication on http://www.grants.gov, which constitutes a binding offer by the applicant.
All award notifications will be posted on the ETA Homepage (http://www.doleta.gov). Applicants selected for award will be contacted directly before the grant’s execution. Non-selected applicants will be notified by mail or email and may request a written debriefing on the significant weaknesses of their proposal.
Selection of an organization as a grantee does not constitute approval of the grant application as submitted. Before the actual grant is awarded, ETA may enter into negotiations about such items as program components, staffing and funding levels, and administrative systems in place to support grant implementation. If the negotiations do not result in a mutually acceptable submission, the Grant Officer reserves the right to terminate the negotiations and decline to fund the application. DOL reserves the right to not fund any application related to this SGA.
1. Administrative Program Requirements
All grantees will be subject to all applicable Federal laws, regulations, and the applicable OMB Circulars. The grant(s) awarded under this SGA will be subject to the following administrative standards and provisions:
i. Non-Profit Organizations – OMB Circular A–122 (Cost Principles), relocated to 2 CFR Part 230, and 29 CFR Part 95 (Administrative Requirements)
ii. Educational Institutions – OMB Circular A–21 (Cost Principles), relocated to 2 CFR Part 220, and 29 CFR Part 95 (Administrative Requirements).
iii. State, Local and Indian Tribal Governments – OMB Circular A–87 (Cost Principles), relocated to 2 CFR Part 225, and 29 CFR Part 97 (Administrative Requirements).
iv. Profit Making Commercial Firms – Federal Acquisition Regulation (FAR) – 48 CFR part 31 (Cost Principles), and 29 CFR Part 95 (Administrative Requirements).
v. All Grant Recipients must comply with the applicable provisions of The Workforce Investment Act of 1998, Public Law No. 105-220, 112 Stat. 936 (codified as amended at 29 U.S.C. 2801 et seq.) and the applicable provisions of the regulations at 20 CFR 660 et seq.
vi. All entities must comply with 29 CFR Part 93 (New Restrictions on Lobbying), 29 CFR Part 94 (Government wide Requirements for Drug-Free Workplace (Financial Assistance)), 29 CFR 95.13 and Part 98 (Government wide Debarment and Suspension, and drug-free workplace requirements), and, where applicable, 29 CFR Part 96 (Audit Requirements for Grants, Contracts, and Other Agreements) and 29 CFR Part 99 (Audits of States, Local Governments and Non-Profit Organizations).
vii. 29 CFR Part 2, subpart D—Equal Treatment in Department of Labor Programs for Religious Organizations, Protection of Religious Liberty of Department of Labor Social Service Providers and Beneficiaries.
viii. 29 CFR Part 31—Nondiscrimination in Federally Assisted Programs of the Department of Labor—Effectuation of Title VI of the Civil Rights Act of 1964.
ix. 29 CFR Part 32—Nondiscrimination on the Basis of Handicap in Programs or Activities Receiving Federal Financial Assistance.
x. 29 CFR Part 35— Nondiscrimination on the Basis of Age in Programs or Activities Receiving Federal Financial Assistance from the Department of Labor.
xi. 29 CFR Part 36—Nondiscrimination on the Basis of Sex in Education Programs or Activities Receiving Federal Financial Assistance.
xii. 29 CFR Part 37 – Implementation of the Nondiscrimination and Equal Opportunity Provisions of the Workforce Investment Act of 1998.
xiii. 29 CFR Parts 29 and 30—Labor Standards for the Registration of Apprenticeship Programs, and Equal Employment Opportunity in Apprenticeship and Training, as applicable.
2. Other Legal Requirements:
i. Religious Activities
The Department notes that the Religious Freedom Restoration Act (RFRA), 42 U.S.C. Section 2000bb, applies to all Federal law and its implementation. If your organization is a faith-based organization that makes hiring decisions on the basis of religious belief, it may be entitled to receive Federal financial assistance under Title I of the Workforce Investment Act and maintain that hiring practice even though Section 188 of the Workforce Investment Act contains a general ban on religious discrimination in employment. If you are awarded a grant, you will be provided with information on how to request such an exemption.
ii. Lobbying or Fundraising the U.S. Government with Federal Funds
In accordance with Section 18 of the Lobbying Disclosure Act of 1995 (Public Law 104-65) (2 U.S.C. 1611), non-profit entities incorporated under Internal Revenue Service Code Section 501(c) (4) that engage in lobbying activities are not eligible to receive Federal funds and grants. No activity, including awareness-raising and advocacy activities, may include fundraising for, or lobbying of, U.S. Federal, State or Local Governments (see OMB Circular A-122).
iii. Transparency Act Requirements
Applicants must ensure that it has the necessary processes and systems in place to comply with the reporting requirements of the Federal Funding Accountability and Transparency Act of 2006 (Pub. Law 109-282, as amended by section 6202 of Pub. Law 110-252) (Transparency Act), as follows:
All applicants, except for those excepted from the Transparency Act under sub-paragraphs 1, 2, and 3 below, must ensure that they have the necessary processes and systems in place to comply with the subaward and executive total compensation reporting requirements of the Transparency Act, should they receive funding.
Upon award, applicants will receive detailed information on the reporting requirements of the Transparency Act, as described in 2 CFR Part 170, Appendix A, which can be found at the following website: http://edocket.access.gpo.gov/2010/pdf/2010-22705.pdf
The following types of awards are not subject to the Federal Funding Accountability and Transparency Act:
Federal awards to individuals who apply for or receive Federal awards as natural persons (i.e., unrelated to any business or non-profit organization he or she may own or operate in his or her name);
Federal awards to entities that had a gross income, from all sources, of less than $300,000 in the entities' previous tax year; and
Federal awards, if the required reporting would disclose classified information.
3. Other Administrative Standards and Provisions
DOL’s Uniform Administrative Requirement at 29 CFR 97.36(c)(1) requires government grantees to conduct procurement transactions in a manner providing for open competition in accordance with state, local, or tribal procurement rules. In the proposed implementation of the Pay for Success grant program, we require the grant applicant to identify in its application the intermediary and independent validator the applicant has selected to perform the proposed Pay for Success project. Therefore, the procurement of these entities must have been completed (or largely completed) before the applicant submits its application, and before selection or award. Nevertheless, you must conduct your procurements of the intermediary and validator according to state, local, or tribal procurement rules, which may include sole-source procurement. You should make use of all flexibility offered by state, local, or tribal law in determining the most appropriate procurement method for the circumstances. You must document your procurement procedures so the information will be available in the event of an audit, to satisfy your obligation to demonstrate that the procurement was completed according to the state local, or tribal procurement rules. Acceptance of this grant application does not constitute approval of the procurement process.
4. Special Program Requirements
Evaluation
We will require that the program or project partnership cooperate with a formal evaluation of the Pay for Success model. We will engage an independent evaluator (not the independent validator associated with the partnership) to assess the implementation, outcomes, and benefits of projects funded under this solicitation. By accepting the grant funds, you and your partners agree to make available to the evaluator(s) all data and documents that might be needed, including contracts and agreements and individual records on participants, employers and funding, according to guidelines that we will provide. Grantees and partners must also agree to provide access to program operating personnel and participants, as specified by the evaluator(s), including after the expiration date of the grant. Please note that, this evaluation may make use of program MIS data, local administrative data, financial data, and program progress reports. It is critical that you keep this information up to date and accurate for both performance measurement and evaluation purposes. Such cooperation may include:
allowing access to pertinent documents;
hosting site visits;
facilitating interviews with staff at all levels of involvement: grantee, intermediary, outcome validator, investors, service providers; and
providing other data, as required.
Changes to the statement of work.
Upon grant award, the proposal will become the grant’s statement of work. We strongly discourage any changes to the target population, outcome(s), intermediary, and validator, and any such changes would require prior Grant Officer approval. Changes to the intervention strategy and source of up-front project funding may be made without prior Grant Officer approval; however, you must notify the Grant Officer of these changes as they occur and provide appropriate documentation to update the statement of work.
Submission for Payment
The Department uses an electronic Payment Management System, which allows grantees to draw down funds from the grant. For Pay for Success, the grantee may draw down funds as costs are incurred for its own program management and oversight responsibilities under the grant, and the validator’s operating costs in accordance with the approved budget in Section IV. B. Part I. All funds related to the intervention (including all intermediary costs, service delivery and any return on investment) will only be released upon receipt of an invoice from the grantee accompanied by a report from the validator clearly documenting that the grant outcomes specified in the statement of work were met. The report must also provide detailed documentation for the Department to determine if the validation methodology outlined in the grant was followed. Release of funds will be made within 30 days upon our confirmation that the outcomes were achieved and the methodology was followed.
Grantees must agree to meet DOL reporting requirements. Quarterly financial reports, quarterly progress reports, and MIS data must be submitted by the grantee electronically. The grantee is required to provide the reports and documents listed below:
1. Quarterly Financial Reports
A Quarterly Financial Status Report (ETA 9130) is required until such time as all funds have been expended or the grant period has expired. Quarterly reports are due 45 days after the end of each calendar year quarter. Grantees must use DOL’s Online Electronic Reporting System and information and instructions will be provided to grantees.
2. Quarterly Performance Reports
The grantee must submit a quarterly progress report within 45 days after the end of each calendar year quarter. The report must include quarterly information regarding grant activities. It must summarize project activities, including the current stage of program implementation; progress towards achieving the outcome goals, including number of people served; significant milestones of the grantee, intermediary, investors, and validator; and related results of the project. It should thoroughly document the partnership activities and decision-making structure used to implement the Pay for Success model. The last quarterly progress report that grantees submit will serve as the grant’s Final Performance Report. This report should provide both quarterly and cumulative information on the grant activities. DOL will provide grantees with formal guidance about the data and other information that is required to be collected and reported on either a regular basis or special request basis. Grantees must agree to meet DOL reporting requirements.
3. Record Retention
Applicants must be prepared to follow Federal guidelines on record retention, which require grantees to maintain all records pertaining to grant activities for a period of not less than three years from the time of final grant close-out.
For further information about this SGA, please contact Linda Forman, Grants Management Specialist, Office of Grants Management, at (202) 693-3416. Applicants should e-mail all technical questions to [email protected] and must specifically reference SGA/DFA PY 11-13, and along with question(s), include a contact name, fax and phone number. This announcement is being made available on the ETA Web site at http://www.doleta.gov/grants and at http://www.grants.gov.
A. Web-Based Resources
We maintain a number of web-based resources that may be of assistance to applicants. For example, the Workforce Innovation Fund website (http://www.doleta.gov/workforce_innovation/) is ETA’s primary resource for information on the Fund and Pay for Success, including links to the White House Fact Sheet on Pay for Success (http://www.whitehouse.gov/omb/factsheet/paying-for-success) and the Nonprofit Finance Fund’s Pay for Success Learning Hub website (http://payforsuccess.org/), which contains suggested readings, tools, and other resources on PFS.
B. Workforce3One Resources
1. We encourage applicants to view the information gathered through the Pay for Success Pilot Project: National Listening Session Webinar held with Federal agency partners, state, local, and tribal government entities, local practitioners, and other industry stakeholders. The webinar can be found on Workforce3One.org at: https://www.workforce3one.org/view/5001203950384005936/info.
2. ETA encourages applicants to view the online tutorial, “Grant Applications 101: A Plain English Guide to ETA Competitive Grants,” available through Workforce3One at: http://www.workforce3one.org/page/grants_toolkit.
IX. Other Information
OMB Information Collection No 1225-0086, Expires November 30, 2012.
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 20 hours per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments about the burden estimated or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Labor, to the attention of the Departmental Clearance Officer, 200 Constitution Avenue NW, Room N1301, Washington, DC 20210. Comments may also be emailed to [email protected]. PLEASE DO NOT RETURN THE COMPLETED APPLICATION TO THIS ADDRESS. SEND IT TO THE SPONSORING AGENCY AS SPECIFIED IN THIS SOLICITATION.
This information is being collected for the purpose of awarding a grant. The information collected through this “Solicitation for Grant Applications” will be used by the Department of Labor to ensure that grants are awarded to the applicant best suited to perform the functions of the grant. Submission of this information is required in order for the applicant to be considered for award of this grant.
Signed _____________________, in Washington, D.C. by:
Eric D. Luetkenhaus
Grant Officer, Employment and Training Administration
Attachment A: Abstract Template
Abstract
Pay for Success Application
Applicant Name:
Applicant Type: State workforce agency, local government agency, Federally recognized tribal government
Applicant City/State:
Geographic Areas Served by the Intervention:
Project Name:
Funding Level Requested:
______________ program management costs
______________ validation costs
______________ implementation costs with intermediary
Names of the Entities Serving as the:
Intermediary
Investor(s)
Independent Validator
Description of the Problem and Target Population’s Characteristics:
Description of the Proposed Outcome(s) and Outcome Target(s):
Description of the Intervention Strategy:
Expected Cost Savings and/or Efficiency Gains from Intervention:
Public Contact Information: (email or website)
Attachment B: Glossary
Comparison Group – A group which does not receive the services or treatment being studied, to which outcomes are compared.
Control Group - A group from which the treatment or service is withheld by lottery.
Experimental design (random assignment) - A sample selection technique in which individuals are assigned to a treatment (e.g., to receive a specific service or be served in a specified manner) or to a control group by lottery. The two groups are compared to detect the difference (if any) made by the product and/or service. Such a design provides the most rigorous and widely accepted evidence of effectiveness.
http://www.socialresearchmethods.net/kb/random.htm
http://www.socialresearchmethods.net/kb/desexper.php
Logic model - A theory of how an idea, action, or service will improve outcomes. Logic models are usually presented as charts showing contextual factors, inputs, activities, outputs, and outcomes and their inter-relationships. More information about logic models can be found at: http://www.doleta.gov/workforce_innovation.
Outcome - The general impact on the target population that the program intends to achieve as a result of the intervention. Examples could include an increase in employment, employment retention, or long-term earnings for a target group.
Related terms:
Outcome measure – A formula that provides an assessment of a program’s impact and is applied to both target and comparison groups. The outcome measure is determined using relevant program data and has defined units of measurement by which the impact can be tracked. Your outcome measure could be expressed as an equation, clearly identifying the units of measurement used in the numerator and the denominator. Examples could include formulas for calculating average annual earnings, the employment retention rate and time spent drawing unemployment insurance benefits.
Outcome target – The specific level of achievement that the project intends to attain. Outcome targets can be a defined change of an outcome measure or a percentage improvement of that outcome measure when compared to a comparison or control population. Outcome targets must be defined relative to the comparison or control group. Examples could include a specified reduction in duration of unemployment insurance, a percentage increase in earnings, an increase in employment retention rate, or an increase in credential attainment rate.
Outputs - The in-program results a program produces that may be interim markers to gauge progress toward achieving the outcome target. Examples include numbers of certifications awarded, numbers of program graduates, and individuals placed in jobs.
A basic example illustrating the above terms:
For a proposed target population, the desired outcome may be an increase in employment. The corresponding outcome target could be defined as an increase in employment as defined by a 10% improvement in the employment retention rate, relative to a comparison group that did not receive the intervention. The level of achievement of the outcome target must be a direct result of the proposed intervention, which could include providing training for participants from the target population, The outcome measure is then the formula for calculating the employment retention rate, as calculated by dividing the number of participants who are employed in the third quarter after the end of the intervention, by the total number of participants in the intervention. The outputs in this example could be interim markers such as the number of participants completing the training class.
Quasi-experimental study - An evaluation design that attempts to approximate an experimental design and can support causal conclusions, without random assignment. Sophisticated analytic techniques are used to control for factors that might be associated with the outcome being analyzed. http://www.childtrends.org/Files/Child_Trends-2008_01_16_Evaluation6.pdf
Predominant types of quasi-experimental studies:
Carefully matched comparison group design - A type of quasi-experimental study that attempts to approximate an experimental study. More specifically, it is a design in which project participants are matched with non-participants based on key characteristics that are thought to be related to the outcome. These characteristics include, but are not limited to: 1) prior test scores and other measures of academic achievement (preferably, the same measures that the study will use to evaluate outcomes for the two groups); 2) demographic characteristics, such as age, disability, gender, English proficiency, ethnicity, poverty level, parents’ educational attainment, and single- or two-parent family background; 3) the time period in which the two groups are studied (e.g., the two groups are children entering kindergarten in the same year as opposed to sequential years); and 4) methods used to collect outcome data.
http://www.uwex.edu/ces/pdande/resources/pdf/agenda.pdf
Propensity Score Matching – A matched comparison group design which employs a predicted probability of group membership - e.g., treatment vs. control group - based on observed predictors, usually obtained from logistic regression to create a comparison group. This method is used when it is not possible to assign participants of an intervention randomly to a treatment and control group.
http://www.chrp.org/love/ASACleveland2003Propensity.pdf
http://erx.sagepub.com/content/29/6/530.abstract
Interrupted time series design - A type of quasi-experimental study in which the outcome of interest is measured before and after the treatment for program participants only. If the program had an impact, the outcomes after treatment will have a different level from those before treatment. That is, the series should show an “interruption” of the prior situation at the time when the program was implemented. Adding a comparison group and multi-point time series substantially increases the reliability of the findings.
http://www.gvglass.info/papers/tsx.pdf
Regression discontinuity design study - A quasi-experimental study design that assigns individuals to a treatment or comparison group based on a numerical rating or score of a variable unrelated to the treatment such as the rating of an application for funding. Another example would be assignment of eligible students, teachers, classrooms, or schools above a certain score (“cut score”) to the treatment group and assignment of those below the score to the comparison group.
http://www.amstat.org/sections/srms/proceedings/y2008/Files/301149.pdf
http://www.ssc.wisc.edu/~ctaber/742/rd.pdf
Treatment Group - A group to which the treatment or service is provided.
1 Preventative services are those services that stop an undesirable outcome – such as unemployment, underemployment, skill deficiencies, or re-offending – from happening. Examples include services that help students at risk of dropping out complete high school, provide assistance in persisting in post-secondary education or training, or give ex-offenders services and alternatives to avoid re-offending.
2 Vulnerable populations may include low-wage and less-skilled workers, as well as youth, ex-offenders, individuals with disabilities, and/or other economically and educationally disadvantaged persons.
3 Memorandums 07-16 and 06-19. GAO Report 08-536, Privacy: Alternatives Exist for Enhancing Protection of Personally Identifiable Information, May 2008, http://www.gao.gov/new.items/d08536.pdf.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | DEPARTMENT OF LABOR |
| Author | frugoli.pam |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |