Supporting_Statement_(Part_B)_01 15 13

B. STATISTICAL METHODS

1. Universe and Respondent Selection

No sampling or other statistical methods will be used to select the survey respondents or extrapolate or generalize the study results to a wider population. In the adult pilot, we will collect data on all 1,031 study participants recruited in the first year of the data. Administrative data from credit reports and checking account transactions will be collected for all those participants for which it is available: in the baseline year, credit reports were able to be matched to survey data for 884 participants and account data is available for 275 participants that opened checking accounts through the program. All study participants were surveyed at six months following enrollment, and the response rate to that survey was 64%. We expect the response rate to the 12 month survey to be lower and perhaps as low as 30%.

For the youth pilot, all eligible students in the interventions will be invited to participate in the study, and all study participants will be asked to respond to baseline and follow-up surveys. In the first year of the youth pilot, we recruited fewer students into the study than anticipated, only 530 to 556 students at a response rate of 40%. In order to increase the sample size, we are conducting the pilot again in the same site with a population of 760 students, and we estimate that we will be able to recruit an additional 300 students for a total sample of approximately 870. In addition, we are replicating the youth pilot in a second site with a larger student population, 2,500 4^th grade students, and hope to recruit 750 additional students.

2. Procedures for Collecting Information

The adult program research pilot is a partnership with the New York City Office of Financial Empowerment (OFE) and the University of Wisconsin-Madison Center for Financial Security (CFS) to provide, and test the impact of, access to a free, City employee checking account and one-on-one financial counseling provided by NYC’s Financial Empowerment Centers (FEC) to participants in the City’s Parks Opportunity Program (POP). Specifically, the research will test the outcomes of these interventions alone and in combination on (1) the beneficial use of a checking account by program participants and (2) the general financial well-being of participants. Data collection in the second year will build from what was collected in the first year of the program from the 1,031 POP participants who enrolled in the study. In the first year, data was collected using administrative data (transactional data from the checking accounts and credit reports) and through surveys administered at baseline and six months following enrollment. (. In this additional year of data collection, administrative and survey data will be collected for study participants at 12 months following enrollment. The surveys will use the same survey instrument used at six months and will be administered through the mail. In addition, we will conduct one-on-one interviews with up to 30 study participants to document their experience in pilot in an effort to capture “best practices” from the implementation. The interviews will take 45 minutes, and participants will be recruited over the phone.

The youth program research pilot will be conducted in two sites: Eau Claire, Wisconsin and Amarillo, Texas. The pilot in Eau Claire is a partnership with Royal Credit Union (RCU), the Eau Claire, Wisconsin School District, and the University of Wisconsin-Madison Center for Financial Security (CFS) to provide, and test the impact of, access to RCU’s credit union in school branches in some of the Eau Claire public schools and additional financial education activities, alone and in combination. Study participation will be completely voluntary, and those students who choose to participate will be surveyed in person on-site at school to collect baseline information in the form of a financial assessment evaluating financial knowledge, attitudes, and behaviors relevant to the participants’ age level. The same assessment would be administered towards the end of the school term to allow for the analysis of any changes associated with the interventions. Finally, administrative transaction data will be collected for students with youth savings accounts to assess both students’ level of engagement with their accounts and changes in banking behavior over the study period.

The pilot in Amarillo uses the same research design, with a few modification, and is designed to answer the same research questions about the impact of financial education alone and in combination with access to accounts in schools. In Amarillo, the pilot is a partnership with CFS, the Amarillo Independent School District, Happy State Bank (HSB), and implementation partners the Center for Public Policy Priorities and the Texas Council for Economic Education. The data collection procedures are the same as in Eau Claire: study participation is completely voluntary, and students will be surveyed on-site on financial knowledge, attitudes and behaviors. This same assessment will be administered again following the implementation of the intervention (financial education). Transaction data will be collected for students who open youth savings accounts to assess students’ level of engagement with their accounts and the in-school banking program.

Our findings in both pilots will provide valuable information about the effectiveness of interventions that combine financial education with access to financial products. However, the pilots test specific interventions on specific target populations. For this reason, the results of the study can provide insight into the potential effectiveness of similar interventions on similar populations, but they cannot be directly generalized beyond the study sample. In addition, while some treatments in these pilots are randomized, they are not randomized at an individual level (e.g. access to counseling in the adult pilot and access to financial education in the youth pilot). Access to the school credit union program in the youth pilot in Eau Claire is not random, but based on Royal Credit Union’s current programming in the Eau Claire schools. In Amarillo, we are randomizing access to the in-school banking programs in 15 of 36 schools, but in-school programs previously existed in three of the district’s elementary schools. These pilots are therefore not pure randomized control trials, and we will use statistical strategies to control for baseline differences between the treatment groups. We hope that these pilots lay the groundwork for more extensive research in this area.

Detailed information about the two pilots is provided in the remainder of this section.

Adult Pilot

Key Research Questions

We will examine how the effects of financial counseling in combination with access to a safe and affordable checking account differ from the effects of account access alone. In addition, we will examine the effects of intensity of account access by streamlining the account enrollment process for half the participants. We are interested in the effects of financial counseling and intensity of account access on:

1) The beneficial use of a checking account, and

2) The general financial well-being of participants.

Measuring account activity will show whether individuals are managing their accounts sustainably and using them in ways that promote financial stability. We will further measure whether participants’ overall financial health improves, and to what extent, as the result of the account access and financial counseling components of the program.

We hypothesize that participants provided greater account access will be more likely to take-up accounts. Bank accounts facilitate asset accumulation and access to other mainstream financial products, and reduce dependence on expensive alternative financial services (Barr, 2004; Barr and Sherraden, 2005; Belsky and Calder, 2004; Seidman and Tescher, 2003; Stegman and Faris, 2003). Therefore, we also expect that account take-up will improve participants’ financial well-being. Because financial counseling offers assistance and engagement with the checking accounts, we hypothesize that the provision of counseling in combination with account access will result in better use of the account and greater financial well-being than account access alone.

To date, the implementation of the pilot project has been highly successful. The bank account application rate was approximately 48% in a population that was 66% unbanked before beginning the program and 55% elected to have their paychecks deposited electronically. Through qualitative research involving participants and stakeholders, we will explore the factors that enabled and hindered participants’ bank account and direct deposit take-up by addressing the following research questions:

1) What aspects of the program prompted participants to sign up for a bank account?

2) What aspects of the program prompted participants to sign up direct deposit?

3) Why did some persons who did not have a bank account or direct deposit at the time of the commencement of the program not sign up for these products?

Answers to these questions will help us formulate best practices to share with asset building practitioners working with un- and under-banked populations.

PILOT RESEARCH DESIGN & IMPLEMENTATION

We are testing our hypothesis within one of the nation’s largest transitional employment programs, the New York City Parks Opportunity Program (POP), administered by the City’s Department of Parks and Recreation. POP offers six months of full-time employment coupled with job search counseling and other enrichment activities to adults receiving public assistance. POP participants represent a population that is underserved by mainstream financial services, and through this pilot they are introduced to mainstream transaction accounts and direct deposit. However, this population might need additional assistance to overcome barriers to banking as well as help with understanding new banking products and their benefits. We therefore test whether offering financial counseling improves participants’ bank product use and overall financial situation better than the product offers alone.

Between January and early May, 2012, OFE research specialists worked onsite at POP locations across four New York City boroughs to enroll POP participants in the study and offer free checking accounts with no overdraft fees with Popular Community Bank, formerly Banco Popular (see Appendix A for the account terms). During their POP orientation day, new POP enrollees were offered Popular accounts and direct deposit of their City paychecks to either a bank account or prepaid card (if the participant already has a card). A Popular staff member was present on site to facilitate the account opening. Participants filled out direct deposit forms while opening their accounts and received their account number that day. All participants also received basic information about direct deposit and account benefits and use, referred to as “light touch” financial education.

Study recruitment occurred about two weeks later during a POP professional development day, after POP participants were assigned to their work site. Participants were offered a $25 gift card as an incentive to participate in the study, provided informed consent, and completed a baseline survey (attached in Appendix C). On the same day, participants were offered another $25 incentive if they had enrolled in direct deposit. Study participants who did not open an account previously were offered a second opportunity to open an account and receive the incentive. OFE specialists provided “light touch” financial education at both the POP orientation day and the session two weeks later.

As part of a randomized field experiment, about one-half of the study participants were assigned to the opportunity to attend one-on-one financial counseling during their prescribed work hours. The counseling is provided by NYC’s Financial Empowerment Centers (FECs), which offer free financial counseling at over 20 locations city-wide. FEC counselors are trained to help clients on a range of financial needs, such as money management, budgeting, selecting safe and affordable financial products, and credit and debt management. OFE’s research specialists introduced financial counseling and facilitated FEC appointment scheduling for treatment groups at the same time participants were enrolled in the study. Participants offered counseling were excused from a full day of work to attend their initial session, and received normal compensation for the work day. Participants may also have returned for a second session on work time. Additional sessions may also have been compensated at the counselor’s recommendation.

The financial counseling treatment in this study was randomized by month and borough, rather than by individual. During the first half of the study period, POP sites in two of the four boroughs offered financial counseling to participants. Because treatment effects may be correlated with differences between sites, the counseling assignment was changed after approximately half the participants were recruited. At this point, the sites that did not previously offer counseling became the sites offering counseling. This permits an analysis of site specific effects, as well as treatment effects, in isolation.

It is also important to note that take-up of counseling is observed for each site-month. This will allow additional analyses of peer effects, as sites with higher take-up of counseling may show effects on clients offered counseling but who did not take it up due to peer-to-peer interactions.

The following chart shows how the treatment was assigned during the baseline year:

Table 1: Treatment Assignment
	Recruitment period
Borough	January-February 2012	March-May 2012
Bronx	FEC offer	No FEC offer
Brooklyn	FEC offer	No FEC offer
Manhattan	No FEC offer	FEC offer
Queens	No FEC offer	FEC offer

The study examines the effects of financial counseling in combination with access to direct deposit and a checking account using an intent-to-treat (ITT) research design. This design controls for selection bias by measuring the effects of access to services rather than the impacts of the services themselves. This is an important aspect of this study—the impact of “treatment on the treated” (i.e., the impact of financial counseling on participants who attended counseling) will also be analyzed, but by examining the effects of the offer alone, the study can overcome selection biases from unobservable characteristics that may affect both the likelihood of someone deciding to enroll in services and the outcomes of interest (for example motivation or conscientiousness). While variation in access to counseling is assigned, account access to transaction accounts is universal. Some participants choose not to open accounts, while some are not qualified for accounts based on prior history. This analysis will examine both account holders and non-account holders separately for this reason.

Ideally access to accounts would have also been randomized in this research design. During the project’s development, the feasibility of introducing variation in account access was explored using opt-in and opt-out approaches to direct deposit enrollment and account-opening. However, it was operationally cumbersome to alter the offer of direct deposit among the sites. This study is primarily focused on the take up of accounts, counseling and the interaction of the two. The unbiased effects of counseling (intent-to-treat) can be estimated using casual techniques common in field experiments. The effects of accounts cannot be estimated in such a way to overcome the biases inherent in which clients selected to open and use accounts.

New POP participants were recruited into the study from January to early May, 2012. OFE research specialists presented in front of approximately 1,300 POP participants between January and May, and 1,031 participants consented to participate in the research. Of these, 884 had credit records available and were able to be matched to the baseline survey data. Baseline data show that the study sample represents a financially vulnerable and largely unbanked population:

• 46% reported that they felt they had little or no control over their finances.

• Nearly 40% reported that they save regularly, and less than 28% reported having any savings, suggesting savings for this group is used for short-run needs and as a form of liquidity.

• 66% of the study participants reported having no bank account in the month before starting POP, and nearly half of those reported not having a bank account in the past.

• 87% reported having an Electronic Benefits Transfer (EBT) card for receipt of public cash assistance in the month before starting POP.

A total of 58% of the POP participants in the study enrolled in direct deposit and 52% applied for a new checking account with Popular Community Bank. However, approximately one-third of the applications to open Popular accounts were denied. If those denied accounts had also attempted to enroll in direct deposit, the rate of enrollment in that service will also be lower once final data is available.

Very preliminary data from clients in the first cohorts for which we have matched baseline and follow-up credit reports offer some insights into the potential findings. After 6 months, the 148 clients in this cohort showed a 20 point gain in credit scores if assigned to counseling, and a 30 point gain if assigned counseling and the client also had a counseling appointment scheduled. The mechanism driving this improvement appears to be the curing of delinquent accounts. This is an early snapshot of the data, but is suggestive of positive findings.

In the option year, we propose to continue to follow the sample we recruited during the baseline year. We would continue collecting data on the participants at six months post enrollment in the study and collect an additional wave of data 12 months post enrollment in the study. Financial counseling and bank account access likely had short-term effects on POP participants’ financial wellbeing. We aim to examine the effects of these powerful interventions on the adult study treatment and control groups a year later. Tracking data from Popular, credit reports and the FECs for up to 12 months could reveal more significant, longer term impacts. Extending data collection beyond the first year would also provide an opportunity to observe participants’ account use and financial well-being after completing the POP program. The account features negotiated by the City for its employee checking accounts continue when City employment ends, as long as customers complete a minimum of five transactions per month. Observing impacts once participants leave the POP may inform government efforts to design sustainable transactional accounts for low-income individuals.

Pilot

Qualitative Research

The successful implementation of the AFCO pilot program during the baseline period suggests that there are key components of its design that may add to “best practices” in the field. In order to better understand the program’s successful implementation, we propose additional qualitative research involving the program stakeholders to document the program’s implementation as well as explore stakeholders’ perspectives regarding what worked and why. This information is not captured by a quantitative analysis of the administrative and survey data alone. These “best practices” in an employer-based offer of direct deposit and on-site access to banking for a historically unbanked population could prove to be important additions to the literature on integrating financial wellness programming into government-sponsored programs. Ideally the report will enable successful replication of this program.

The qualitative research would analyze the design factors (e.g. peer-to-peer communication strategy, on-site banking offer) as well as the engagement of key stakeholders that contributed to the successful implementation of the pilot. The data collection plan would consist of interviews with nine or fewer key stakeholders, including NYC Department of Parks program administrators, Popular leadership and branch staff and FEC leadership and counselors. As the data collection will take place after the initial pilot implementation has concluded, interviews with key informants will also focus on questions about how key components of the pilot’s design will be integrated into ongoing programs. In Interviews with Popular leadership and branch staff, we would also delve into the reasons participants were denied accounts since approximately 25%-30% of applicants were unable to open accounts during the program, likely due to negative Chexsystems reports.

In addition to the stakeholder interviews, we propose to conduct one-one-one interviews with approximately 30 study participants in order to better understand the reasons for the program’s success as well as gain insight into how the program could be improved. CFED would contact potential participants over the phone. Potential participants will be informed of the goals of the study and they will be asked if they would be willing to participate in a 45 minute, in-person interview with a CFED staff member at the site where they received their initial POP orientation. Potential participants will also be told that they will receive $50 financial incentive for their participation. Potential participants will be recruited from the following groups of study participants: persons who did not try to open a Popular account, persons who tried to open a Popular account but were unable to do so, and persons who successfully opened a Popular account. At the actual interview, participants will be required to sign a consent form in order to participate in the interview (see Appendix D for participant interview protocols and consent forms).

Measurement

In order to answer the key research questions as stated above, we are tracking outcomes from a survey (at intake, six months and twelve months for clients who continue to participate), credit reports (at intake, six months and twelve months when clients have a record), and bank account data (for those who have Popular accounts):

Outcomes associated with financial capability or well-being

• Total revolving debt from credit reports

• Delinquent accounts from credit reports

• Percent use of available revolving credit from credit reports

• Self-reported savings-related behaviors from surveys

• Use of check cashing, pay day and pawn shop services from surveys

• Self-assessed financial knowledge from surveys

• Financial attitudes from surveys

Outcomes associated with Popular account use

• Number of monthly transactions

• Account longevity

• Use of direct deposit (if available after program completion)

• Average monthly balance

• Frequency of non-sufficient funds fees (e.g. for bounced checks)

We expect that individuals receiving counseling will be more likely to make regular deposits, use the account for online bill payments, maintain the account for a longer period, and pay fewer fees (compared clients not offered counseling). We hypothesize counseling reduces outstanding revolving debt, incidence of delinquencies, and other measures of financial capability. Individuals receiving counseling would be more likely to have greater financial knowledge and exhibit financial attitudes that promote and reflect greater financial security.

Data Collection

The outcomes of interest will be measured using a combination of administrative and self-reported data. Self-reported data will be collected using surveys. A baseline survey assessing current banking status in addition to financial situation, behaviors, and attitudes was administered onsite during study recruitment. A follow-up survey assessing account use, financial counseling, financial health in addition to financial knowledge, behavior, and attitudes will be administered at six and twelve months following enrollment. Most of the survey questions have been adapted from previously-fielded questions used in the literature and prior research but some survey questions also address areas not yet adequately explored in previous research. The survey has been developed and will be implemented with the assistance of the University of Wisconsin Survey Center.

For the six month follow-up survey (see Appendix C), participants are asked to complete the survey onsite at exit seminars that POP participants are scheduled to attend six months after they begin the program. Participants who have left the program early, or who are otherwise not available to take the survey onsite will be surveyed by mail. We expect some participants to leave the program early as they find full-time employment elsewhere, resign, are terminated, or are assigned to other programs. Participants receive a $10 Metro Card as an incentive. On-site administration of the six-month survey took place between June and October and administration through the mail will continue through December. During the first two months of the survey administration, the response rate to the on-site survey has been high; nearly 100% of those who attend the exit seminars complete the survey. However, the attendance rate at the seminars is lower than expected with approximately 50% of those scheduled attending the seminar. Those not in attendance receive the survey in the mail. Up to three reminder mailings to return the survey are sent to participants. This strategy reduces response bias resulting from differences between participants who return the mail survey and those who do not. To date, the response rate to the mailed survey is approximately 30%. The twelve-month survey will be administered entirely through the mail, and participants will receive $10 cash as an incentive for completing he survey and a $10 Metro Card after completed surveys are returned. We anticipate that the response rate for the mailed twelve-month survey will be 30% or lower.

Administrative data will be collected from a variety of sources at six and 12 months following study enrollment. The Parks Department will provide demographic and employment data. Popular will share account data for participants who signed the bank’s data release form. The FECs will provide data regarding study participants’ financial counseling attendance. Data regarding debt levels, delinquencies, credit utilization rate, and credit establishment will be collected by conducting “soft pulls” of study participant credit reports at intake and then at six and 12 months. We expect to continue to match credit records to survey data for the 884 participants that had matching data at baseline. The following table outlines the adult pilot data collection and sources:

Data Source	Account Use (outcomes)	Financial Well-Being (outcomes)	Additional Data Points (controls)
Bank data up to 12 months	Number of monthly transactions Account longevity Use of direct deposit Average monthly balance Number of returned (bounced) checks
Credit reports baseline, 6 months, 12 months		Debt levels Delinquencies Percent use of available revolving credit Take up of other affordable, mainstream financial products or services (e.g. establishing credit)
Follow-up survey approximately 6 and 12months		Financial knowledge (knowledge assessments) Financial attitudes (e.g. future orientation, perceived financial well- being, sense of financial control and satisfaction) Financial behavior (e.g. use of alternative financial services, saving behavior)
Intake survey baseline		Financial attitudes Financial behavior Baseline banking status Banking history	Demographics
POP administrative data up to 6-months			Demographics Employment information Income information Benefits information POP start and end date POP start and end date
Financial Empowerment Center administrative data up to 24 months			Number of counseling sessions Service plans Milestones achieved

Data Analysis

As discussed earlier, the data will be analyzed using an intent-to-treat research design. Specifically, we will measure the effects of the counseling and account offers rather than take-up. The analysis will involve several comparisons. First, we will compare outcomes for clients assigned to the counseling offer versus those who were not assigned the counseling offer. In order to account for differences between these two groups other than the counseling offer, we will use OLS regression to control for age, race and family structure as well as self-reports of financial distress, financial knowledge and banking status. The analysis will explore changes in the following outcome variables: credit score, debt balance, accounts in collection, perceived financial control and banking activity. Next we will perform the same estimates conditional on both being assigned to counseling (intent to treat) as well as signing up for a counseling appointment. This second set of analyses will allow us to assess counseling effects rather than simply the offer of counseling. Finally we will re-run both these estimates using an interaction between counseling) and opening an account with Popular. This will allow us to assess the effect of opening a bank account alone and in combination with financial counseling. We will then explore a number of alternative analyses involving the inclusion of sample restrictions to examine heterogeneity in effects, as well as the inclusion of POP site and month fixed effects. This additional analysis will allow us to assess the degree to which program outcomes varied across sites and over time.

Sample Size and Detectable Effects

Baseline recruitment is complete at 1,031. . Assuming 250 in each comparison group, the following are examples of approximate minimum detectable differences between the groups:

• $120 difference in savings

• 7% difference in the proportion of individuals who have bounced at least one check in the last 6 months

• 10 percentage point difference in amount of available credit

• $1,800 difference in revolving debt

• $2,600 difference in amount past due

• Unit difference in the number of bills in collections

• Third of a deviation difference in self-reported financial knowledge and actual financial knowledge (based on a sample of 175 since we expect a survey response rate of 70 percent)

• 16% difference in proportion of individuals who feel they have little or very little control over their financial situation (based on a sample of 175 since we expect a survey response rate of 70 percent)

These effects were estimated using prior studies with similar populations (2-tailed test, power=0.8, alpha=0.05). .

Research Ethics

Researchers will follow strict ethical standards for human subjects research while implementing these pilots. All consent, data collection, and confidentiality protocols have been approved and will continue to be monitored by the UW-Madison Institutional Review Board (IRB). CFED will contract with Chesapeake IRB for the review of the qualitative research protocol and consent forms.

Research Timeline

October 2012

• Interim report on baseline year (October 26, 2012)

• Draft and finalize qualitative protocols

• Finalize IRB protocols for authorization for human subjects research

October – December 2012

• Continue administering six-month follow-up survey

• Continue pulling 6-month credit reports and collecting data on bank account use

January-February 2013

• Analyze data

• Draft report on implementation of the pilot program for the field

January -March 2013

• Conduct key informant interviews

• Con with POP participants

February – June 2013

• Field 12-month follow-up survey

• Pull 12-month credit reports and collect data on bank account use

March 2013

• Draft interim report on six-month data

July-August 2013

• Analyze data

September 2013

• Draft final report (outline due July 29, final paper due September 27)

Youth Pilot

Key Research Questions

This study will examine whether elementary school students (specifically 4^th and 5^th graders) learn more from personal finance instruction when they have the opportunity to open and use a savings account through a credit union or bank branch in their school. Students are randomly assigned to education by classroom or by teacher across schools with and without credit union or bank branches.

The research design will provide four comparison groups:

	Financial education	No financial education
Credit union or bank in school	Credit union or bank in school + Financial education	Credit union or bank in school + No financial education
No credit union or bank in school	No credit union or bank in school + Financial education	No credit union or bank in school + No financial education

This design will allow us to address the following research questions:

• Do students gain financial knowledge and understanding by participating in a financial education curriculum?

• Do student learn more when they also attend a school with a credit union branch?

• Are students more likely to open a bank account and/or make deposits if they are participating in a financial education curriculum?

We expect that financial education will increase financial knowledge and understanding. We also expect that students in schools with financial institutions may learn more from financial education lessons than students receiving education alone. These students have a greater opportunity to apply their learning by making regular deposits into a savings account. We also expect that in schools where a credit union or bank program has a significant presence, even students without accounts will benefit. The availability and visibility of banking services might increase the perceived relevance of the financial education curriculum and improve learning among students without accounts. Finally, we hypothesize that students with a credit union branch in their school will be more likely to participate in the credit union or bank program (by opening accounts and/or making deposits) when they learn about financial topics in class.

Pilot Research Design & Implementation

In the option year, we propose to implement the study in two sites: a continuation of the pilot in Eau Claire, Wisconsin, and a replication of the pilot with a few additional features in Amarillo, Texas.

Continuation of Pilot in Eau Claire

In the baseline year of the project, the study was implemented in the 4^th and 5^th grade classrooms in Eau Claire, Wisconsin. In the option year, we will continue the pilot with the incoming 4^th and 5^th grade class during the spring 2013 semester.

Credit Union School Banking Program

Royal Credit Union (RCU) operates student branches in six of the district’s 13 elementary schools as part of its School $ense program. RCU has 21 offices (18 in Wisconsin and three in Minnesota), 120,000 members, and over $1 billion in assets, making it one of the largest credit unions in the area. It is the top home lender in Wisconsin’s Eau Claire and Chippewa counties and offers a range of services, from health savings accounts and business services to credit counseling.

RCU’s School $ense program began in 1993 at the request of an elementary school principal who had received an inquiry from a parent about in-school savings programs. The program has expanded to 14 elementary schools, three middle schools, and three high schools in the region.

Modeled after the Save for America program, School $ense allows children and a parent or guardian (joint member) to open a savings account with RCU, which furnishes the required $5 initial deposit (account terms are listed in Appendix B). Joint members must apply online or in-person at RCU offices to open the account, but from then on children can make deposits and withdrawals at school. RCU also hosts Kids Club – a youth savings program similar in structure to School $ense – out of its offices.

School branches are operated once or twice a week and are set up as tables outside the cafeteria. Teachers collect deposits from their students and deliver them to RCU staff. Deposits can be of any amount, and elementary school students can withdraw up to $20 at a time with a joint member’s signature. Students are encouraged to identify a savings goal and to track progress toward that goal.

Student tellers (4^th and 5^th graders at the elementary level, and all grades in middle and high schools) process the deposits with RCU staff at lunchtime. These tellers are hired through a true-to-life process; students fill out an application and interview for a position. Students walking to lunch can discuss their accounts with the tellers. They can also redeem small prizes, earned every fourth deposit, at this time. Often, students who do not have accounts inquire about the program and are advised to discuss opening an account with their parents.

RCU has demonstrated a strong commitment to working with local schools in western Wisconsin, and has a well-established relationship with the Eau Claire Area School District. During the 2010 to 2011 school year, 2,100 student members made nearly 16,000 deposits. At the six Eau Claire elementary schools with an RCU branch, students made about 5,100 transactions and deposited over $65,000 during the year. For every 500 deposits made at a school site, RCU donates $250 to the school. These donations amounted to almost $6,200 during the 2010 to 2011 school year.

Baseline Pilot Design and Implementation

During the Spring 2012 semester, every 4^th and 5^th grade student in the school district participated in a 5-lesson classroom financial education program called Money F-I-T. Half the district’s classrooms were randomly assigned to participate in the program during the study period, and the remaining classrooms participated immediately after the study period.¹ In designing the Money F-I-T lessons, we adapted the Financial Fitness for Life curriculum developed for students in grades 3-5 drawing primarily from the lessons addressing savings, financial decision-making, and money management. These are topics relevant to students’ use of saving accounts. We created a curriculum that could be delivered in five approximately 45-minute lessons. Overall, teacher feedback on the lessons was quite positive. The teachers reported that the material was appropriate for the students’ learning level and held their interest. Importantly, most teachers felt that students had not previously studied the material and that the lessons were a valuable addition to the curriculum.

The lessons were delivered by the students’ regular classroom teachers or RCU educators. Teachers who choose to deliver the material themselves attended a 3-hour training session during a professional development in-service day, and were compensated $100 for time spent preparing for lessons outside the normal work day. Teachers could alternatively elect to have the material delivered to their classrooms by RCU educators, who attended the same training session. RCU staff members have previous experience implementing financial education programs for students, either in the classroom or as part of training sessions for student tellers. The training session played an important role in preparing teachers and RCU educators to teach the material to students. Because we would expect teachers and credit union staff members to have different levels of experience with both youth education and personal finance, the training session was intended to standardize the financial education treatment.

During the study period, students in schools with RCU branches had weekly opportunities to access their accounts (e.g., every Wednesday). This provided multiple opportunities for students to be exposed to accounts or make deposits and withdrawals over the course of the spring term.

The following chart illustrates the research design as implemented in Eau Claire:

During the initial year of the study we recruited 4th and 5th grade study participants across the 13 elementary schools in Eau Claire. Each school has two or three classes per grade, with approximately 1,500 4th and 5th graders total in the district. We originally estimated that about 50 percent of students would enroll in the study. To enroll students in the study, we needed to collect student assent as well as parental consent to use the student assessment data and to release account data from RCU. Numerous messaging strategies were used to inform parents about study recruitment, including brochures, fliers, and letters sent home with students, school newsletter articles, emails sent directly to parents, and local media coverage. However, consent in the baseline year was lower than expected, with only 569 students in the final sample.

As the sample recruited in the baseline year was smaller than expected, we propose to repeat the pilot in Eau Claire for the income 4th grade class during the Spring 2013 semester. Continuing the pilot with the same research design will allow us to create a larger sample, improving the quality of the analysis. Approximately 760 students are currently enrolled in the district’s current 4th grade. Assuming households consent to participate in the study at the current consent rate of about 40 percent, we estimate that we will be able to recruit 300 additional students into the study, for a total sample of approximately 870 students. The larger sample would improve our ability to identify measurable impacts of the program and control for confounding factors (the initial goal was to have a sample close to 900 in order to meet our estimated minimum in order to measure detectable effects with reasonable statistical power). Because 4th grade teachers and Royal Credit Union staff have already received training on the financial education curriculum and school staff members have demonstrated their capacity to facilitate data collection (via student assessments), we will be able to implement the same research design with minor changes. Our major challenge in the youth pilot was parental consent, so in an effort to maximize response rates and reduce the time burden on parents, we have made the decision not to collect information through a parent survey. Data from the parent survey in the first year was used as control variables in our analytical model. However, we also ran the models without the control variables to determine the degree to which the results were being driven by the presence of these control variables, and the results without the controls were qualitatively similar.

We also want to use the opportunity of continuing the pilot for an additional year in Eau Claire to test the long-term effects of the financial education and account access on students’ financial knowledge and behavior. We will test this by administering an additional follow-up survey to the 760 current 5th graders approximately a year after the students participated in the pilot last year as 4th graders.

Pilot Replication in Amarillo, Texas

We plan to replicate the baseline youth pilot with the 4th grade class in Amarillo, Texas in order to test the program’s efficacy in a more ethnically and economically diverse context. While the implementation of the study in Eau Claire has been very successful, the policy relevance of the findings are somewhat limited by the fact that Eau Claire is a small, relatively homogenous school district. In the Amarillo Independent School District (ISD) elementary schools, 66% of students are economically disadvantaged as compared to 45% in Eau Claire. Less than half of Amarillo’s elementary students are white, non-Hispanic and 37% are Hispanic and 10% are African American. In Eau Claire, 80% of elementary students are white, non-Hispanic and only 6% are Hispanic or African American. Across 36 elementary schools in Amarillo ISD, there are approximately 2,500 students.

Conducting the pilot in Texas also provides us with the opportunity to research newly mandated financial literacy curriculum standards. In 2011, Texas passed a law requiring that personal financial literacy (PFL) training be incorporated into the K-8 math curriculum. The Texas State Board of Education approved the new PFL mathematics standards for Texas Essential Knowledge and Skills (TEKS) in April of 2012. While teachers are not required to begin teaching the new PFL math requirements until the fall of 2014 when new text books will be adopted, some districts are opting to train their teachers on these standards early. The personal financial literacy TEKS focus on teaching students to make thoughtful, well-informed decisions about important aspects of personal finance, including earning income, spending, saving, borrowing, and managing money. The financial education delivered to students in the replication of the AFCO pilot in Texas will incorporate the majority of Texas’ new PFL curriculum standards. Given this, our research findings would have a large and eager audience in Texas as well as a broader audience among other states considering incorporating personal financial literacy into their K-12 curriculum and connecting in-school banking programs to their PFL curriculum.

We have partnered with OpportunityTexas, a project of the Center for Public Policy Priorities (CPPP), a leading advocate for increased economic opportunities for low- and moderate-income Texans, to implement the AFCO youth pilot in Amarillo. Opportunity Texas is collaborating with the Texas Council on Economic Education (TCEE), an organization that provides teacher staff development trainings throughout the state of Texas in economics and financial literacy, to develop the financial education lessons and to provide the trainings to the teachers.

While we want to maintain the same basic elements of the research design from the pilot in Eau Claire in the first year, the project in Amarillo allows us to alter the research design to test additional elements, including randomizing access to the in-school banking program.

In-School Banking Program

The financial institution partner in Amarillo is Happy State Bank (HSB), a bank with over $1.6 billion in assets with 30 branches in 20 communities across the Texas panhandle. HSB has operates in-school bank branches, called Kid’s Banks, in 16 elementary schools, three of which are in Amarillo ISD. The Kid’s Banks operate much the same as RCU School $ense program. Children open a savings account jointly with a parent or guardian either through forms sent home from school with children, online or in person at a HSB branch. After the account has been opened, the child can begin making deposits at the branch at school. HSB recruits 5^th grade students through an application process to operate the in-school branches. The days that the Kids Banks are open in the schools, referred to as banking days, are determined by the principal, but occur approximately every two weeks.

As HSB currently has a presence in only three of the Amarillo ISD elementary schools, HSB has agreed to expand the program to an additional, randomly selected 15 schools during the current school year so that approximately half (1200 students) of the 4^th grade students will have access to accounts. While only the 4^th grade students will participate in the study, the Kid’s Banks will be available for all students in the school. Because HSB is expanding the program dramatically, the one change they are making to their program in the 15 new schools is that they will not be able to hire the 5^th graders to operate the bank branches. Staff does not believe they will have time to complete the application and interview process with the students. HSB has also agreed to schedule banking days at least once per week during the study period so that students have multiple opportunities to open accounts and make deposits.

While starting new Kid’s Bank program in randomly selected schools makes our research design more robust, it does make the implementation of the pilot more challenging. Rather than studying the use of existing bank accounts, we will need to encourage a number of students and their parents to open new savings accounts. Children and parents are first informed about the opportunity to open accounts through letters and account applications that are sent home with the students from school. To incentivize account opening, we will offer a $25 deposit to half of the 4th grade students in the schools with Kid’s Banks, approximately 600 students. The students who are offered the incentive will be randomly chosen in order further randomize access to accounts. The Amarillo ISD superintendent has requested that the random selection happen in the classroom with the involvement of the students so that the process is transparent. The teachers will run the randomization like a lottery in the classrooms at the same time that students are given information about HSB accounts to take home to their parents. Based on prior efforts to open accounts in school settings, we anticipate an account take up rate of approximately 20% or lower.

Financial Education Lessons

The administration of the treatment of financial education will be implemented similarly to the pilot in Eau Claire with some minor alterations. The lessons will be 45 minutes long and will be taught once per week by regular classroom teachers in the 4^th grade math classes. In order to cover the new personal finance TEKS in Texas, the number of lessons has been expanded from five to six. The additional lesson was developed by TCEE but adapted from the Financial Fitness for Life curriculum. TCEE will be training the teachers on the lessons in a day-long training. To monitor that teachers are teaching the lessons during the scheduled times and to gather input from the teachers on the curriculum, teachers will be asked to complete an online form providing feedback after each lesson. Teachers will be compensated $15 for each form they submit.

In Eau Claire, the financial education was randomized across the schools with and without credit union branches by classroom. In Amarillo, some of the schools have departmental math teachers, who teach more than one section of students and in some schools, up to three or four sections. So that each teacher is teaching the same lessons at the same time to all of their classes, the financial education in Amarillo will be randomized by teacher rather than by classroom. There are 85 4^th grade math teachers in Amarillo ISD, and 42 have been randomly selected to teach the lessons during the study period. The remaining 43 will teach the lessons following the study period, in late April and May 2013.

In order to enroll students in the study and use the student assessment data, we will have to collect parent consent as well as student assent in addition to consent to release account data. As mentioned previously, the rate of parents providing consent in Eau Claire was lower than expected at approximately 40%. To try to improve response rates, we are also not going to collect data through a parent survey. We believe that the data from the parent survey is not as valuable as a potentially higher consent rate. In further efforts to streamline the documents sent home to parents, we have made some modifications to the consent required for parents to enroll their students in the study (see attached consent forms). We have combined the consent forms for enrollment in the study and releasing the transactional data on accounts. Student asset will no longer be required to be written assent, but teachers will get verbal assent prior to the administration of the student surveys. Finally, consent forms will also be sent home with students from school as well as administered in the mail to those parents who do not respond to the first request. Even with these simpler procedures, we anticipate a lower response rate from parents in Amarillo as they are less familiar with the University of Wisconsin and have less experience with participating in research. Therefore, we hope to get a response rate of approximately 30%, which would result in a sample of approximately 750 students.

Measurement

The impacts of treatment (financial education) will be measured using a simple assessment test provided at the start and end of the study period at both the Amarillo and Eau Claire sites. The test will assess financial knowledge, attitudes, and behaviors appropriate to the participants’ age level. The FFFL curriculum includes nationally benchmarked assessment tests, which will be used for the knowledge portion of the assessments, with a few modifications to the assessment in Amarillo to test concepts in the Texas curriculum standards. With approximately 200 participants in each comparison group, we should be able to detect a 5% marginal change in scores on a 25-item financial knowledge assessment with two potential answers per item using a 1-tail test. We are not able to estimate minimum detectable effects on savings amounts precisely, but assuming an average account balance increase of $30, differences of $12-14 in the balance increase between comparison groups will be statistically significant We will not collect demographic data through a parent survey in an attempt to streamline forms parents have to complete to enroll their child in the study and maximize parental consent. In the absence of data on the parents, we will use data on demographics, school attendance and grades at the school level to serve as additional controls. We will also collect banking data reflecting the intensity of students’ engagements with their savings accounts, if they have them. Proposed data to be obtained from for each study participant include:

• Date account was opened

• Number of deposits over the study period

• Dollar amount of deposits over the study period

• (for RCU) Prizes redeemed by each student for deposits over the study period

The following table outlines data collection and sources for the youth pilot:

Data source	Outcomes	Additional data points (controls)
Student assessments Pre-/post-intervention	Financial knowledge Financial attitudes
School District baseline		School-level Demographics
RCU data Duration of study period	Whether the student has an RCU account Date account was opened Number of deposits and withdrawals over the study period Dollar amount of deposits over the study period Prizes redeemed by each student for deposits over the study period	Whether the student is an RCU teller in their school2
Bank data (replication of pilot in Amarillo, TX) Duration of study period	Whether the student has or opened an account Date account was opened Number of deposits and withdrawals over the study period Dollar amount of deposits over the study period	Whether the student was offered an account opening incentive

Data Analysis

A difference-in-differences approach will be used to compare how student outcomes among the four sample groups differ, demonstrating whether school credit union or bank branches and the financial education program are associated with differences in financial knowledge and behavior, individually and in combination. Subgroup analysis may allow us to examine whether impacts differ across subpopulations if the sample size is sufficient.

We will likely explore multiple estimation strategies, including ordinary least squares (using classroom fixed effects and clustering standard errors at the school level) and potentially hierarchical linear modeling. We will also control for baseline school and child characteristics. In Amarillo, the effects of the offer of account incentives will also be modeled as fixed effects.

Research Ethics

Researchers will follow strict ethical standards for human subjects research while implementing these pilots. All consent, data collection, and confidentiality protocols have been reviewed and will continue to be monitored by the Institutional Review Board (IRB) which oversees human subjects research at the University of Wisconsin-Madison.

Research Timeline

Continuation of Eau Claire Pilot

September – December 2012

• Obtain approval for revisions to research design and implementation plan

• Confirm implementation plan with school administrators

• Interim report on baseline year (October 26, 2012)

February 2013

• Administer baseline student assessment (2 weeks before financial education starts)

February –May 2013

• Obtain parental consent for participation and collect baseline data from parent survey

March April 2013

• Administer financial education program to treatment group (one lesson per week for 5 weeks)

• Students with access to accounts have weekly opportunities to make deposits

April 2013

• Administer follow-up assessment (one week after financial education lessons are completed)

May – June 2013

• Administer financial education program to control group

June - August 2013

• Analyze data from both pilot continuation and baseline year

September 2013

• Draft Final report on Eau Claire pilot (outline due July 15, final paper due September 15)

Amarillo, Texas Pilot

September 2012

• Confirm selection of schools sites and financial institution and obtain official school administrator authorization for research

September – November 2012

• Confirm financial education curriculum

• Develop survey instruments and accompanying documents

• Develop teacher and parent communications materials

• Confirm implementation plan with school administrators and financial institution

October – November 2012

• Finalize IRB protocol for authorization for human subjects research

November 2012

• Train teachers delivering the financial education lessons

January - March 2013

• Implement in-school bank branch program in randomly selected schools

• Administer baseline student assessment

• Students with access to accounts have opportunities to make regular deposits

• Administer financial education program to treatment group

February - May 2013

• Obtain parental consent for participation and collect baseline data from parent survey

March 2013

• Administer follow-up assessment

April-May 2013

• Administer financial education program to control group

June-September 2013

• Analyze data and draft report

September 2013

• Draft Final report (outline due July 29, final paper due September 27)

3. Methods to Maximize Response

In order to achieve our desired sample size, we need to recruit participants effectively from the pool of potential participants and maximize participants’ likelihood of responding to surveys.

Adult pilot

We will improve our response rates by offering incentives to complete the 12-month follow survey. As study participants are no longer in the POP program, there are no further opportunities to administer the survey in person. Therefore all surveys will be administered through the mail with a $10 pre-incentive to complete the survey. Respondents will receive a $10 Metro Card as a token of appreciation for responding to the survey. The UW Survey Center will also use tested techniques to maximize the response rate. This includes multiple waves of survey mailings and postcard reminders.

Youth pilot

Parents of potential study subjects will receive consent forms in the mail and forms will also be sent home from school with children. As we are no longer asking parents to complete a survey, we are not providing an incentive, but we have streamlined the forms that we have asked parents to complete in order to enroll their children in the study in an effort to improve response rates. In addition, we will improve response rates by informing parents of the study and building interest in advance of the mailing; parents will learn about the study through school newsletter articles and letters sent home with students. In both Eau Claire and Amarillo, we have used local media to also build awareness and excitement for the research project.. All students will complete the pre/post assessments (though the data will only be used when parents have consented). Teachers will be provided with detailed instructions for administering the assessments to improve responses.

4. Testing of Procedures

Adult pilot

Survey questions used in the 12-month follow-up survey are the same as the approved six-month follow-up survey, and are similar to the questions used in the principal investigators’ study evaluating the impacts of financial counseling at NYC Financial Empowerment Centers (so that results will be comparable). These questions were taken/adapted from the literature and/or developed with the assistance of survey experts at the University of Wisconsin Survey Center (UWSC). The questions were tested with a sample of POP participants before fielding.

Youth pilot

The youth pilot questions were mainly taken/adapted from the literature. The financial knowledge questions were developed by the Council on Economic Education (CEE) and correlate with the Financial Fitness for Life curriculum used in the educational intervention. These test questions have been nationally tested and benchmarked by CEE. In the Amarillo pilot, questions were also developed by the Texas Council for Economic Education (TCEE), a statewide affiliate of the national CEE, to meet the standards of the recently approved Texas K-8 financial literacy curriculum standards.

We previously provided supplemental document noting the sources of survey questions, and which were developed specifically for this study. We can provide that documentation again if necessary.

5. Contacts for Statistical Aspects and Data Collection

Dr. J Michael Collins

Faculty Director, Center for Financial Security at the University of Wisconsin-Madison

608-262-0396

References

Baker, C., & Dylla, D. (2007). Analyzing the relationship between account ownership and financial education. Asset Building Program, New America Foundation.

Barr, M. S. (2004). Banking the poor: Policies to bring low-income Americans into financial mainstream. In Brookings Institution, Research Brief.

Barr, M. S., & Sherraden, M. W. (2005). Institutions and inclusion in saving policy. In N. Retsinas & E. Belsky (Eds.), Building assets, building wealth: Creating wealth in low-income communities. Washington, DC: Brookings Institution Press.

Belsky, E., & Calder, A. (2004). Credit matters: Low-income asset building challenges in a dual financial service system (Joint Center for Housing Studies Working Paper Series, BABC 04-1). Cambridge, MA: Joint Center for Housing Studies, Harvard University.

Elliott, W., & Beverly, S. (2011). The role of savings and wealth in reducing ‘wilt’ between expectations and college attendance. Journal of Children and Poverty, 17(2), 165-185.

Holden, K., Kalish, C., Scheinholtz, L., Dietrich, D., & Novak, B. (2009). Financial literacy programs targeted on pre-school children: Development and evaluation (La Follette School of Public Affairs Working Paper No. 2009-009). Madison, WI: La Follette School of Public Affairs, University of Wisconsin-Madison.

Lucey, T. A., & Giannangelo, D. M. (2006). Short changed: The importance of facilitating equitable financial education in urban society. Education and Urban Society, 38(3), 268-287.

Seidman, E., & Tescher, J. (2003). From unbanked to homeowner: improving the supply of financial services for low-income, low-asset customers. (Joint Center for Housing Studies Working Paper Series, BABC 04–4). Cambridge, MA: Joint Center for Housing Studies, Harvard University.

Sherraden, Margaret. S., Johnson, L., Guo, B. and Elliott, W. (2010). Financial capability in children: Effects of participation in a school-based financial education and savings program. Journal of Family and Economic Issues, 32(3), 385-399.

Stegman, M. A., & Faris, R. (2003). Payday lending: A business model that encourages chronic borrowing. Economic Development Quarterly, 17, 8–32.

1 District administrators felt strongly that all students should benefit from the financial education component of this study. Therefore, students in the financial education control group received the 5 lessons after the study’s follow-up assessment had been completed.

2 Student tellers are “hired” by RCU to assist staff in processing transactions and marketing accounts to students.