State Energy Program Evaluation

OMB Control No. XXXXXXXX

ID-1: Codes and Standards (Program Managers)

OMB Control No. XXXXXXXX

This interview guide is designed to provide direction to the interviewer to ensure that all relevant topics are explored to the extent possible and appropriate with the respondent. Note that our interviews are meant to be somewhat informal and open ended – not all topics will be covered in all interviews and we expect that some interviews will lead to the exploration of topics not included in this guide.

Information in [BRACKETS] will be customized to reflect the unique PA program [PROGRAM] and year of the PA offering [PROGRAM YEAR].

BACKGROUND INFORMATION (to be filled in prior to interview):

Program Administrator Name:
Year:
BPAC Area
2008 Budget:
2008 Market Title Sampled
2009-2010 ARRA budget
2009-2010 ARRA Market Title Sampled
Known programmatic activities prior to interview:
Description of target markets:
Structure of SEP/ARRA funded activities from informal discussions with SEP representative from first round discussions (from database)
Contact Name:
Contact Company:
Contact Phone:
Contact Disposition:

PART 1. INTRODUCTION

My name is [INTERVIEWER NAME] from [INTERVIEWER ORGANIZATION]. The US Department of Energy’s State Energy Program has hired us to gather information on the results of efforts to improve building energy codes and their enforcement that received funding in from the State Energy Program and the American Recovery and Reinvestment Act. This interview is being conducted as part of an evaluation of the State Energy Program being conducted by Oak Ridge National Laboratory on behalf of the U. S. Department of Energy.

The U.S. Department of Energy (DOE) would like to inform each individual that the information requested here is being solicited under the statutory authority of Title III of the Energy Policy and Conservation Act of 1975, as amended, which authorizes DOE to administer the State Energy Program (SEP). This information is being sought as part of a national evaluation of SEP, the purpose of which is to reliably quantify Program accomplishments and help inform decisions on future operations. The sole use of the information collected will be for an analysis of national-level Program impacts. Disclosure of this information is voluntary and there will be no adverse effects associated with not providing all or any part of the requested information. The survey should take no more than 95 minutes for you to complete.

The information from all respondents will be combined for analysis purposes and data will not be released in a way that would reveal an individual respondent. If you prefer not to answer a question, just let me know and we’ll go on to the next question. If you have any questions about this study, you can contact [MAIN STUDY CONTACT?].

PART 2. IDENTIFY THE APPROPRIATE RESPONDENT(S)

Our understanding is that you are the program manager for Codes and Standards programs that received SEP/ARRA funding in [YEAR]? Is this correct:

Yes – Correct person and year for SEP/ARRA funded Codes & Standards programs	[Continue]
Yes – Correct person, wrong year for SEP/ARRA funded Codes & Standards programs	[Correct year and Continue]
No – Incorrect person for SEP/ARRA funded Codes & Standards programs	[Ask for correct person for Codes and Standards Programs]
No – There were no SEP/ARRA funded Codes & Standards programs	[Thank you and terminate. Contact main state energy office representative to verify that there were no programs that corresponded to description in the PAGE database.]

1. Were you involved in the design and/or management of programs to improve building energy codes and/or their enforcement in [STATE] that received SEP/ARRA funding in [PROGRAM YEAR]?

1 Yes

2 No [IF NOT, COLLECT CONTACT NAME AND TELEPHONE NUMBER. IF NO ONE AT ORGANIZATION IS KNOWLEDGEABLE ABOUT THE PROGRAM, STATE DETAILED REASON]

Yes	[GO TO Q3]
No	[ASK Q2]

2. Who else at your organization is most knowledgeable about the overall allocation of funding to Non-Residential retrofit programs that received SEP/ARRA funding?

Name/contact info/role: ________________________________________

Name/contact info/role: ________________________________________

Name/contact info/role: ________________________________________

3. [IF OTHERS MENTIONED IN Q2] Among you and [INSERT QUESTION 2 RESPONSE], who is most familiar with the activities conducted by your organization for [PROGRAM] in [PROGRAM YEAR]?

[IF NOT RESPONDENT, THANK AND TERMINATE AND CONTACT MOST KNOWLEGEABLE PERSON]

4. What were your responsibilities in regard to retrofit programs that received SEP/ARRA funding in [PROGRAM YEAR]?

5. In what year did you first become involved with the retrofit programs that received SEP ARRA funding?

6. Are you still involved with managing this program? If not, when did your involvement end?

7. How long have you been involved in administering programs funded by SEP?

8. Based on information provided by DOE, you received $__________ in SEP/ARRA funding for Codes and Standards programs. Is this correct?

Listed Amount	Corrected Amount
[AMOUNT ]_______	[AMOUNT ]
	[REMAINING AMOUNT]

PART 3: PROGRAM DESIGN AND OPERATIONS

This next set of questions seeks information on the design and operations of the program.

9. What specific activities is your organization carrying out to promote the adoption of improved energy codes or improvement in code enforcement and compliance. This would include all activities, whether funded by the U. S. Department of Energy or other sources. Examples would include technical support for adoption of energy efficiency codes at the state and local level, support for legislative deliberations on code adoption, training of local code officials, training of builders and designers, and so forth.

10. Which of these activities receives funding from the U. S. Department of Energy’s State Energy Program (SEP) or the American Recovery and Reinvestment Act (ARRA)?

11. What percentage of total funding from the U. S. Department of Energy for Codes and Standards efforts do each of these separate activities receive? [SHOULD ADD TO 100%.]

12. Which of these components receive resources such as funding, staff support, or other in-kind contributions from other organizations? Please identify the organization and briefly characterize the kinds of assistance provided.

RESPONSE MATRIX FOR ITEMS 9 - 12

9.	10.	11.	12.
Activity to Support Code Improvements	SEP Funded?	% SEP Funding	Other Organizations Providing Support/Nature of Support/Funding Dollars
a.
b.
c.
d.
e.

FOR EACH PROGRAM MENTIONED IN ITEM 9, ASK 13-15

13. Do these activities have separate program names? If so, what are they?

14. Are you or another program manager in charge of any of these programs? [IF THERE ARE OTHER PROGRAM MANGERS, ASK Q15]

15. Can we contact the program manager regarding [PROGRAM NAME]?

[IF YES, RECORD THEIR NAME AND CONTACT INFORMATION]

RESPONSE MATRIX FOR ITEMS 13-15

13. Program Name	14. Additional program manager	15. Permission to contact
a.	[YES/NO]	[RECORD CONTACT INFO]
b.	[YES/NO]	[RECORD CONTACT INFO]
c.	[YES/NO]	[RECORD CONTACT INFO]
d.	[YES/NO]	[RECORD CONTACT INFO]
e.	[YES/NO]	[RECORD CONTACT INFO]

[THESE QUESTIONS ARE DESIGNED TO FLESH OUT PROGRAM DETAILS TO DEVELOP THE PROGRAM LOGIC MODEL. THE LOGIC MODEL ARTICULATES WHAT THE PROGRAM IS TRYING TO ACHIEVE (AND DID ACHIEVE), THROUGH WHAT INTERVENTIONS, AND WITH RESPECT TO WHICH MARKET ACTORS. THIS INFORMATION WILL BE USED TO DEVELOP A MAP OF THE PROGRAM’S DOMAIN FOR ATTRIBUTION PURPOSES.]

16. What were the ultimate objectives of [PROGRAM NAME], as originally planned?

________________________________________________________________________

17. Have these objectives changed since the inception of the program? IF YES: Please describe the current objectives and the rationale for changing them from the original.

_________________________________________________________
_________________________________________________________
_________________________________________________________

18. Has your organization developed a formal logic model for [PROGRAM NAME]?

[IF 18 = YES, then ASK 19]

19. Can you please share it with us?

________________________________________________________________________

20. In what year did the program begin?

________________________________________________________________________

21. Please provide additional details on the activities you are carrying out to advance energy efficiency building codes and their enforcement. [PROBE ON EACH OF THE FOLLOWING]:

ADVOCACY FOR CODE IMPROVEMENTS/ADOPTION AT THE STATE LEVEL
TECHNICAL SUPPORT FOR CODE IMPROVEMENTS/ADOPTION AT THE STATE LEVEL
ADVOCACY FOR CODE IMPROVEMENTS/ADOPTION AT THE LOCAL LEVEL
TECHNICAL SUPPORT FOR CODE IMPROVEMENTS/ADOPTION AT THE LOCAL LEVEL
OUTREACH AND GENERAL EDUCATION ACTIVITIES TARGETED TO LOCAL BUILDING DEPARTMENTS AND CODE OFFIALS:
TRAINING AND TECHNICAL ASSISTANCE TARGETED TO LOCAL BUILDING DEPARTMENTS AND CODE OFFICIALS
PROVISION OF FINANCIAL OR IN-KIND SUPPORT FOR IMPROVED ENERGY CODE ENFORCEMENT ACTIVITIES BY BUILDING DEPARTMENTS.
OUTREACH AND GENERAL EDUCATION ACTIVITIES TARGETED TO BUILDERS, CONSTRUCTION MANAGERS, AND DESIGNERS:
TRAINING AND TECHNICAL ASSISTANCE TARGETED TO BUILDERS, CONSTRUCTIONMANAGERS, AND DESIGNERS
OTHER: (SPECIFY) _
OTHER: (SPECIFY) _
OTHER: (SPECIFY) _

22. INPUTS: What resources are being used to deliver the program? [PROBE EACH OF THE FOLLOWING]

STATE ENERGY OFFICE STAFF: NUMBER, TYPE, PERCENT OF TIME
OTHER GOVERNMENT STAFF: NUMBER, TYPE, PERCENT OF TIME
STAFF OF OTHER AGENCIES OR ORGANIZATIONS: NUMBER, TYPE, PERCENT OF TIME
CONTRACTED SERVICES: PROBE NAME OF PROVIDER AND TYPE OF SERVICE, APPROXIMATE NUMBER OF CONTRACTOR STAFF OR SIZE OF CONTRACT
TECHNICAL SERVICES
FINANCIAL SERVICES
MARKETING SERVICES
ADMINISTRATIVE SERVICES

23. OUTSIDE FUNDING: Are organizations other than the U. S. Department of Energy and the state government contributing funding to this program? IF YES, PROBE:

NAME OF THE ORGANIZATION(S)

AMOUNT OF OUTSIDE FUNDING PROVIDED IN THE PERIOD UNDER EVALUATION (2008 OR ARRA PERIOD)

PERCENTAGE OF TOTAL OVERALL FUNDING FOR THE PA UNDER EVALUATION IN 2007, 2008, ARRA PERIOD, AS RELEVANT REPRESENTED BY OUTSIDE FUNDING.

USES OF THE FUNDING PROVIDED

24. RESULTS/OUTPUTS: Could you summarize the results your program has achieved so far?

NUMBER OF PARTICIPANTS ENROLLED

NUMBER OF UNITS OF SERVICE DELIVERED (AUDITS, WORKSHOP ATTENDANCE, ETC)

NUMBER OF INSTALLATION PROJECTS SUPPORTED

ESTIMATED SAVINGS FROM INSTALLATIONS SUPPORTED

25. RESULTS VERSUS GOALS: Were you able to (Do you anticipate being able to) achieve the level of results you included in your program plans and applications to the U. S. Department of Energy?

IF NO: What circumstances prevented you (are preventing you) from achieving those goals?

1. Characterization of Baseline Construction Practices

This next set of questions seeks information on common construction practices related to energy efficiency during the period immediately prior to [PROGRAM YEAR]. In particular we are interested in practices related to lighting, heating, cooling and related controls, shell insulation and air sealing. Are you able to provide information on standard practices in your jurisdiction, based on your experience in reviewing plans and inspecting projects or on your experience in overseeing such operations? Your general impressions and best approximations are fine for these purposes.

Yes: PROCEED TO BASELINE QUESTIONS

No: IS THERE SOMEONE ELSE IN THE ORGANIZATION WHO CAN PROVIDE THAT INFORMATION?

PROBE TO IDENTIFY STAFF WITH EXPERIENCE IN EITHER RESIDENTIAL, COMMERCIAL, OR BOTH.

26. In this question, I am going to read a specifications for a series of commercial building elements that reflect the International Code Council’s 2009 International Energy Conservation Code. I would like you to tell me what percent of the projects your organization reviewed in the year prior to [PROGRAM YEAR] met this specification. I would also like you to tell me what the most frequent value for the specification was. So, for example, the IECC 2009 requires that unitary cooling equipment from 65,000 to 135,000 BTU/hr have an EER of at least 11.0. We would like to know what percent of projects with cooling equipment of that type met the 11.0 EER standard, and what the typical alternative EER level was in the year prior to PROGRAM YEAR]. Finally, since construction practices vary a great deal in the commercial sector, we are seeking your best estimates for three different building types: offices, retail, and institutional, such as schools. REFER TO THE ANSWER GRID. IT MAY BE BEST TO EMAIL THE ANSWER GRID IN ADVANCE AND HAVE THE RESPONDENT FILL IT OUT PRIOR TO THE CALL.
    

1. What percentage of [Office, Retail, Schools] projects completed in [YEAR PRIOR TO PROGRAM YEAR] met the following specifications?
   
   
   

2. What was the typical alternative specification at the time?

27. In this question, I am seeking the same kind of information for residential buildings from single family homes to 4 units, and for multifamily buildings with 5 or more units.

26. Commercial Baseline Practices

		a. % Meeting Spec.			b. Alternative Specification
Component	IECC 2009 Specification*	Office	Retail	Inst.	If Applicable
Roof Insulation	R-19 for standing seam roof /single fiberglass layer; R-19 + R13 for multiple layers
Wall Insulation	R-19 for single faced fiberglass insulation layer between wall panels & steel frame. R-13 + 5.6 rigid insulation
Maximum Window Area	40% of above-grade wall area
Unitary AC/Air Cooled	< 65 kBtu/hr, ~ 5 tons SEER 13.0 65 – 135 kBtu/hr, ~ 10 tns EER 11.0 135 – 240 kBtu/hr ~ 20 tns EER 10.6 > 240 kBtu/hr EER 9.5
Packaged Thermal AC & Heat Pumps	AC EER 12.5 HP EER 12.3
Water cooled chillers	< 300 tons .634 kW/ton 300 – 600 tons .576 kW/ton
Thermostatic Controls	Automatic setback and shut-off capability for 7 daily schedules/week
Demand Control Ventilation	Required in all spaces > 500 sf with average occupancy > 40 persons/1000 sf
Energy (Heat) Recovery Ventilation	All fans > 5,000 CFM and > 70% outside air supply
Economizers	Required in all cooling systems > 54 kBtu
Interior Lighting Controls	Dual switching to reduce connected load by 50% or occupancy sensors
Interior Lighting Controls	Automatic shut-off via time clock, EMS, or occupancy sensors
Interior Lighting Power Density	Office 1.0 Watt/sf Retail 1.5 Watt/sf School/University 1.2 Watt/sf
Exterior Lighting Control	Motion sensor or time clock

* Some specifications for shell elements change depending on climate zone. Consult IECC 2009 for correct values.

27. Residential Baseline Practices

		a. % Meeting Spec.
Component	IECC 2009 Specification	1 – 4 units	> 5 units	b. Alternative Specification If Applicable
Ceiling Insulation	CZ 1-3 R-30 CZ 4-5 R-38 CZ 6-8 R-49
Wall Insulation	CZ 1-4 except Marine R-13 CZ 4 Marine-6 R-20 CZ 7-8 R-21
Floor Insulation	CZ 1-2 R-13 CZ 3-4 except Marine R-19 CZ 4 Marine - 6 R-30 CZ 7-8 R-38
Basement Insulation	CZ 1-2 0 CZ 3 R-5 Sheathing or R-13 Cavity Fill CZ 4 Marine - 6 R-10/R-13 CZ 7-8 R-15/R-19
Slab Insulation	CZ 1-3 0 CZ 4-5 R-10 to 2 ft depth CZ 6-8 R-10 to 4 ft depth
Fenestration U Factor	CZ 1 1.20 CZ 2 0.65 CZ 3 0.50 CZ 4 – 8 0.35
Air Leakage	Tested – maximum 7 air changes/hr at 33.5 psf (50 pascals) or detailed visual inspection by independent inspector
Duct Insulation	R-8 for all supply ducts in attic; others R-6
Duct leakage	Rough-in and post construction testing of all ducts in non-conditioned areas. Maximum 8 cfm leakage per 100 sf of conditioned space
Mechanical Ventilation	Automatic or gravity dampers on all outdoor air intakes and exhausts
Heating & Cooling System Sizing	Per Manual J or International Residential Code
Interior Lighting	50% of lamps in permanent fixtures must be high efficacy (Fluorescent, CFL, LED)

2. Effect of Program on Acceleration of Statewide Code Adoption

IF ONE OF THE OBJECTIVES OF THE PROGRAM WAS TO ACCELERATE ADOPTION OF IECC 2009 AS THE STATEWIDE ENERGY EFFICIENCY CODE, ASK THIS SECTION. OTHERWISE SKIP TO NEXT SECTION.

28. Please describe the process by which changes to building codes are introduced and approved in your state. PROBE ROLES OF VARIOUS STATE AND LOCAL GOVERNMENT UNITS AND BRANCHES OF GOVERNMENT.

_________________________________________________________
_________________________________________________________
_________________________________________________________

29. On what date was IECC 2009 adopted as part of the state’s building code?

30. What is the effective date of the code change?

31. What energy conservation or efficiency codes, if any, were in place prior to the adoption of IECC 2009?
    
    
    
    

32. Do you think your state would have adopted or updated the statewide energy code by 2020 in the absence of the incentives and supports offered by ARRA?

1. Yes

2. No

3. Don’t know
   
   
   

33. IF 32 = NO, ASK: Why do you say that?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

34. IF 32 =YES, ASK: In the absence of the requirement that the state adopt IECC 2009 as its energy code, in what year do you think the it would have come into effect in this state?

2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020

35. What are your main reasons for this assessment?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

Additional follow-up questions

36. During the 5 years prior to [PROGRAM YEAR], had there been efforts to adopt or update a statewide energy efficiency building code?

1. Yes

2. No

3. Don’t know

37. IF 36 = YES, ASK: Which organizations advocated for adoption of a new statewide energy efficiency code?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

38. What organizations or constituencies opposed adoption of new codes?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

39. What were their principal objections to the adoption of new codes?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

40. How effective were the advocates and opponents of the code in enlisting political support for their views? What evidence do you have for this assessment?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

41. What were the outcomes of these efforts? Was a new or updated code adopted?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

42. On a scale of 1 to 10, where 1 means “not at all important” and 10 means “very important”, how important was your state energy office’s involvement in the development and adoption of the energy efficiency building code?

ENTER 1 – 10, 99 FOR DK/REF: ________________

43. Why do you say that?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

44. Were there other business, economic, or political factors in the state that were supporting or working against the adoption of energy codes immediately prior to [PROGRAM YEAR]? IF YES: What were they?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

3. Effect of the Program on Local Code Adoption

IF ONE OF THE OBJECTIVES OF THE PROGRAM WAS TO SUPPORT AND ACCELERATE ADOPTION OF LOCAL ENERGY EFFICIENCY CODES, ASK THIS SECTION. OTHERWISE SKIP TO NEXT SECTION. YOU SHOULD NOT ASK THIS SECTION IF YOU HAVE ASKED SECTION D.

According to descriptions of the state activities provided to us, one of their objectives was to promote the development and adoption of energy efficiency codes in selected local jurisdictions, including yours.

45. What building energy efficiency codes, if any, were in place in those jurisdictions prior to this effort?

__________________________________________________________________
__________________________________________________________________
_________________________________________________________________

46. Can you describe the codes that those jurisdictions adopted with the assistance of the state energy office?

__________________________________________________________________
__________________________________________________________________
_________________________________________________________________

47. Do you think your jurisdiction would have adopted any building energy code by 2020 in the absence of the programs and support offered by the state energy office?

1. Yes

2. No

3. Don’t know
   
   
   

48. IF 47 = NO, ASK: Why do you think that?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

49. IF 48 =YES, ASK: In the absence of the programs and support you received from the state energy office, in what year do you think that the energy efficiency codes?

2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020

50. What are your main reasons for this assessment?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

4. Effects of Programs to Strengthen Code Enforcement

ASK THIS SECTION ONLY IF PA INCLUDES ELEMENTS TO IMPROVE CODE ENFORCEMENT AT THE LOCAL OR STATE LEVELS. SUCH ELEMENTS INCLUDE TRAINING OF CODE OFFICIALS, TECHNICAL ASSISTANCE TO CODE OFFICIALS, STIPENDS TO HIRE ADDITIONAL CODE OFFICIALS, PURCHASE OF COMPLIANCE SOFTWARE, ETC.

51. Did you observe changes in the quality or level of code enforcement activities among the local building departments that you assisted through programs funded by SEP or ARRA?

1. Yes

2. No

3. Don’t know

52. IF YES: Could you describe those changes for me?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

53. On a scale of 1 to 10, where 1 means “very unlikely” and 10 means “very likely”, how likely is it that the participating jurisdictions would have undertaken these improvements to your enforcement activities if the support from the state energy office had not been available?

ENTER 1 – 10, 99 IF DK OR REF _____________________________

54. Why do you say that? PROBE AVAILABILITY OF RESOURCES, LEVELS OF TRAINING, AVAILABILITY OF COMPLIANCE TOOLS, LEVELS OF UNDERSTANDING, ETC.

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

55. Do you believe that the efforts of the participating jurisdictions to improve enforcement have resulted in improved compliance with energy efficiency elements of the building code?

1. Yes

2. No

3. Too early to tell

4. DK/REF

56. IF 55 = Yes or No, ASK: What observations lead you to say that?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

57. Are there factors other than increased enforcement efforts that have contributed to increased compliance rates? IF YES: Could you identify those factors.

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

58. IF 57 = YES: Compared to these other factors, would you say your enforcement efforts had less, more, or about the same level of influence in changes in compliance rate?

1. Less

2. More

3. About the same

4. DK/REF

5. Effects of Programs to Train Builders and Architects in Code Issues

ASK THIS SECTION ONLY IF PA INCLUDES ELEMENTS TO EDUCATE BUILDERS AND DESIGNERS ON CODE REQUIREMENTS AND COMPLIANCE MECHANISMS

59. Do you believe that builders and designers who made use of your training and education programs in building energy codes made changes in their practices in regard to design and code compliance?

1. Yes

2. No

3. DK/REF
   
   
   

60. IF 59 = YES: Could you describe those changes for me?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

61. What factors, other than your organization’s efforts, have led to changes in voluntary levels of compliance with energy efficiency aspects of the building codes?

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

62. IF 65 = YES: Compared to other these other factors, would you say these education efforts have had less, more, or about the same level of influence in changes in compliance practice?

1. Less

2. More

3. About the same

4. DK/REF

PART 4. RESOURCE CONTRIBUTIONS FROM OTHER PROGRAMS

[ASK PART 4 IF OTHER ORGANIZATIONS SUCH AS UTILITIES OR OTHER GOVERNMENT COOPERATED IN THE DELIVERY OF THE PA THROUGH CONTRIBUTIONS OF STAFF TIME, FUNDING, OR OTHER SERVICES.]

LEAD-IN: Earlier we spoke about contributions that other organizations made to the non-residential retrofit programmatic activity. I’d like to ask a few more questions about that.

63. First, prior to 2008, had you cooperated with this (these) organization(s) in delivering energy programs to increase energy efficiency programs?

IF YES, ASK

1. Please describe how you worked together with the other organization(s) and what specific activities they undertook.

2. What kinds of resources did the other organizations provide?

1. Did your organization take the initiative to arrange for cooperative activities related to the program? Did the other organizations take the initiative? Or did the cooperation arise through some other mechanism?

65. Did representatives of the two (or more) organizations meet to jointly plan how the organizations would cooperate in the development of the program?

66. Do representatives of the organization meet on a regular basis to review program accomplishments and plan future activities?

IF YES: How often does that occur?

67. Is there regular interaction among the organizations in delivering the program services?

IF YES: Could you describe that interaction for me?

68. What do you believe are the other organizations’ main motivations for contributing to the delivery of the program?
    
    
    
    

69. If the opportunity to offer joint programming with your agency had not been available, do you believe the other organizations would offer programs to promote energy efficiency in non-residential facilities?

YES/NO

Why do you say that?

70. If the other organizations had not cooperated in delivering the program as they did, would your organization have changed the type of services it provides to promote energy efficiency in non-residential facilities?
    
    YES/NO

IF YES, PROBE:

. How would your organization have changed the roster of services provided?

. In the absence of cooperation from other organizations, would the level of resources your organization dedicated to these services have been less, the same, or greater?

. Why do you say that?

PART 6: DATA AVAILABILITY

[ONLY ASK THIS SECTION IF DATA ARE NOT AVAILABLE ALREADY]

[ASK SECTION 4 FOR EACH OF THE PROGRAMS LISTED IN REPSONSE MATRIX A. IF NECESSARY, SCHEDULE A CALLBACK WITH THE APPROPRIATE PROGRAM MANAGER]

NOTE: QUESTIONS 31-34 ON DATA AVAILABILITY WILL BE REVIEWED WITH PROGRAM SPONSORS PRIOR TO FINAL SELECTION OF THE PROGRAM INTO THE SAMPLE. WE WILL ALSO ASK TO RECEIVE A COPY OF THE DATABASES OR FILE FORMATS SO THAT WE CAN MAKE A JUDGMENT REGARDING THE ADEQUACY OF THE DATABASE TO SUPPORT EVALUATION ANALYSIS. THESE QUESTIONS WILL BE FOR VALIDATION OF OUR REVIEW.

71. Please describe the types of data that you tracked in [PROGRAM YEAR] on program participants. Do you have lists of:

1. Key contacts from the implementation organization (including contact information)

2. Program participants, that is, local code officials, workshop attendees or training recipients, recipients of technical assistance, with contact information

(IF DATA IS AVAILABLE BUT RESPONDENT IS NOT KNOWLEDGEABLE/DOES NOT HAVE ACCESS TO IT, RECORD NAME OF CONTACT WHO CAN ANSWER FOLLOW-UP QUESTIONS ABOUT DATA AVAILABILITY. PROBE FOR WHETHER ANYONE MAINTIANS SUCH DATA – POTENTIALLY EXTERNAL TO THE PA). IF THERE IS NO ONE CAN PROVIDE SUCH DATA THEN THANK AND TERMINATE).

72. Have there been any evaluations of this program? Are data from these evaluations available?

73. (FOR EACH TYPE OF DATA AVAILABLE) Is this data in electronic format? (If yes, discuss steps needed to secure permission to access the data via secure file transfer. If no, determine how data records are maintained and how they could be accessed.)

Thank you for your time and insights

20

OMB Control No. XXXXXXXX