Pretesting Results 2

PROOF OF CONCEPTS TEST
FIELD REPRESENTATIVE (FR) DEBRIEFING QUESTIONNAIRE
(n = 33)
Instructions: The purpose of this questionnaire is to obtain feedback on your experiences during the
Proof of Concepts (POC) test. The current assumption is that there will be no changes in the FR’s job as
a result of the RO restructuring. We would like to get information on your experiences during the POC
test to determine if your experiences are consistent with this assumption. We are particularly interested
in any changes in your interactions with other team members or the RO, and the resulting impact on how
you did your job. Your responses will be used to help us improve our implementation strategies for the
RO realignment and the training we provide the Field Supervisor.
Please complete the questionnaire and mail it via UPS to: U.S. Census Bureau, FLD/MRB (5H051), 4600
Silver Hill Road, Suitland MD 20746 no later than COB Friday, November 4, 2011. You may charge up
to one hour to complete the survey. Please charge your time to project code 0329020 and task code 530.
Thank you for your feedback. Your responses will be kept confidential!
1. During and prior to the POC test, in an average week, how much time did you spend talking
(in person or on the phone) with your Field Supervisor or your immediate supervisor?
I talked with my Field Supervisor/immediate supervisor…
Several
About
Several
About an Less than Other –
hours
an hour hours per
hour per
an hour
please
per day
per day
week
week
per week specify
(1)
(2)
(3)
(4)
(5)
(6)
a.DURING the POC test…
3.0
3.0
27.3
30.3
33.3
3.0
b. PRIOR to the POC test…
0.0
3.1
9.4
31.3
53.1
3.1

2. During and prior to the POC test, how satisfied were you with the amount of time your Field
Supervisor or immediate supervisor was able to spend with you?
Very
Very
satisfied Satisfied Somewhat satisfied / Dissatisfied dissatisfied
with the with the somewhat dissatisfied
with the
with the
amount
amount
with the amount of
amount of
amount of
of time
of time
time
time
time
(1)
(2)
(3)
(4)
(5)
a.DURING the POC test, I was
66.7
30.3
3.0
0.0
0.0
b.PRIOR to the POC test, I was
54.8
41.9
3.2
0.0
0.0

1 

3. If you were dissatisfied with the amount of time your Field Supervisor and/or immediate

supervisor were able to spend with you during and/or prior to the POC test, please explain
below what caused the dissatisfaction.
SEE ATTACHED
During the POC test, did the amount of time your Field Supervisor spent discussing the
following activities with you appear to increase, decrease or stay the same?
During the POC test, the…
Amount of
Amount of
Amount of review
review time
review time
time stayed the
increased
decreased
same
(1)
(2)
(3)
a. Hours claimed on your T&A
22.6
0.0
77.4
b. Mileage claimed on your T&A
9.7
3.2
87.1
c. Expenses claimed on your T&A
16.7
0.0
83.3
d. Number of contacts per case
25.0
3.1
71.9
e. Timing of the contacts per case
15.6
3.1
81.3
f. CHI results
16.1
0.0
83.9

4.

5. During the POC test, you had one supervisor who was the point of contact for all questions,

issues or concerns for all surveys. Prior to the POC test, you had an SFR and possibly several
Program and/or Department Supervisors to go to if you had questions, issues or concerns.
How often were you able to get immediate help when you contacted your supervisor(s)?

a.DURING the POC test…
b. PRIOR to the POC test…

My supervisor(s) provided immediate help…
Always
Frequently Occasionally
Rarely
Never
(1)
(2)
(3)
(4)
(5)
62.5
28.1
6.3
3.1
0.0
53.1
40.6
3.1
3.1
0.0

6. Overall, how helpful was your Field Supervisor with any problems, issues or concerns you had
during the POC test?
(90.9) Very helpful – go to question 8

(9.1) Somewhat helpful – go to question 8

(0.0) Not very helpful
(0.0) Not at all helpful

2 

7. If you felt your Field Supervisor was not helpful, please explain how your Field Supervisor

could have been more helpful?
SEE ATTACHED
8. Overall, how helpful were your SFR/Program Supervisor(s) with any problems, issues or
concerns you had before the POC test?
(74.2) Very helpful – go to question 10
(25.8) Somewhat helpful – go to question 10

(0.0) Not very helpful
(0.0) Not at all helpful
9. If you felt your SFR/Program Supervisor(s) were not helpful, please explain how they could

have been more helpful?
SEE ATTACHED
10. During the POC test, how often did you contact your Field Supervisor with a question about
each of the topics listed below?
During the POC test, I contacted my Field Supervisor…

a.Your workload
b. A difficult respondent
c. Respondent letters
d. Problem referrals
e. Clarification of a concept
or procedure on one of your
surveys
f. Survey-specific emails
g. Automation
h. Payroll
i. Leave
j. Safety
k. Incident Reports
l. Supplies and materials
m. PII or Confidentiality
n. Personal matters
o. Other- specify: Fast Data;
code/general questions;
transfer – loss of cases

Every
Day
(1)
3.1
0.0
0.0
0.0

Several
times a
week
(2)
15.6
21.2
9.1
21.2

Once
a
week
(3)
15.6
24.2
24.2
9.1

Less than
once a
week
(4)
28.1
30.3
27.3
21.2

Once
a
month
(5)
15.6
18.2
15.2
9.1

Less than
once a
month
(6)
3.1
0.0
9.1
18.2

0.0

15.2

3.0

21.2

24.2

0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0

21.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
3.0

12.1
6.3
6.1
3.0
0.0
0.0
0.0
0.0
3.0

18.2
9.4
15.1
3.0
3.0
3.1
9.1
12.1
12.1

0.0

10.0

10.0

0.0

3 

Never
(7)

N/A
(8)

18.9
3.0
12.1
9.1

0.0
3.0
3.0
12.1

12.1

21.2

3.0

18.2
18.7
15.1
6.1
3.0
0.0
24.2
3.0
18.2

12.1
40.6
39.4
33.3
21.2
15.6
24.2
27.3
15.1

15.2
21.9
21.2
39.4
66.7
65.6
36.4
48.5
39.4

3.0
3.1
3.0
15.1
6.1
15.6
6.1
9.1
9.1

10.0

0.0

40.0

30.0

11. Prior to the POC test, how often did you contact your SFR/Program Supervisor with a

question about each of the topics listed below?
Prior to the POC test, I contacted my SFR/RO/Program Supervisor…
Once a
week
(3)

0.0
0.0
3.1
0.0

Several
times a
week
(2)
9.4
9.4
6.2
3.2

15.6
21.9
9.4
12.9

Less
than
once a
week
(4)
15.6
34.4
25.0
25.8

3.1

3.1

3.1

12.5

18.7

0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0

6.2
0.0
3.1
0.0
0.0
0.0
0.0
0.0
0.0

3.1
0.0
0.0
0.0
0.0
0.0
0.0
0.0
3.1

18.8
6.3
6.3
0.0
3.1
3.1
3.1
3.1
3.1

0.0

10.0

0.0

0.0

Every
Day
(1)
a.Your workload
b. A difficult respondent
c. Respondent letters
d. Problem referrals
e. Clarification of a concept
or procedure on one of your
surveys
f. Survey-specific emails
g. Automation
h. Payroll
i. Leave
j. Safety issues
k. Incident Reports
l. Supplies and materials
m. PII or Confidentiality
n. Personal matters
o. Other – specify: task
codes clarification

Once
a
month
(5)
34.4
18.7
15.6
9.7

Less
than
once a
month
(6)
15.6
9.4
18.7
19.3

Never
(7)

N/A
(8)

6.2
0.0
15.6
12.9

3.1
6.2
6.2
16.1

34.4

18.8

6.3

15.6
12.5
6.3
3.1
0.0
0.0
9.4
6.3
15.6

18.8
50.0
37.5
40.6
21.9
9.4
37.5
34.4
28.1

28.1
28.1
40.6
40.6
68.7
75.0
46.9
50.0
43.8

9.4
3.1
6.3
15.6
6.3
12.5
3.1
6.3
6.3

20.0

0.0

50.0

20.0

12. If you feel there was a change in how often you had to contact your Field Supervisor during

the POC test compared with how often you had to contact your SFR/Program Supervisors
prior to the POC test, please explain below why you think this occurred?
SEE ATTACHED

4 

13. During the POC test, if you called the Field Supervisor with a question about one of the topics

listed below, how satisfied were you with your Field Supervisor’s ability to answer your
question(s) about the following topics?
Somewhat
Very
satisfied /
Very
satisfied Satisfied
somewhat Dissatisfied dissatisfied
(1)
(2)
dissatisfied
(4)
(5)
(3)
a.Your workload
60.6
18.2
3.0
6.1
0.0
b. A difficult respondent
57.6
27.3
3.0
6.1
0.0
c. Respondent letters
54.5
21.2
6.1
6.1
0.0
d. Problem referrals
45.4
21.2
3.0
3.1
0.0
e. Clarification of a concept
or procedure on one of your
54.5
21.2
6.1
3.0
0.0
surveys
f. Survey-specific emails
60.6
24.2
0.0
3.0
0.0
g. Automation
30.3
21.2
0.0
6.1
0.0
h. Payroll
50.0
18.7
3.1
3.1
0.0
i. Leave
34.4
12.5
0.0
0.0
0.0
j. Safety issues
25.0
12.5
0.0
0.0
0.0
k. Incident Reports
21.9
6.2
0.0
0.0
0.0
l. Supplies and materials
37.5
21.9
3.1
3.1
0.0
m. PII or Confidentiality
46.9
12.5
0.0
3.1
0.0
n. Personal matters
38.7
12.9
0.0
3.2
41.9
o. Other – specify: how to
handle a case (i.e., what to
11.1
0.0
0.0
0.0
0.0
do to get respondents to do
the survey)

N/A
(6)
12.1
6.1
12.1
27.3
15.2
12.1
42.4
25.0
53.1
62.5
71.9
34.4
37.5
3.2
89.9

14. If you were NOT satisfied with your Field Supervisor’s ability to answer your questions on a

topic, please list the topic(s), the question(s) that your Field Supervisor could not answer, and
what you did to get your question(s) answered. (Please print)
SEE ATTACHED

5 

15. Prior to the POC test, if you called your SFR/Program Supervisors with a question about one

of the topics listed below, how satisfied were you with his/her ability to answer your question(s)
about the following topics?
Somewhat
Very
satisfied /
Very
satisfied Satisfied
somewhat Dissatisfied dissatisfied N/A
(1)
(2)
dissatisfied
(4)
(5)
(6)
(3)
a.Your workload
48.5
33.3
6.1
6.1
0.0
6.1
b. A difficult respondent
39.4
39.4
9.1
0.0
0.0
12.1
c. Respondent letters
45.4
33.3
0.0
0.0
0.0
21.2
d. Problem referrals
27.3
39.4
3.1
0.0
0.0
30.3
e. Clarification of a concept
or procedure on one of your
51.5
27.3
0.0
0.0
0.0
21.2
surveys
f. Survey-specific emails
43.8
28.1
0.0
0.0
0.0
28.1
g. Automation
31.2
21.9
0.0
0.0
0.0
46.9
h. Payroll
31.2
25.0
0.0
0.0
0.0
43.7
i. Leave
31.2
15.6
0.0
0.0
0.0
53.1
j. Safety issues
21.9
15.6
9.4
0.0
0.0
53.1
k. Incident Reports
12.5
12.5
3.1
3.1
0.0
68.8
l. Supplies and materials
28.1
31.2
0.0
3.1
0.0
37.5
m. PII or Confidentiality
40.6
21.9
0.0
0.0
0.0
37.5
n. Personal matters
28.1
31.2
0.0
0.0
0.0
40.6
o. Other – specify
10.0
10.0
0.0
0.0
0.0
80.0

16. If you were NOT satisfied with your SFR’s/RO’s/Program Supervisors’ ability to answer your

questions on a topic, please list the topic(s), the question(s) that your he/she could not answer,
and what you did to get your question(s) answered.
SEE ATTACHED

6 

17. During the POC test, do you think it took more time, less time or the same amount of time to
get answers to your question(s) on the following topics?
During the POC, it took…

a.Your workload
b. A difficult respondent
c. Respondent letters
d. Problem referrals
e. Clarification of a concept or procedure on one of your
surveys
f. Survey-specific emails
g. Automation
h. Payroll
i. Leave
j. Safety issues
k. Incident Reports
l. Supplies and materials
m. PII or Confidentiality
n. Personal matters
o. Other – specify: Fast Data

More time
to get
answers
(1)
12.5
15.6
46.9
17.2

Less time
to get
answers
(2)
15.6
15.6
3.1
17.2

Same amount
of time to get
answers
(3)
71.9
68.8
50.0
65.5

20.0
25.8
10.0
12.5
11.1
3.7
7.7
25.8
6.7
3.3
10.0

20.0
16.1
3.3
12.5
3.7
7.4
7.7
6.5
13.3
13.3
0.0

60.0
58.1
86.7
75.0
85.2
88.9
84.6
67.7
80.0
83.3
90.0

18. If it took more/less time to get your questions answered during the POC test, please explain

below how you handled/dealt with the situation.
SEE ATTACHED

7 

19. During the POC test, if you could not reach your Field Supervisor or she was unable to answer your
question(s) about the topics below, who did you contact?
If unable to reach my Field Supervisor, I contacted…
an
a
the
the
RO
Program
RO
another another OtherSSF Clerk Supervisor Coordinator
FR
SFR
specify N/A
(1)
(2)
(3)
(4)
(5)
(6)
(7) *
(8)
a.Your workload
0.0
3.7
14.8
0.0
0.0
0.0
3.7
77.8
b. A difficult respondent
3.6
0.0
7.1
0.0
3.6
0.0
3.6
82.1
c. Respondent letters
3.6
39.3
3.6
0.0
0.0
0.0
3.6
50.0
d. Problem referrals
3.6
3.6
7.1
0.0
0.0
0.0
3.6
82.1
e. Clarification of a
concept or procedure on
3.6
3.6
10.7
0.0
0.0
0.0
3.6
78.5
one of your surveys
f. Survey specific emails
7.1
10.7
3.6
0.0
0.0
0.0
3.6
75.0
g. Automation
7.1
0.0
10.7
3.6
0.0
0.0
10.7
67.9
h. Payroll
3.6
3.6
10.7
0.0
0.0
0.0
3.6
78.5
i. Leave
3.6
0.0
14.3
0.0
0.0
0.0
7.1
75.0
j. Safety issues
3.6
0.0
7.1
0.0
0.0
0.0
3.6
85.7
k. Incident Reports
3.6
0.0
7.1
0.0
0.0
0.0
3.6
85.7
l. Supplies and materials
3.6
17.8
3.6
3.6
0.0
0.0
3.6
67.8
m. PII or Confidentiality
3.6
3.6
10.7
0.0
0.0
0.0
3.6
78.5
n. Personal matters
3.6
3.6
7.1
0.0
0.0
0.0
3.6
82.1
o. Other – specify:
7.7
0.0
0.0
0.0
0.0
0.0
7.7
84.6
Everything
*Other -specify: Left message or email – always got contacted; Automation staff; SSF almost always returned
calls; I was told not to call anyone but my FS;

20. During and prior to the POC test, how often did you receive feedback on your performance

(i.e. response and production rates) from your Field Supervisor and SFR/Program
Supervisor(s)?
I received performance feedback…
Several
Several
Less than
times a Once a times a
Once a
once a
week
week
month
month
month
(1)
(2)
(3)
(4)
(5)
15.2
33.3
15.2
21.2
15.1
9.7
16.1
6.4
35.5
32.3

a.DURING the POC test…
b. PRIOR to the POC test…

8 

21. Did the number of surveys you worked on change during the POC test?
(30.3) Yes – I worked more surveys.
(3.0) Yes – I worked fewer surveys.
(66.7) No - the number of surveys I worked did not change during the POC test.

22. Did your workload change during the POC test?

(42.4) Yes - my workload was increased. – go to question 23
(6.1) Yes - my workload was decreased. – go to question 24
(51.5) No - my workload did not change during the POC test. – go to question 26

23. How did you feel about the increase in your workload during the POC test?

(37.5) I was happy that I got more work.
(12.5) I was NOT happy that I got more work.
(50.0) It did not matter to me. (But had hoped someone would call me first to let me know to
expect the increased case load.)

24. How did you feel about the decrease in your workload during the POC test?

(16.7) I was happy that I got less work.
(50.0) I was NOT happy that I got less work.
(33.3) It did not matter to me.

25. Why do you think your workload changed during the POC test?

(1)
(1)
(4)
(6)
(10)

I had too many cases assigned to me.
I was working too many surveys.
Several surveys had sample increases and my help was needed.
Not enough FRs were working due to sickness, vacations, etc.
Other – please explain: SEE ATTACHED

26. During the POC test, how timely were survey-specific emails, materials and memos?

(36.4) Always on time - go to question 28
(48.5) Usually on time

(9.1) Occasionally on time
(6.1) Rarely on time

(0.0) Never on time

9 

27. To what extent did receiving late survey-specific emails, materials and memos affect your

ability to do your work?
(34.6) It had a big impact on my ability to do my work.
(34.6) It had a small impact on my ability to do my work.
(30.8) It had NO impact on my ability to do my work.

28. Overall, how did having one person to report to - instead of having multiple survey supervisors

to report to - impact your job as an FR?
(38.7) It made my job a lot easier. – go to question 29
(6.5) It made my job a little easier. – go to question 29
(32.3) It made my job a little more difficult. – go to question 30

(6.5) It made my job a lot more difficult. – go to question 30
(16.1) It did NOT have any impact on my job. – go to question 31
29. If having one person to report to made your job easier, please explain how your job became a

little/lot easier.
SEE ATTACHED
30. If having one person to report to made your job more difficult, please explain how your job

became a little/lot more difficult.
SEE ATTACHED
31. Overall, did you like having one person (i.e., the Field Supervisor) to report to for all surveys?
(74.2) Yes – go to question 32
(25.8) No – go to question 33
32. What did you like about having one person (i.e., the FS) to report to for all surveys?
SEE ATTACHED
33. What did you dislike about having one person (i.e., the FS) to report to for all surveys?
SEE ATTACHED
34. From a Field Representative’s viewpoint, what are the advantages of this new RO structure?

SEE ATTACHED

10 

35. From a Field Representative’s viewpoint, what are the disadvantages of this new RO
structure?
SEE ATTACHED
36. Having participated in the Proof of Concepts test, do you think the Field Supervisor’s job is
one that you would like to have?
(34.5) Yes
(65.5) No – please explain: SEE ATTACHED
37. If there are any additional comments you would like to share about your experiences during

the POC test and its impact on your job as a Field Representative, please describe them below.
If you need more space, please attach an additional piece of paper.
SEE ATTACHED

THANK YOU FOR TAKING THE TIME TO GIVE US YOUR FEEDBACK!!

11