Attachment F - Individual Diary Analysis Plan

Attachment F - Individual Diary Analysis Plan_07_09_13.docx

Consumer Expenditure Surveys: Quarterly Interview and Diary

Attachment F - Individual Diary Analysis Plan

OMB: 1220-0050

Document [docx]
Download: docx | pdf


Analysis Categories

Variables of Interest

Counteracting Influences

Matching criteria

We will match research and Production the factors of:


  1. internet access

  2. Smartphone access


We will analyze whether Research and Production are different on the following and match accordingly:

  1. age

  2. race

  3. sex

  4. education

  5. income


We will also determine whether double-placed diaries differ from single-placed diaries with regard to expenditures, controlling for 2-week versus 1-week recall. If they do, we will match the Research sample to Production double-placed diaries.

Objective 1: Operational issues and Overall Assessment for each diary mode

A simple diary should be able to be completed quickly and easily.

R’S GENERAL COMMENTS

  1. Respondents’ positive impressions (open-ended, ID_R1)

  2. How to improve the process (open-ended, ID_R3)

  3. How secure it felt (ID_R4)

R’S BURDEN

  1. Respondent estimates of time needed to complete diary [paradata]

  2. Respondent assessments of ease/difficulty of logging in (ID_R2)

FR ASSESSMENT OF R.

  1. FR assessment of R. cooperativeness (FR and ID_FR)

FR’S GENERAL COMMENTS – Not in CAPI right now, since not at the case-level. This will asked during a debriefing session

  1. What liked about the process

  2. What disliked about the process

  3. How to improve the process

  4. How it changed workload

NPC CODERS ASSESSMENT

  • Exact questions TBD

Content of Help Desk inquiries

  • DSD to Access database contents for help desk tickets

We will only have this information for the test group, so no comparisons between research-production can be made.



Objective 2: Item level analysis



At a CU level, there should be no differences in data quality between the paper and individual diaries.


  1. Number of entries per section1

  2. Compare CAPI entries for cases with no expenditure entries

  3. Average number of entries per week compared with production numbers that are:

  1. CUs where no recall carried out (74% of total) – 34.7 entries (Mean)

  2. All CUs (including CUs with FR-assisted recall) – 31.6 entries (Mean)

  1. Average number of blank diary pages per week compared with production numbers that are:

  1. 2.1 blank diary pages per week (Mean)2

  1. Expenditure amounts per section – note we only have power for the “Food” category

  2. There should be equal amount or fewer entries that are

a) rounded off dollars with no cents (e.g., $4.00) – if possible to analyze

b) typical estimating amounts of $1.00, $5.00, $10.00, $20.00 etc.)

7. Item missingness at category level for $ amount, etc.

8. Audit trail files (stored as txt files) that Phase 3 generates during questionable code review – if possible to analyze

9. Rate of duplicate entries for expenditures a. receipt entries by FR (into the CAPI system) that are duplicates to the CU entries

b. entries by the CU (since different people can make different entries)

10. Fails phase 2 edits (case blank)

11. Screen for outliers in respondent debriefing questions

  1. Review timing data for this section




NPC CODERS ASSESSMENT

  • Exact questions TBD – such as number of overrides at NPC


If interviewers have not been adequately trained on the diary or if they do not spend adequate time explaining the sections during placement, then the beneficial aspects of the diary may be obscured.

Objective 3: Response



A mobile and web diaries should seem more convenient than a paper diary and therefore compliance should be improved.



  1. Diary Placement Rates (overall, since this is a double-placement)

  2. 1st Diary Completion Rates – using the same completion rules as Production

  3. 2nd Diary Completion Rates – using the same completion rules as Production

  4. Type B and C Rates taking into account sceenouts

  5. Rate of Total Recalls

  6. Cross-tab how FR felt about the materials (not good) versus DQ of respondent






Interviewers will be less familiar with the new diaries and this may make them hesitant in their presentation and placement.


Interviewers may also recognize that this is only a test, leading them to be less diligent in pursuing non respondents because these Type A’s will not be held against them.


Objective 4: Data Entry Patterns

  1. Number of log-ins per case

  2. Examine relationship between log-ins per case and FR phone calls during the collection period using CHI data

  3. Start/stop time stamps by diary day (to determine multiple times per day versus only data entry at the end of the week)

  1. Analyze log-in at least once on consecutive days versus log-in sporadically

  2. Analyze log-in at least once on consecutive days versus TR

  3. Analyze data entry at least once on consecutive days versus data enter sporadically



Raw paradata from ASD to be used:

1.       Browser/device information

2.       Authentication information (successful, unsuccessful, date, time)

3.       Respondent IDs

4.       Page load date/time

5.       Answer and answer change (date/time)





















Debriefing Questions

ID_intro. Now I am going to ask ^NAME’s experience with the diary.

  1. Enter 1 to continue

ID_R1 What did you like about this application?


ID_R2 How easy or difficult was it to log in? Would you say –

  1. Very easy?

  2. Easy?

  3. Neither easy nor difficult?

  4. Difficult?

  5. Very difficult?

  6. Not Present

ID_R3 How could we improve the process of recording your expenses?

ID_R4 When using the mobile optimized application/web diary to record your expenses, how secure do you feel that your data is? Would you say –

  1. Completely secure?

  2. Somewhat secure?

  3. Neither secure nor unsecure?

  4. Somewhat unsecure?

  5. Completely unsecure?

  6. Not Present

ID_FR Would you say the respondent was –

  1. Very cooperative?

  2. Somewhat cooperative?

  3. Neither cooperative nor uncooperative?

  4. Somewhat uncooperative?

  5. Very uncooperative?


1 The Hong Kong study (Grootaert, 86) found higher reporting in individual diary cases for clothing and footwear. Pearl (1979) also expected improved individual-level reporting on the CE for ‘small clothes’, reading material, public transportation and personal services.

2 This counts a page as blank if recall was used, but does not count as a blank if page was et as a vacation day.

55645301 2 Page 2


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleDIARY FIELD TEST ANALYSIS PLAN
AuthorSTINSON_L
File Modified0000-00-00
File Created2021-01-25

© 2024 OMB.report | Privacy Policy