OPA Survey Review Form

Survey Review Checklist

1. Background

Review Type: Survey Focus Group Personal Interview Phone/CATI Interview Other: _______

Review Divisions: CRM DPAC DS H&R JAMRS PERSEREC R&R

Reviewer: Mr. Mike DiNicolantonio

Review Completion Date: 2/24/2020 (Initial); 5/1/2020 (Second)

Study Title/Acronym: United States Military Entrance Processing Command (USMEPCOM) Military Entrance Processing Station (MEPS) Customer Satisfaction Survey

Licensing Requirements: RCS (Internal) OMB (Public) Both

Study PI/Contact Information: Mr. Matthew Lince, Futures Division Chief, Office of Strategic Planning and Transformation; 847.688.3680, Extension 7242

Study Sponsor/Office/Contact Information: Mr. Matthew Lince, Futures Division Chief, Office of Strategic Planning and Transformation; 847.688.3680, Extension 7242

Documents Reviewed: Instrument Supporting Statement Communications Other:

2. General Feedback

1. If I returned any survey documents, they should be the versions used going forward for remaining coordination activities following acceptance of quality control checks (e.g., formatting, grammatical revisions).

3. Instrument

   1. As indicated in Bullet #2 below, the quality of the revised document was much better than the initial submission; well done. However – I see content has changed, which is OK but I had to review it. The following is new feedback that must be addressed before I provide a conditional approval letter.

1. Section 5.1 – Question 1; Section 5.4 – Question 2; Section 6.1 – Question 3; Section 7.1 – Question 4; Section 8.1 – Question 5; Section 9.1 – Question 6: Remove “extremely”. Or revert back to the previous format where there was a scale that included “extremely satisfied” as a response option. There is no need to include this qualifier – the word extremely creates an unnecessary (and biased) connotation of the question; asking if the respondent was satisfied provides a clear data point that can be used to measure satisfaction.

Response: Completed. Removed the word “extremely”. OPA: Confirmed revisions. (5/14/20)

2. I assume Section 5.1 – Question 1 and Section 5.4 – Question 2 do not include N/A as a response option because they will have had to be at a MEPS facility and must interact with MEPS staff, correct? There is no way any processing would not be at a facility or require interaction with staff, right?

Response: There is remote processing; however, those applicants are not offered the survey. Only applicants physically at a MEPS facility are offered the option to take the survey. OPA: Concur; thank you for the clarification. (5/14/20)

3. Question 9.2 – Is N/A a valid response option? Are there cases when an applicant would not see this video or at the very least not be offered a viewing?

Response: An applicant may have multiple MEPS visits; however, they only view the video once. OPA: Concur; thank you for the clarification. (5/14/20)

4. Question 9.3 – Why is N/A included as a response option?

Response: It is a valid response. We added a qualifier. It now states, “Not Applicable; Pre-informed.” An applicant may be aware of all the information in the video prior to viewing, so N/A would be appropriate. OPA: Concur and confirmed revision; thank you for the clarification. (5/14/20)

5. Question 9.4 – Is N/A a valid response option? Are there cases when a recruiter would not explain this process to a potential recruit?

Response: N/A is applicable as we process applicants who have been through the process before, i.e., prior service, disquals, etc. OPA: Concur; thank you for the clarification. (5/14/20)

6. Question 9.5 – Why is this question not constructed like the questions identified in bullet A above? And would N/A be a response option?

Response: It is constructed this way as it mirrors questions 5.2, 5.5, 6.2, 7.2, and 8.2. OPA: Concur; thank you for the clarification. (5/14/20)

2. I assume the survey instrument seen by the respondent is much cleaner than the one presented for review. For example – words such as “fingerprinting area” do not wrap as they do in this version. I highly recommend a better quality document be sent to OMB for review; they should expect to see an instrument presented in better shape than this version.

Response: Completed. OPA: Confirmed revisions. (5/1/20)

3. Privacy Statement/Introduction – Please see the returned version and update per comments added.

Response: Completed. OPA: Revisions were not made; revise and resubmit based on comments provided in returned version (as well as the first version I sent back). (5/1/20). Apologies. We made the change, but it did not save on the version sent to you. It is now correct. Dual statement removed. OPA: Confirmed revisions. (5/4/20)

4. Public Burden Text – Please see returned version and update per comment added.

Response: Completed. OPA: OBE; entire section removed. (5/1/20)

5. Public Burden Text – Why is the last sentence about returning to the survey to MEPS HQ included? The supporting statements indicate this survey is taken during processing; Part A explicitly states there is no paper version of the survey. So how and why would an applicant mail this survey back to this address?

Response: Updated verbiage to reflect current electronic survey process. OPA: OBE; entire section removed. (5/1/20)

6. Open-Ended Questions – DoD surveys require that text stating “Do not include personally identifiable information” be included next to all open-ended questions. One of these questions specifically asks “If you experienced anyone who demonstrated outstanding customer service or encountered anyone who presented unprofessional behavior, please provide specific comments by identifying the letter designator for each area.”

1. There is a high likelihood in doing so – individuals are going to be identifiable. For example – having the respondent identify their MEPS and Military Service at the beginning of the survey and then identify “Aptitude Testing Personnel” as unprofessional, an individual could be adversely impacted. In addition, this “Aptitude Testing Personnel” could be identified by name.

2. This may be allowable but a DoD Privacy Office (or OMB) needs to approve the collection of this identifiable information and the risk associated with this identification.

Response: Our intent is to collect enough actionable information (may include PII).

OPA: I am requiring a USMEPCOM Privacy Officer (or other DoD Privacy Office if not available in USMEPCOM) confirm that identifiable information produced by a third party via this survey/system can be used to investigate, contact, or otherwise follow-up with either the survey participant or an individual identified by name or other means (e.g., a Buffalo Recruiter).

I have reviewed the PIA and related SORN, as well as spoken to a DHRA Privacy Officer, about the reporting of a potential UCMJ violation by a third party via the survey for use in an investigation. Based on guidance received, there is some concern if this were to occur via the proposed survey. As a very basic example – it’s one thing to indicate via the survey a hotel employee in Buffalo was not efficient or was rude. It would be something entirely different if a Buffalo hotel employee was accused via the survey of sexual harassment or assault. And the PIA may cover any such case – I simply am not an expert in this area and would like a USMEPCOM/DoD Privacy Office to confirm data collected via this system can be used for such purposes. If this is allowable – then the personally identifiable information text is not required.

If so – I would also require that text be included in the introduction, privacy, and/or consent sections of the survey that a participant sees before they choose to continue stating as much. In other words – that completing the survey and information identifying individuals either negatively or positively can be used for follow-up purposes not only with the survey participant but the individual being identified.

You’ve actually done a pretty good job of this in the authority section at the end of the survey. You’ve stated the primary purpose of the survey is assessing programs and improving customer service, which I agree with. And you indicate all comments will be reviewed regardless of identify disclosure, which is good. But you also need to add that comments could be used for follow-up purposes, to include investigations of behavior or performance, if applicable. (5/1/20)

Response: Email attached. USMEPCOM Privacy Officer conducted review; there is no objection to doing an investigation from information learned from these surveys. There is no limitation included in the instructions or notices. Added to the OMB Authority Section, “Information learned from this survey can be used for proper law enforcement or UCMJ authority. OPA: Concur and confirmed addition of new text; thank you for providing. (5/14/20)

7. Collecting Name, Phone, Email is PII; however, Supporting Statement A states “a Privacy Impact Assessment (PIA) is not required for this collection because PII is not being collected electronically.”

1. A DoD Privacy Office or OMB needs to reconcile and approve this discrepancy.

Response: Current PIA provided; Statement A updated. OPA: Confirmed revisions. (5/1/20)

8. Question 10.2. Removed the number “7.” Reworded question to reduce false positive responses and eliminate confusion. Received feedback from multiple MEPS that the applicants are misinterpreting this question.

Current Question: 10.2. The MEPS is committed to maintaining an environment which is free of discrimination and harassment. Do you believe that the services/assistance provided to you while at the MEPS was made without regard to your religion, race, national origin or gender?

Revised Question: 10.2. Do you feel you experienced discrimination and/or harassment based on your religion, race, national origin or gender? OPA: Concur and confirmed revision; goo catch – well done by the team. (5/14/20)

4. Supporting Statement

Part A

1. Minor formatting and grammatical revisions were made to the document; the returned version should be used going forward for remaining coordination activities.

2. Section 8 – See comment in returned version and update as applicable.

Response: Updated; only required to cite 30-day FRN for this submission. OPA: Confirmed revisions. (5/1/20)

3. Section 10 – It states “a Privacy Impact Assessment (PIA) is not required for this collection because PII is not being collected electronically.”

1. By voluntarily collecting Name, Phone, Email at the end of the survey, you are collecting PII so is this statement accurate? A DoD Privacy Office or OMB needs to reconcile and approve this discrepancy.

Response: See answer to #6 above; PIA required. OPA: Confirmed revisions. (5/1/20)

4. Section 15 – See comment in returned version and update as applicable.

Response: Completed. OPA: Confirmed revisions. (5/1/20)

Part B

1. Minor formatting and grammatical revisions were made to the document; the returned version should be used going forward for remaining coordination activities.

2. Section 3 – See comment in returned version and update as applicable.

Response: Completed. OPA: Confirmed revisions. (5/1/20)

4. Communications

   1. See comments in survey instrument section of this document for feedback on Privacy Statement, Introduction Text, and Public Burden Text. OPA: The only change made to these documents were in the Public Burden Text, which looked to be completely removed and is OK; the privacy and introduction text was not. Revise and resubmit based on comments provided in returned version (as well as the first version I sent back). (5/1/20) Apologies. We made the change, but it did not save on the version sent to you. It is now correct. OPA: Confirmed revisions. (5/4/20)

   2. Modified the following sentence in the Authority Section at the end of the survey, “Your name, phone number and e-mail address will be used by USMEPCOM only to send you a response should you request one.” We felt that “should you request one” contradicted the statement in the Introduction. We modified the sentence to read: Your name, phone number and e-mail address will only be used by USMEPCOM to contact you to follow-up on your comments or to ask you for more information about your comments. This is in line with the Introduction. OPA: Concur and confirmed addition of new text; thank you for providing. (5/14/20)

5. Other

   1. N/A.

1