FLE Post-Scenario Survey TRANSTAC Evaluation
Foreign Language Expert Post-Scenario Survey
Please supply responses that best match your experience for the scenario you just completed.
1. What did you like most about this system, or what was most helpful and useful?
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
2. What did you like least about this system?
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
3. What would you change about this system?
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
4. I found the system easy to understand in this interaction.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
5. What the system said made sense to me.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
6. The system made it easy to have this interaction.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
7. In this interaction, it was easy to get the information I needed.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
8. I knew what I could say or do at each point of this interaction.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
9. The system worked the way I expected it to in this interaction.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
10. The pronunciation of the <focus-language> was clear and understandable.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
11. The <focus-language> words were put together in a way that was coherent and comprehensible.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
12. When the system didn't understand me, it was easy to correct.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
13. How widely will the <focus-language> spoken by the system be understood by other <focus-language> speakers?
0 1 2 3 4 5
N/A Almost None Few Many Most Nearly All
14. How easily will other <focus-language> speakers be able to speak in a way that is understood by the system?
0 1 2 3 4 5
N/A Nearly With Some With Little Effortlessly
Impossible Difficulty Effort Effort
15. How appropriate is the <focus-language> spoken by the system to the situation(s) simulated in this session?
0 1 2 3 4 5
N/A Completely Somewhat Minimally Rather Completely
Inappropriate Inappropriate Acceptable Appropriate Appropriate
16. Please include, if necessary, additional comments regarding the system's use of <focus-language>:
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
17. How much of the English could you hear during the dialog?
0 1 2 3 4 5
N/A None A Little Some A Lot All
18. If you heard any of the English, what influence did it have on you?
0 1 2 3 4 5
N/A None Not Much A Little Some A Lot
19. Please choose one of the following options to describe your experience in the interaction you just completed.
0 1 2 3 4 5
N/A Many Some A Few Very Few No
Problems Problems Problems Problems Problems
20. If you experienced problems in the last dialog-interaction as reported above, please indicate the cause(s) of these problems (check all that apply).
O Scenario content
O Scenario flow
O Translation system problems
O Human partner problem(s)
O Other (please describe): _________________________________________________________
__________________________________________________________________________________________________________________________________________________________________________________________
21. Towards the end of the conversation, I was interacting with the system in a different manner than at the beginning.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
22. If you changed the way you interacted with the system, it involved the following, please check all that apply.
O Speaking more slowly
O Speaking more quickly
O Speaking more clearly
O Speaking more loudly
O Speaking more softly
O Waiting more time to speak once my “speech button” is pressed
O Waiting less time to speak once my “speech button” is pressed
O Using simpler and/or shorter sentences/phrases
O Using longer and/or more complex sentences/phrases
O Speaking more “naturally” for me
O Other (please describe): _________________________________________________________
__________________________________________________________________________________________________________________________________________________________________________________________
23. If the translation system seemed to have problems with certain words or phrases, please list any you can recall.
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
24. How confident were you in the system's ability to help you communicate effectively overall?
1 2 3 4 5
not somewhat marginally rather strongly
confident doubtful confident confident confident
25. I would use this system in the field in its current state of functionality.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
26. Based on my experience in this interaction, I would recommend this system for future similar interactions.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
27. If the system had worked as intended in this interaction, I would recommend this system for future similar interactions.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
28. What would you suggest to improve the training you received before starting the interactions?
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
29. The system instructions provided a clear understanding of how to use the system.
0 1 2 3 4 5
N/A Strongly Disagree Disagree Neutral Agree Strongly Agree
30. What situations can you imagine that might best match this system's capabilities?
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
31. Please provide any additional comments here.
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
NOTE: This survey contains collection of information requirements subject to the Paperwork Reduction Act. Notwithstanding any other provision of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to comply with, a collection of information subject to the requirements of the Paperwork Reduction Act, unless that collection of information displays a currently valid OMB control number. The estimated response time for this survey is 6.2 minutes. The response time includes the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information." OMB Number: 0693-0043 Expiration: 10/31/2012.
Thank you for your participation.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | FLE # __________ |
| Author | Sherri L. Condon |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |