Baldridge National Quality Program Research: Phase II Online Survey

Baldrige Brand and Customer Requirements Research

(Draft) Online Survey Questionnaire (Phase II: Concept Testing)

E-MAIL INVITATION FOR BALDRIGE-PROVIDED SAMPLE LIST (BRANDED)

Dear X:

Aeffect, Inc. is an independent marketing research firm that is conducting a brief survey on behalf of the Baldrige Performance Excellence Program, formerly known as the Baldrige National Quality Program. The survey focuses on quality and performance improvement initiatives in companies and organizations like yours, with the purpose of the survey to gain feedback in order to improve Baldrige program offerings and services. We are not selling anything; we are simply interested in the past experiences and opinions of top executives like yourself on this very important topic. The survey will take about 10 to 15 minutes to complete. Your contact information and survey answers will remain confidential and anonymous. You can take the survey at: www.aeffect.com. We thank you in advance for your valuable input.

E-MAIL INVITATION FOR AEFFECT PURCHASED SAMPLE LIST (UNBRANDED)

Dear X:

Aeffect, Inc. is an independent marketing research firm that is conducting a brief survey on quality and performance improvement initiatives in companies and organizations like yours. We are not selling anything; we are simply interested in the past experiences and opinions of top executives like yourself on this very important topic. The survey will take about 10 to 15 minutes to complete. Your contact information and survey answers will remain confidential and anonymous. You can take the survey at: www.aeffect.com. We thank you in advance for your valuable input.

BURDEN STATEMENT

NOTE: This questionnaire contains collection of information requirements subject to the Paperwork Reduction Act (PRA). Notwithstanding any other provisions of the law, no person is required to respond to, nor shall any person be subject to penalty for failure to comply with, a collection of information subject to the requirements of the PRA, unless that collection of information displays a currently valid OMB Control Number.  The estimated response time for this questionnaire is 10 to 15 minutes.  The response time includes the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this estimate or any other aspects of this collection of information, including suggestions for reducing the length of this questionnaire, to the National Institute of Standards and Technology, Attn., POINT OF CONTACT, EMAIL ADDRESS, PHONE NUMBER. The OMB Control No. is 0693-0031, which expires on 02/29/2012.

Respondents	Number of Respondents	Responses Per Respondents	Average Hours Per Response	Response Burden in Hours
Total	400	1	15/60	100

SCREENING QUESTIONS AND AWARENESS OF BALDRIGE

1. Are you responsible or do you share responsibility for making decisions regarding the improvement of your organization’s performance (e.g., improving leadership/management practices, strategic planning, customer satisfaction, knowledge management, employee retention, process management, performance results, etc.)?

A. Yes	1
B. No	2	THANK AND END
C. Don’t Know	DK

2. What one category best describes your organization?

Business (n=100)		QUOTA
Manufacturing	1	(n=50)
Service	2	(n=50)
Education (n=100)
Education (K-12)	3	(n=50)
Education (Higher Education)	4	(n=50)
Health Care (n=100)
Health Care	5	(n=100)
Government/Non-Profit (n=100)
Government	6	(n=50)
Not-for-profit	7	(n=50)
Definitely not one of these categories	8	TERMINATE
Don’t Know	-1

[If Manufacturing, Service, Government, or Non-Profit in Q2 (CODES 1, 2, 6, OR 7), SKIP TO Q4.]

3. Is your organization for-profit or not-for-profit? [OBTAIN NATURAL FALLOUT. IF DIFFICULTY MEETING NOT-FOR-PROFIT QUOTA IN Q2, MAY CLASSIFY AS NON-PROFIT IF CODE 2 IN Q3.]

For-profit	1
Not-for-profit	2

4. Approximately how many employees does your organization employ at all locations?

1-9 employees	1	[SMALL] If Codes 1 or 2 in Q2, Obtain Quota (n=33)
10-49 employees	2
50-99 employees	3
100-299 employees	4	[MEDIUM] If Codes 1 or 2 in Q2, Obtain Quota (n=33)
300-499 employees	5
500-999 employees	6	[LARGE] If Codes 1 or 2 in Q2, Obtain Quota (n=33)
1,000-4,999 employees	7
5,000 employees or more	8
Don’t Know	-1	THANK AND END

5. What quality or performance excellence organizations, programs, approaches or awards are you aware of? Please enter your response below. You may enter as many as come to mind.

__________________________________________________________________________

6. Which of the following quality or performance excellence organizations, programs, approaches or awards have you heard of? You may have already mentioned some of these in the previous question, but choose all that apply. [ROTATE]

	Aware
A. The Baldrige Performance Excellence Program, formerly known as the Baldrige National Quality Program	1
B. Malcolm Baldrige National Quality Award	2
C. Balanced Scorecard	3
D. The Deming Award	4
E. State or Local Quality Awards	5
F. Baldrige-based Consultants	6
G. The International Organization for Standardization (ISO)	7
H. The Joint Commission on the Accreditation of Healthcare Organizations (JCAHO)	8
I. Lean	9
J. J.D. Power and Associates	10
K. Six Sigma	11
L. ISO 9000 or other ISO standard	12
M. Shingo Prize	13
N. None	18
O. Don’t Know	19

[ASK Q7 ONLY IF RESPONDENT ANSWERS CODES 1 OR 2 (AWARE OF BALDRIGE) IN Q6. OTHERWISE, SKIP TO Q9]

IMPORTANT NOTE: THE BALDRIGE NATIONAL QUALITY PROGRAM CHANGED ITS NAME TO THE BALDRIGE PERFORMANCE EXCELLENCE PROGRAM IN OCTOBER OF 2010. FROM THIS POINT FORWARD IN THE SURVEY, IT WILL BE REFERRED TO BY ITS NEW NAME.

7. Which of the following Baldrige Performance Excellence Program products or services, if any, have you heard of? Choose all that apply. [ROTATE]

A.	Criteria for Performance Excellence	1
B.	Education Criteria for Performance Excellence	2
C.	Health Care Criteria for Performance Excellence	3
D.	easy Insight Self-Assessment Tool	4
E.	Are We Making Progress Questionnaires	5
F.	Quest for Excellence Conference	6
G.	Baldrige Regional Conferences	7
H.	Baldrige Board of Examiners	8
I.	Award Process and Feedback	9
J.	Baldrige Case Studies	10
K.	Award Recipient Videos	11
L.	Award Recipient Profiles and Application Summaries	12
M.	None of the above	13

[IF NOT AWARE OF ANY PRODUCTS OR SERVICES IN Q7 (CODE 13), CODE RESPONDENT AS “NON-CUSTOMER” (QUOTA n=200) AND SKIP TO Q9.]

8. Please indicate when, if ever, your company or organization…? [ROTATE]

		In past year	About 1-2 years ago	About 2-4 years ago	More than 4 years ago	Never	Don’t Know
A.	Applied for the Malcolm Baldrige National Quality Award	1	2	3	4	5	-1
B.	Received a Malcolm Baldrige National Quality Award	1	2	3	4	5	-1
C.	Downloaded or used self-assessment materials/questionnaires from the Baldrige website	1	2	3	4	5	-1
D.	Downloaded or read through the Baldrige Criteria for Performance Excellence from the web site	1	2	3	4	5	-1
E.	Implemented some or all of the Baldrige Criteria for Performance Excellence	1	2	3	4	5	-1
F.	Had an employee serve as an Examiner for the Malcolm Baldrige National Quality Award	1	2	3	4	5	-1
G.	Attended a local meeting associated with the Malcolm Baldrige National Quality Award or Baldrige Performance Excellence Program	1	2	3	4	5	-1
H.	Attended a Quest for Excellence Conference (Official Conference of the Malcolm Baldrige National Quality Award) or a Baldrige Regional Conference	1	2	3	4	5	-1

IF CODES 1-2 FOR ANY ITEM IN Q8, CODE RESPONDENT AS “CURRENT CUSTOMER.”

IF CODES 3-4 FOR ANY ITEM IN Q8 BUT NO CODES 1-2, CODE RESPONDENT AS “LAPSED CUSTOMER.”

QUOTA n=200 FOR TOTAL OF “CURRENT CUSTOMERS” AND “LAPSED CUSTOMERS.”

IF CODES 5 OR -1 FOR ALL ITEMS IN Q8, CODE RESPONDENT AS “NON-CUSTOMER” (QUOTA n=200).

QUESTIONS FOR ALL RESPONDENTS

9. When it comes to improving your organization’s performance, which of the following are your greatest needs? Choose all that apply. [ROTATE]

10. Of these needs [CHOSEN IN Q9], please rank the top 3 in order of importance, with 1 being most important, 2 being second most important, and 3 being third most important. [IF ONLY PICK ONE, SKIP TO Q11. IF ONLY PICK TWO RESPONSES, THEY ONLY RANK TWO.

		Q9 Need	Q10 Rank
A.	Improved information management	1
B.	Improved workforce recruitment and retention	2
C.	Stronger leadership	3
D.	Strategic planning	4
E.	Customer engagement and loyalty	5
F.	Knowledge management	6
G.	Workforce focus (employee/faculty/staff)	7
H.	Process management and improvement	8
I.	Organizational performance measurement and analysis	9
J.	Better quality employees/staff/faculty	10
K.	Creation of internal performance assessment processes	11
L.	Organizational sustainability	12
M.	Other (Specify)_____________________	13

11. Which of the following quality or performance excellence activities, if any, has your company or organization been involved with or used in the past few years? Please choose all that apply.

My company or organization has… [ROTATE]

A.	Applied for a local or state quality/excellence award	1	IF NO TO CODES 1-4 (ALL 4), SKIP TO Q.13
B.	Applied for a quality/excellence award specific to my industry	2
C.	Applied for another type of quality/excellence award	3
D.	Joined a local or state quality/excellence program	4
E.	Conducted an internal award process or self-assessment	5
F.	Used Lean principles to improve organization performance	6
G.	Used Six Sigma principles to improve organization performance	7
H.	Used ISO principles to improve organization performance	8
I.	Used Baldrige Criteria for Performance Excellence to improve organization performance	9
J.	Used other principles/criteria to improve organization performance (Please Specify) _______	10
K.	Not been involved with any quality/excellence programs or tools	11

12. What was your organization’s primary goal in applying for an award or joining a quality or excellence program? Please choose one.

A.	Feedback	1
B.	Recognition	2
C.	To improve overall business performance	3
D.	To validate progress on improvements	4
E.	Other (Specify)______	5
F.	Don’t Know	-1

13. How familiar are you with each of the following programs or performance principles? [ROTATE]

		Not At All Familiar	Not Very Familiar	Somewhat Familiar	Very Familiar	Don’t Know
A.	Baldrige Performance Excellence Program	1	2	3	4	-1
B.	ISO	1	2	3	4	-1
C.	Lean	1	2	3	4	-1
D.	Six Sigma	1	2	3	4	-1
E.	Balanced Scorecard	1	2	3	4	-1
F.	Shingo Prize	1	2	3	4	-1
G.	The Joint Commission on the Accreditation of Healthcare Organizations (JCAHO)	1	2	3	4	-1

14. [ASK FOR BRANDS MENTIONED IN Q6] Which of these quality or performance excellence organizations, programs, approaches or awards has your organization ever used, participated in, or competed for? Record any other similar organizations, programs, approaches or awards that you are aware of that are not listed and answer the question for them as well. Choose all that apply. [ROTATE]

15. [ASK FOR BRANDS MENTIONED IN Q14] Of these, which quality or performance excellence awards have your organization received in the past few years or from which organizations has your organization received such an award? Choose all that apply.

16. [ASK FOR BRANDS MENTIONED IN Q6] Of all of the quality or performance excellence awards listed or that you entered, which one of these awards is most valued in your industry today? Select one award only.

	Q14: Used, participated in, competed for	Q15: Awards won	Q16: Most Valued Award
A. The Baldrige Performance Excellence Program	1	1	1
B. Malcolm Baldrige National Quality Award	2	2	2
C. Balanced Scorecard	3	3	3
D. The Deming Award	4	4	4
E. State or Local Quality Awards	5	5	5
F. Baldrige-based Consultants	6	6	6
G. The International Organization for Standardization (ISO)	7	7	7
H. The Joint Commission on the Accreditation of Healthcare Organizations (JCAHO)	8	8	8
I. Lean	9	9	9
J. J.D. Power and Associates	10	10	10
K. Six Sigma	11	11	11
L. ISO 9000 or other ISO standard	12	12	12
M. Shingo Prize	13	13	13
N. Other (specify: _____________________)	14	14	14
O. Other (specify: _____________________)	15	15	15
P. Other (specify: _____________________)	16	16	16
Q. Other (specify: _____________________)	17	17	17
R. None	18	18	18
S. Don’t Know	19	19	19

[ONLY ASK Q17 IF “VERY FAMILIAR”/”SOMEWHAT FAMILIAR”/”NOT VERY FAMILIAR” WITH BALDRIGE (CODES 2, 3, AND 4 IN Q13). OTHERWISE, SKIP TO Q18.]

17. When you think of the Baldrige Performance Excellence Program and the Malcolm Baldrige National Quality Award, what comes to mind? Please type in your answer below and be as detailed as possible.

18. How strongly do you agree or disagree with each of the following statements about the quality/performance excellence programs or performance principles below?

RESPONDENT ANSWERS FOR ONLY THOSE BRANDS THEY ARE VERY/SOMEWHAT FAMILIAR WITH (CODES 3 AND 4 IN Q13). RESPONDENT ANSWERS Q18 SERIES FOR A SINGLE BRAND, EITHER BALDRIGE (IF PARTICIPANT IS FAMILIAR WITH IT) OR ANOTHER BRAND. ROTATE LIST.

[Brand List: Baldrige National Quality Program, ISO, Lean, Six Sigma, Balanced Scorecard]

	[BRAND …]	Strongly Strongly Disagree Agree										Don’t Know
A.	Is relevant to my organization	1	2	3	4	5	6	7	8	9	10		-1
b.	Is worth the effort to implement	1	2	3	4	5	6	7	8	9	10		-1
c.	If implemented, demonstrates an organization is committed to quality and performance excellence	1	2	3	4	5	6	7	8	9	10		-1
d.	Is as important today as it was when it was created	1	2	3	4	5	6	7	8	9	10		-1
E.	Is motivating	1	2	3	4	5	6	7	8	9	10		-1
f.	Is important to management	1	2	3	4	5	6	7	8	9	10		-1
G.	Is important to customers and stakeholders	1	2	3	4	5	6	7	8	9	10		-1
h.	Is important to employees	1	2	3	4	5	6	7	8	9	10		-1
i.	Is valued around the world	1	2	3	4	5	6	7	8	9	10		-1
j.	Requires a great deal of involvement/commitment	1	2	3	4	5	6	7	8	9	10		-1

[IF NOT “VERY FAMILIAR” OR “NOT AT ALL FAMILIAR” WITH BALDRIGE (CODES 3 OR 4 IN Q13), SKIP TO CONCEPTS (Q25). OTHERWISE CONTINUE.]

BALDRIGE PERFORMANCE EXCELLENCE PROGRAM AND MALCOLM BALDRIGE NATIONAL QUALITY AWARD

The next series of questions focuses on the Baldrige Performance Excellence Program and everything it offers, including the Malcolm Baldrige National Quality Award. Please answer these questions the best you can, regardless of your familiarity with the Program and Award, as all perceptions are valuable.

19. Overall, how would you rate or grade the reputation of the Baldrige Performance Excellence Program and the Malcolm Baldrige National Quality Award? Would you say it is…?

A.	Excellent	5
B.	Very good	4
C.	Good	3
D.	Fair	2
E.	Poor	1
F.	Don’t know	-1

20. How well does each statement describe the Baldrige Performance Excellence Program? [ROTATE]

		Does Not Describe Describes At All Very Well					Don’t Know
a.	A performance feedback program	1	2	3	4	5	-1
B.	A group of business consultants	1	2	3	4	5	-1
C.	A government watchdog group	1	2	3	4	5	-1
D.	A performance training program	1	2	3	4	5	-1
E.	A system for improving the output of an organization	1	2	3	4	5	-1
F.	A tool for assessing the current state and success of an organization	1	2	3	4	5	-1
G.	A performance education program	1	2	3	4	5	-1
H.	A national award for quality or performance excellence	1	2	3	4	5	-1
I.	A program that offers guidelines and criteria for making business improvements	1	2	3	4	5	-1
J.	A government-supported group that provides financial support to businesses	1	2	3	4	5	-1
K.	A program that enhances the competitiveness, quality, and productivity of U.S. organizations	1	2	3	4	5	-1

21. How strongly do you agree or disagree with each of the following statements about the Malcolm Baldrige National Quality Award? [ROTATE]

		Strongly Strongly Disagree Agree					Don’t Know
A.	The Baldrige Award is a prestigious recognition of high quality performance	1	2	3	4	5	-1
B.	There are substantial organizational benefits from receiving the Baldrige Award	1	2	3	4	5	-1
C.	The costs for applying for the Baldrige Award outweigh its potential benefits	1	2	3	4	5	-1
D.	Applying for the Baldrige Award is a very complicated process	1	2	3	4	5	-1
E.	I would want my organization to invest in becoming a recipient of this award	1	2	3	4	5	-1
F.	I associate a recipient of this award with leadership	1	2	3	4	5	-1
G.	I associate a recipient of this award with efficiency	1	2	3	4	5	-1
H.	I associate a recipient of this award with spending a lot of time and money on the effort	1	2	3	4	5	-1
I.	I associate a recipient of this award with success	1	2	3	4	5	-1

22. What steps have you taken in terms of applying for the Malcolm Baldrige National Quality Award in the future? Choose all that apply.

a.	We have talked about it	1
B.	We have investigated it	2
C.	We are still considering whether or not to apply in the future	3
D.	We are currently involved in the application process	4
E.	We applied this year	5
F.	We have decided not to apply	6
G.	No discussions at all/no steps taken	7

23. Regardless of whether or not it is your decision, how likely is your organization to apply for the Malcolm Baldrige National Quality Award in the next few years?

a.	Definitely will	5	GO TO Q24A
B.	Probably will	4
C.	May or may not	3	GO TO Q24B
D.	Probably will not	2
E.	Definitely will not	1

24. A. Why do you feel that way? Please enter in your answer below and be as detailed as possible.

____________________________________________________________ (SKIP TO Q25)

B. Why do you feel that way? Please enter in your answer below and be as detailed as possible.

________________________________________________________________________

CONCEPT TESTING

You are now going to see a few ideas for how the Baldrige Performance Excellence Program might describe itself. This is not an advertisement or copy for an ad. It is merely a way of conveying how the organization might describe itself to current and prospective customers and users of its services. Please read the concept and then you will be asked a series of questions about the concept. You will be able to refer to the concept while you are answering the questions, so there is no need to copy down the concept.

EXPOSE PARTICIPANTS TO EACH CONCEPT (CONCEPTS 1, 2, 3, and 4). ROTATE CONCEPTS. ASK FOR EACH:

25. What is your overall reaction to this concept?

Very Positive	5
Somewhat Positive	4
Neutral	3
Somewhat Negative	2
Very Negative	1
Don’t Know	-1

Please indicate how much you agree or disagree with the following statements about the concept or that are based on the concept.

Strongly Agree	5
Somewhat Agree	4
Neither	3
Somewhat Disagree	2
Strongly Disagree	1

26. This organization and what it offers is unique compared to what else is available.

27. This organization and what it offers is relevant to my company/organization and its needs.

28. Coming from Baldrige, this concept is believable.

29. This concept is clear and understandable.

30. I anticipate that this organization would meet the needs of my company/organization.

31. I am interested in obtaining materials and guidance related to quality and performance excellence from this organization.

32. I would recommend this organization and what it can offer to a colleague in my company or someone I know in another organization.

33. I am more interested in the Baldrige Performance Excellence Program based on this concept.

34. Here are all of the concepts you just saw. Please indicate your preference for the concepts by ranking them from “1” to “4,” where “1” is your most preferred concept, “2” is your second most preferred concept, and so on through “4” for your least preferred concept.

Concept	Ranking
Concept A: “vbvbvbvb”	_____
Concept B: “vbvbvbvb”	_____
Concept C: “vbvbvbvb”	_____
Concept D: “vbvbvbvb”	_____

35. Again, here are all of the concepts you just saw. Now we would like to know how strong your preference is for each of these concepts. Please divide 100 points between the four concepts. The more points you give to a concept, the stronger your preference for that concept. You can give all 100 points to a single concept, you can give 25 points to each of the four concepts, or you can distribute the points in any manner that you wish. Just remember that the total has to add up to 100 points.

Concept A: “vbvbvbvb”	_____
Concept B: “vbvbvbvb”	_____
Concept C: “vbvbvbvb”	_____
Concept D: “vbvbvbvb”	_____
Total	100

DEMOGRAPHICS

36. Which of the following best describes the role you have at your organization?

Owner	1
Senior Officer (President, VP C-Level)	2
Director	3
Manager	4
Supervisor	5
Administrator	6
Other (Specify: ______________________)	7

37. How long have you been in your current position?

Less than one year	1
1 – 2 years	2
3 – 5 years	3
6 – 10 years	4
More than 10 years	5

38. Which of the following best describes your department or area of work?

Production	1
Human Resources	2
Finance	3
Marketing	4
Quality Control	5
Management	6
Other (Specify)	7

39. What current sources do you use for gathering information on quality and performance excellence? Please choose all that apply. [ROTATE]

A.	Industry conferences	1
B.	Industry publications (SPECIFY: __________________________)	2
C.	Web sites (SPECIFY: __________________________)	3
D.	Books by experts in the field (SPECIFY: ________________________)	4
E.	Internal sources (wikis, white papers, etc.)	5
F.	Senior leadership	6
G.	Other (SPECIFY: _______________________________)	7
H.	None	8

40. So that we can track the geographic representation of participants, please indicate the 5-digit zip code for the location of your business/organization.

__ __ __ __ __

[THANK AND END]

NIST – Baldrige Survey (Phase II) – February 11, 2011