CPQS Sampling Plan

2.2.1

Target Population and Sampling Frame

The target population consists of USPTO continuous customers who have historically filed more
than six patent applications per year. The sample unit will be the continuous customer filing those
applications. These eligible customers will be identified on the file of agency records provided. If
apparent ineligible records are discovered, we will identify those cases and convert the file to an
eligible applicant-level file, from which the sample will be selected. We will use auxiliary information
on the sampling frame, such as the frequency of customer contact, volume of patent activity, and
technology areas to select a representative sample.

2.2.2

Sample Design and Selection

In Fall 2007 Westat staff “refreshed” the sampling frame initially selected for the first 5 waves of
data collection on the CPQS. We have conducted four administrations of the CPQS using that
updated frame and will be able to use it for an additional 2 more waves of data collection. In
refreshing the sampling frame, we took all 10,000 cases from the original sampling frame and
updated eligibility status and contact information. After January 2010, we will need to start over and
select a new sampling frame as information on the cases will have aged too much to be usable for
more than 12 waves of data collection. After the end of the first year on this contract, when
selecting a new sampling frame is necessary, Westat will submit a plan for selection of the new
sampling frame to the USPTO following the sample design criteria set out above and using auxiliary
information on the sampling frame, such as the frequency of customer contact, volume of patent
activity, and technology areas to select a representative sample.
As we have done in the past, Westat will be responsible for the panel design aspects of the sample
design. This includes maintaining the customer panel in terms of representativeness, adding new
members to account for attritions, adding new rotation groups, retaining nonrespondents, and
tracing lost members to establish eligibility and maintain high response rates. Table 2-1 is an
example of the rotation pattern used in the first 7 waves of data collection of the CPQS beginning in
Fall 2006. This same pattern of rotation will be continued for upcoming waves of data collection.

______________________________________________________________________________________________________________________________________
Use or disclosure of data contained on this page is subject
Westat
to the restriction on the title page of this proposal or quotation.
4
WPN # 09-523

Table 2-1.

Wave
1
2
3
4
5
6

Example of rotation plan with 50% overlap between successive waves

1
X

2
X
X

3
X
X

4

Group

X
X

5

X
X

6

7

X
X

X

Although the USPTO does not have a predetermined level of precision desired for the survey
estimates, we understand the primary importance of detecting significant differences in wave-towave changes. Table 2-2 provides the minimum detectable absolute differences for domain sample
size yields of completes for a two-tailed t-test with 80 percent power and a 0.05 significance level.
For example, for a domain-level sample size yielding 500 completes, the difference between two
waves needs to be greater than or equal to 0.073 in order to be declared statistically significant at the
0.05 level.
Table 2-2.

Minimum detectable absolute differences

Sample size yields (completes) for each wave
1,000
750
500
250
a

Minimum detectable absolute
differencea
0.051
0.059
0.073
0.103

The calculations assume a unit correlation coefficient between waves equal to 0.6.

The minimum detectable absolute differences were computed based on simple random sampling.
Even though there will be some marginal variance reduction benefits from our proposed design, the
numbers in the above table can be used as a guide for what to expect from the resulting precision
estimates.

2.2.3

Maintaining a Representative Customer Panel

Several changes will occur to membership of the target population of continuous customers. For
instance, some will become ineligible, while others will become newly eligible. Some panel members
will fail to participate in the survey. We are aware that if we do not maintain a customer panel that is
representative of the target population, the possibility of biased survey estimates increases. Thus we
propose the following activities to maintain the panel and reduce nonresponse.
______________________________________________________________________________________________________________________________________
Use or disclosure of data contained on this page is subject
Westat
to the restriction on the title page of this proposal or quotation.
5
WPN # 09-523

Newly eligibles/new rotation groups. To account for attrition, Westat will collaborate with the
USPTO on the frequency of updates to the list of applications. Each year, a revised list of
applications can be reviewed and evaluated for applicants newly eligible as continuous customers.
The newly eligible continuous customers can be sampled and allocated to the remaining random
rotation groups. Depending on the information provided about the existence of a revised list, we
can modify this approach if necessary.
Promoting survey cooperation. Throughout the study, we will be using effective procedures and
current best methods for promoting survey cooperation 3.These goals are reflected in our discussion
of the survey instrument as well as in our descriptions of our proposed contact procedures and
other survey materials. We also describe our approach to converting initial refusals (see Section 2.5,
the Data Collection section of this proposal).
Retaining/tracing wave nonrespondents. Despite our best efforts to promote survey
cooperation, we anticipate survey nonresponse because the population being studied is extremely
busy. We plan to follow up nonrespondents and describe our proposed followup activities and
relatively low-cost tracing activities in Section 2.5, the Data Collection section of this proposal. If
those activities prove to be insufficient and higher cost followup activities are needed to reach the
targeted number of completes for each wave, we will meet with the USPTO to discuss options.
Please note, however, that in Waves 6 and 7 we conducted a follow-up study with nonrespondents
and discovered that with respect to their ratings of overall patent examination quality, the
nonrespondents did not exhibit a more negative perception of overall examination quality than
respondents did.
In addition, we have budgeted for, and will conduct, standard nonresponse adjustments during the
weighting process. Those adjustments are designed to reduce potential bias resulting from the failure
of some selected customers in the sample to respond to the survey (our proposed weighting process
is described in Section 2.6, the Data Processing section).

3

Biemer, P., & Lyberg, P. (2003). Introduction to survey quality. New York: Wiley.

______________________________________________________________________________________________________________________________________
Use or disclosure of data contained on this page is subject
Westat
to the restriction on the title page of this proposal or quotation.
6
WPN # 09-523