Warren NCSU Response

Sarah Warren <[email protected]>

04/19/2009 10:36 AM To

Sharon Nygaard-Scott <[email protected]>

cc

bcc

Subject

Re: Stewardship Programmatic Monitoring - May I please call you, at your

convenience, regarding the survey renewal?

History:

This message has been replied to.

Greetings,

Here are a few comments on some of the OMB categories for comment. You'll

notice that I can only comment on a few of these categories, and not in

the order presented.

Clarity of instructions. The instructions are written in complex

language, with complex sentence structure and use of potentially

unfamiliar terms (e.g., stratified random sample). Although it may be

reasonable to assume that most contractors will understand the language, I

would recommend that this portion of the survey be edited and set to a

high school (rather than college) level on the Flesch-Kincaid reading

level and reading ease scales. This is particularly important because the

survey is to be sent by email.

Currently the instructions would require four years of college to be

understood. Flesch-Kincaid reading ease of 29 and about junior in college

reading grade.

attached example has raised F-K reading ease to 40.9 (@11th grade), but

still needs work. No time, unfortunately, to really revise it.

Clarity of questions.

Several of the initial questions would be defined as "leading questions"

in standard survey research.

The questions (especially those with tables), seem very complicated. Is

there an intent that the 'sample' being sent with the email serve as the

'exact' survey? Can you provide the respondent an option to prefill the

answer boxes?

Has this survey been pre-tested by telephone on a sample of likely

respondents?

I am terribly sorry not to have been able to spend more time on this.

Kind regards, Sarah Warren

Attachment: somewhat revised instructions with attention to increasing

reading ease

Sharon Nygaard-Scott wrote:

Dear Dr. Warren:

Per my voice mail message, may I please call you at your convenience

regarding the renewal of the stewardship programmatic monitoring survey

instrument?

I work for the US Forest Service, Forest Management Group, Washington DC,

with responsibilities including Stewardship Contracting.

I noticed that you are a member of the SE Regional Stewardship Monitoring

and Evaluation Team, per the Pinchot Institute's 2008 report to the Forest

Service.

Every three years, we have to renew the survey instrument, per the Office

of Management and Budget (OMB). This includes consulting with 3-9

individuals outside the Federal Government. The comments received become

part of the approval process. They are written into the Supporting

Statement documentation for the OMB.

The Pinchot Institute, in working with their partners, have suggested the

following changes be made to the original survey (and are incorporated

into the proposed revision below):

Minor word changes for clarity

Minor format changes, for clarity and/or analysis purposes

Addition of response categories, to some questions, based on feedback from

interviewers and/or Maureen McDonough (survey developer), along with

retention of the "other" category for discussion with the interviewee.

OMB's suggested categories for comment include, but are not limited to,

the following (and some may not apply to this particular survey):

Reasonableness of time allowed [PIC is estimating 30-45 minutes]

The frequency of collection [annual basis]

The clarity of instructions

Disclosure

The availability of data

The data elements to be recorded, disclosed, or reported

Record keeping responsibilities, and

Reporting format (if any)

The original survey instrument is as follows:

The proposed updated survey instrument is as follows:

I would be most happy to discuss this with you including any comments you

may wish to share, at your convenience. I can be reached at 202-205-1766

or [email protected].

I appreciate your valuable time and thoughts!

Sharon

Sharon Nygaard-Scott

Sale Preparation & Stewardship Contracting Policy

Forest Management, WO -Yates (3 SW)

[email protected]

(202) 205-1766 - Office

(202) 205-1045 - Fax

--

Sarah T. Warren

Associate Professor

Director of Graduate Programs

3118 Jordan Hall, Box 8008

Department of Forestry and Environmental Resources

North Carolina State University, Raleigh, NC 27695-8008

Telephone: 919-515-7996

Email: [email protected]