Sarah Warren <stwarren@ncsu.edu> 
04/19/2009 10:36 AM	To
	Sharon Nygaard-Scott <snygaardscott@fs.fed.us>
	cc
	
	bcc
	
	Subject
	Re: Stewardship Programmatic Monitoring - May I please call you, at your 
convenience, regarding the survey renewal?

	History:	
		This message has been replied to.

Greetings,
Here are a few comments on some of the OMB categories for comment.  You'll 
notice that I can only comment on a few of these categories, and not in 
the order presented.
Clarity of instructions.  The instructions are written in complex 
language, with complex sentence structure and use of potentially 
unfamiliar terms (e.g., stratified random sample).  Although it may be 
reasonable to assume that most contractors will understand the language, I 
would recommend that this portion of the survey be edited and set to a 
high school (rather than college) level on the Flesch-Kincaid reading 
level and reading ease scales.  This is particularly important because the 
survey is to be sent by email.  
Currently the instructions would require four years of college to be 
understood.  Flesch-Kincaid reading ease of 29 and about junior in college 
reading grade.
attached example has raised F-K reading ease to 40.9 (@11th grade), but 
still needs work. No time, unfortunately, to really revise it.
Clarity of questions.  
Several of the initial questions would be defined as "leading questions" 
in standard survey research. 
The questions (especially those with tables), seem very complicated.  Is 
there an intent that the 'sample' being sent with the email serve as the 
'exact' survey?  Can you provide the respondent an option to prefill the 
answer boxes?  
Has this survey been pre-tested by telephone on a sample of likely 
respondents?  
I am terribly sorry not to have been able to spend more time on this.  
Kind regards, Sarah Warren
Attachment:  somewhat revised instructions with attention to increasing 
reading ease

Sharon Nygaard-Scott wrote: 

Dear Dr. Warren: 

Per my voice mail message, may I please call you at your convenience 
regarding the renewal of the stewardship programmatic monitoring survey 
instrument? 
I work for the US Forest Service, Forest Management Group, Washington DC, 
with responsibilities including Stewardship Contracting. 
I noticed that you are a member of the SE Regional Stewardship Monitoring 
and Evaluation Team, per the Pinchot Institute's 2008 report to the Forest 
Service. 
Every three years, we have to renew the survey instrument, per the Office 
of Management and Budget (OMB). This includes consulting with 3-9 
individuals outside the Federal Government. The comments received become 
part of the approval process.  They are written into the Supporting 
Statement documentation for the OMB. 
The Pinchot Institute, in working with their partners, have suggested the 
following changes be made to the original survey (and are incorporated 
into the proposed revision below): 
Minor word changes for clarity 
Minor format changes, for clarity and/or analysis purposes 
Addition of response categories, to some questions, based on feedback from 
interviewers and/or Maureen McDonough (survey developer), along with 
retention of the "other" category for discussion with the interviewee.

OMB's suggested categories for comment include, but are not limited to, 
the following (and some may not apply to this particular survey): 
Reasonableness of time allowed  [PIC is estimating 30-45 minutes] 
The frequency of collection [annual basis] 
The clarity of instructions 
Disclosure 
The availability of data 
The data elements to be recorded, disclosed, or reported 
Record keeping responsibilities, and 
Reporting format (if any)

The original survey instrument is as follows: 

The proposed updated survey instrument is as follows: 

I would be most happy to discuss this with you including any comments you 
may wish to share, at your convenience. I can be reached at 202-205-1766 
or snygaardscott@fs.fed.us. 
I appreciate your valuable time and thoughts! 

Sharon 

Sharon Nygaard-Scott
Sale Preparation & Stewardship Contracting Policy
Forest Management, WO -Yates (3 SW)
snygaardscott@fs.fed.us
(202) 205-1766 - Office
(202) 205-1045 - Fax 

-- 
Sarah T. Warren
Associate Professor
Director of Graduate Programs
3118 Jordan Hall, Box 8008
Department of Forestry and Environmental Resources
North Carolina State University, Raleigh, NC 27695-8008
Telephone: 919-515-7996
Email: sarah_warren@ncsu.edu