Elder OMB Submission Part B_032612

EJRP OMB Clearance

Supporting Statement

Information Collection Request

Supporting Statement for the Elder Justice Roadmap Project

Part B

Project Officer:

Andy Mao, JD

Elder Justice and Nursing Home Initiative

Fraud Section

Commercial Litigation Branch

Civil Division

Department of Justice

601 D Street, NW, Room 9548
Washington, DC 20530
Phone: (202) 616-0539
Fax: (202) 514-0280
E-mail: [email protected]

(submitted by Laurie Feinberg, M.D., M.P.H, COTR)

Revised March 6, 2012

Section B: Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods…………………………………………………………2

B.2. Procedures for the Collection of Information…………………………………………………….........3

B.3. Methods to Maximize Response Rates and Deal with Nonresponse………………………………….4

B.4. Test of Procedures or Methods to be Undertaken……………………………………………………..6

B.5. Individuals Consulted on Statistical Aspects and Individuals Collection and/or Analyzing Data……8

References……………………………………………………………………………...…………………..9

B. Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods

The sampling and data collection design for this project is not intended to generate estimates, but rather to gain a qualitative understanding of how professionals working in the field of elder justice think about what is needed to help understand, prevent, identify, and respond to elder abuse. The respondent universe is all adults who work in the field of elder justice, either in research or practice. We are not seeking representativeness or generalizability; rather we are purposefully seeking to invite those individuals who are believed to be able to provide relevant and valuable insight and input in this project.

This information collection will not require purposive sampling on the basis of any specific social demographics. Identification and recruitment of professionals in the field of elder justice will be coordinated opportunistically through nominations from Working Group members and Project Planning Group members, and includes representatives of the aging network, financial, health care, legal, mental health, protective service, social service, and victim service systems. Profes­sionals recruited for participation will be involved in frontline services, education, training, policy and research related to elder abuse, and will represent local, state and national reach in their work. Working Group and Project Planning Group members will be asked to identify potential respondents at the widest cross section of who may be affected by issues addressed in this particular project.

It was determined through consultation with subject matter experts that 750 would be the most appropriate number of invited respondents in order to elicit the broadest range of perspectives to ensure saturation of the topic within the parameters of the project resources. Within the identified 750 invited respondents, the subject matter experts will further identify a subset of 250 individuals who will be invited to participate in the sorting activity. These 250 will be selected based on requisite understanding of the interrelationships among elder justice issues, trusted perspective by leaders in the field to to ensure uptake of the project information and results, and appropriate degree of attention and visibility.

B.2. Procedures for the Collection of Information

Concept mapping is a participatory mixed methodology, both quantitative and qualitative, that enables diverse participant groups to develop shared conceptual frameworks that can be used in a variety of policy and practice contexts to identify and encourage complexity. The target population that we expect to recruit will be diverse in geographic location and other respects and will collectively represent a diverse set of professional experiences and perspectives within the field of elder justice, upon which their selection and inclusion will be dependent. The approach for this project will be to purposefully sample on the basis of heterogeneity, which is to non-randomly select a broad range of persons who are likely to contribute the full range of ideas that are relevant to the topic under investigation. The goal is to achieve a broad sampling of ideas rather than a representative sample of individuals.

In order to minimize the burden on participation during the sorting and rating activities of concept mapping, the project Working and Planning Groups will engage in a process to synthesize and reduce the statements elicited during the idea generation activity to 100 or fewer. Following the guidance outlined in Kane and Trochim (2007) the study team will review each of the statements for a) relevance to the focus prompts or within the scope of the question at hand, b) redundancy or duplication; c) clarity of meaning, and d) relative appropriateness for the sorting and rating tasks to be completed. Employing a keyword in context approach to content analysis (Krippendorf, 2004) each statement will be systematically coded, organized and, based on the judgment of the Working and Planning Groups, selected for inclusion in the final set that participants will subsequently sort and rate.

In terms of quality control, the sorting, rating and demographic/background virtually all of the data will be gathered directly over the web, thus eliminating any concerns about mis-entering or mis-reading handwritten data. (Respondents who elect not to respond using a computer on the internet will be allowed to submit responses via hard copy or orally by telephone, but CSI has found that the vast majority of respondents prefer to submit responses online.) The software has some constraints built in that prevent errors in the data entry. For example, since this is an unstructured forced-choice sort method (Weller & Romney, 1988), the software does not allow a statement to be placed in more than one group simultaneously. For the ratings, the software only allows legitimate entries (i.e., the integers 1-to-5).

Before any participant data can be used in data analysis, the software requires that the concept map analyst visually inspect the data and indicate by checking a setting that the data are complete and useable. Without this check, the participant’s data will not show up in any subsequent screen that calls for data analysis. Because the Concept System® software was expressly designed to accomplish the concept mapping process and analysis, there is no danger that the statistical analysis procedure might be mis-specified by the analyst.

B.3. Methods to Maximize Response Rates and Deal With Nonresponse

The brainstorming task will be conducted completely anonymously for all respondents. The number of statements elicited in response to the focus prompt will be measureable, but there will no way to determine the actual response rate, as respondents will be able to provide as many statements as they choose without any way to identify which respondents provided which statements. (See Appendix 10 for Brainstorming Reminder to Participate.)

The response rate for sorting and rating will be calculated over the course of the information collection, as the project website administrators will be able to monitor the progress of each participant according to their username. In order to maximize response rates for brainstorming, sorting and rating, a reminder notice will be sent to all invited respondents at multiple points during the period for each task. To encourage response, those invited to participate in brainstorming will be sent a reminder email. (See Appendix 10 for the Sorting and Rating Reminder to Participate and the Rating Reminder to Participate.)

The facilitated discussion response rate will be calculated based on how many invitees (10 per discussion) actually attend. Sending invitees reminders of the discussions in advance of the sessions will maximize the attendance rate. (See Appendix 10 for the Facilitated Discussion Follow-Up.)

The response rate for leadership interviews will be calculated based on how many invited leaders agree to participate. We will maximize response rates by sending each an advance letter signed by the Subject Matter Expert explaining the purpose and importance of the interview. The advance letter will be followed up with a telephone call. A follow-up email will be sent prior to the scheduled interview, including a reminder of the time and date and thanking the leader for agreeing to participate. (See Appendix 10 for the Leadership Interview Follow-Up.)

Benchmarking analysis on previous web-based concept mapping studies employing a similar sampling and recruitment approach have revealed average response rates of 52.4% for sorting and 68.7% for the initial and 48.0% for the subsequent rating activities¹. It is not possible to know the response rate for brainstorming because it is completely anonymous and each respondent can provide as many responses as he or she chooses.

B.4. Test of Procedures or Methods to be Undertaken

Concept mapping will be the primary method for analyzing the data. The Concept System® software will be used to combine the individual participant’s sort data, and using several multivariate statistical algorithms, will organize the information and displays it in a series of easily readable concept maps. This process will begin with construction from the sort information of an NxN binary, symmetric matrix of similarities, Xij. For any two items i and j, a 1 is placed in Xij if the two items were placed in the same pile by the participant, otherwise a 0 is entered (Weller and Romney, 1988, p. 22). The total NxN similarity matrix, Tij will be obtained by summing across the individual Xij matrices. Thus, any cell in this matrix could take integer values between 0 and the number of people who sorted the statements. The value will indicate the number of people who placed the i,j pair in the same pile. The total similarity matrix Tij will be analyzed using nonmetric multidimensional scaling (MDS) analysis with a two-dimensional solution. The solution will be limited to two dimensions because it is generally easier to work with two-dimensional configurations than with those involving more dimensions, (Kruskal & Wish, 1978). Ease of use considerations is important for decisions about dimensionality. For example, when an MDS configuration is desired primarily as the foundation on which to display clustering results, then a two-dimensional configuration is far more useful than one involving three or more dimensions (p. 58).

The analysis will yield a two-dimensional (x,y) configuration of the set of statements based on the criterion that statements piled together most often are located more proximately in two-dimensional space, while those piled together less frequently are further apart. The x,y configuration will serve as the input for the hierarchical cluster analysis utilizing Ward's algorithm (Everitt, 1980) as the basis for defining a cluster. Using the MDS configuration as input to the cluster analysis in effect will force the cluster analysis to partition the MDS configuration into non-overlapping clusters in two-dimensional space. There is no simple mathematical criterion by which a final number of clusters can be selected. The procedure that is typically followed is to examine the initial cluster solution that was the maximum desirable for interpretation in this context. Then, successively lower cluster solutions will be examined, with a judgment made at each level about whether the merger seems substantively reasonable. The pattern of judgments of the suitability of different cluster solutions will be examined and the final number of clusters selected to preserve the most detail and still yield substantively interpretable clusters of statements.

The MDS configuration of the statement points will be graphed in two dimensions automatically by the Concept System program. This "point map" will display the location of all the brainstormed statements with statements closer to each other generally expected to be more similar in meaning. A "cluster map" will be also generated that displays the original statement points enclosed by polygon-shaped boundaries for the clusters. Additional reports will demonstrate similarities and differences in perception and opinion among the participants.

Additional graphics, including pattern matches and bivariate scatter plots, will display stakeholders’ ratings of the ideas. The 1-to-5 frequency and desirability rating data will be averaged across persons for each item and each cluster. This rating information will be depicted graphically in a "point rating map" that will show the original point map with the average rating per item displayed as vertical columns in the third dimension, and in a "cluster rating map" that will show the cluster average rating using the third dimension. Two additional graphic and statistical analyses will be computed based upon the map results. A “pattern match” is defined as the bivariate relationship between the cluster average ratings for two groups, variables or occasions. This will be visually displayed as a ladder graph or pair-link diagram with two vertical axes that represent the two variables and horizontal lines connecting them to represent the ratings for each cluster. A standard Pearson Product-Moment Correlation (r) will be computed to indicate the overall pattern match. In addition, standard descriptive statistics will be produced (mean, SD, N) that will enable significance tests of differences between ratings on clusters. Pattern match graphs will be used to assess consensus or differences of different participant groups on the relative frequency and desirability. A “go zone” is defined as a within-cluster bivariate plot of average statement ratings for two groups, variables or occasions. Like a pattern match, it also displays the Pearson Product-Moment Correlation (r) between the two variables. The plot is a restricted form of a standard bivariate plot in that it: (a) sets the minimum and maximum values for all plots to the same range (based on minimum and maximum statement average for that variable); and (b) the bivariate space is divided into quadrants based on the cluster average of the x and y variables. This effectively means that every go zone plot will have a quadrant that shows which statements in the cluster were rated above average on both variables, one that shows which statements were below average on both and two that show the statements that were above average on one and below on the other. This plot, like a pattern match, will be used to explore consensus, in this case, within-cluster.

The content derived from the facilitated discussions and the leadership interviews will be integrated into the final conceptual framework through an iterative process of data reduction and synthesis. It is anticipated that the facilitated discussions will yield a substantial volume of information, from which information can be extracted to further elucidate emergent concepts. Using the concept mapping framework as the analytic structure for organizing, reducing and analyzing the content captured during the facilitated discussions, we expect to identify insights, themes, patterns and constructs that will aid in the interpretation of the domain. Traditional approaches to qualitative data reduction, coding, and analysis will be used within the concept mapping structure to illuminate the connections individuals make between and among the concepts.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collection and/or Analyzing Data

The following individuals will be contributing to the statistical design and analysis:

DOJ Staff	Telephone Number	Email Address
Andy Mao, , J.D.	202-616-0539	a[email protected]
Laurie Feinberg, M.D., M.P.H., Medical Officer/COTR	202-305-1789	laurie.s.feinberg@usdoj.gov
Concept Systems, Inc. Staff	Telephone Number	Email Address
Mary Kane, Principal Consultant Scott Rosas, Senior Consultant	607-272-1206 607-272-1206	[email protected] [email protected]
Alyssa Goldman, Co-Project Manager Brenda Pepe, Co-Project Manager	607-272-1206 607-272-1206	[email protected] [email protected]

References

American Association of Retired People, Abused Elders or Older Battered Women? Report on the AARP Forum (Oct. 1992).

American Medical Association. (2008). Report 7 of the Council on Science and Public Health, (A-08), Elder Mistreatment. Available at http://www.ama-assn.org/ama/pub/about-ama/our-people/ama-councils/council-science-public-health/reports/2008-reports.page? Last accessed November 21, 2011.

American Medical Association. (2002). Code of Medical Ethics, Opinion 2.02, Abuse of Spouses, Children, Elderly Persons and Others at Risk. Available at: http://virtualmentor.ama-assn.org/2003/11/rdng1-0311.html. Last accessed November 21, 2011.

Anderberg, M. R. (1973). Cluster analysis for applications. New York, NY: Academic Press, Inc.

Barnett, O., Miller-Perrin, C. L., & Perrin, R. D. (2005). Family violence across the lifespan 2^nd Ed. Thousand Oaks, CS: Sage Publications.

Connolly, M.T. (2010). Where Elder Abuse and the Justice System Collide: Police Power, Parens Patrie, and Twelve Recommendations. Journal of Elder Abuse & Neglect, 22(1-2), 37-93.

Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement, 60(6), 821-836.

Coxon, A. P. M. (1999). Sorting data: Collection and analysis. Thousand Oaks, CA: Sage.

Everitt, B. (1980). Cluster analysis. 2nd edition. New York, NY: Halsted Press, A Division of John Wiley and Sons.

Kane, M., & Trochim, W. (2007). Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage Publications.

Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004). A comparison of web and mail survey response rates.  Public Opinion Quarterly, 68(1), 94-101.

Krippendorf, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Newbury Park, CA: Sage Publications.

Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage Publications.

National Aging Resource Center on Elder Abuse, Elder Abuse and Neglect: A National Research Agenda 16 (Jan. 1991) (unpublished manuscript, on file with National Center on Elder Abuse: Clearinghouse on Abuse and Neglect of the Elderly).

National Center on Elder Abuse. Proceedings National Policy Summit on Elder Abuse. Available at www.ncea.aoa.gov/main_site/pdf/whatnew/proceedings.pdf. Last accessed November 23, 2011.

National Research Council. (2003). Elder Mistreatment: Abuse, Neglect, and Exploitation in an Aging America. Panel to Review Risk and Prevalence of Elder Abuse and Neglect. Richard J. Bonnie and Robert B. Wallace, (Eds.) Committee on National Statistics and Committee on Law and Justice, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Osborn, A. F. (1957). Applied imagination. New York: Scribner.

Patton, M. Q. (1990). Qualitative evaluation and research methods 2^nd Ed. Thousand Oaks, CA: Sage Publications.

Ries, A., Voorhees, C., Gittelsohn, J., Roche, K. & Astone, N. (2008). Adolescents’ perceptions of environmental influences on physical activity. American Journal of Health Behavior, 32(1), 26-39. American Academy of Health Behavior.

Rosas, S. R. (November, 2009). Establishing standards in concept mapping: A meta-review and analysis. Paper presented at the 22nd Annual Conference of the American Evaluation Association, Orlando, FL.

Rosas, S. R., & Kane, M. (2012). Quality and rigor of the concept mapping methodology:  A pooled study analysis. Evaluation and Program Planning, 35(2), 236-245.

Rosenberg, S., & Kim, M. P. (1975). The method of sorting as a data gathering procedure in multivariate research. Multivariate Behavioral Research, 10, 489-502.

Schwartz, R. M. (1994). The skilled facilitator: Practical wisdom for developing effective groups. San Francisco: Jossey-Bass.

Senate Special Committee on Aging. (2002). The Elder Justice Proposal of 2002.

Stiegel, L. (2007). Recommendations for the Elder Abuse, Health, and Justice Fields About Medical Forensic Issues Related to Elder Abuse and Neglect. Journal of Elder Abuse & Neglect, 18(4), 41-81.

University of New Hampshire. (1986). Elder Abuse and Neglect: Recommendations from the Research Conference on Elder Abuse and Neglect (unpublished manuscript, on file with National Center on Elder Abuse: Clearinghouse on Abuse and Neglect of the Elderly).

U.S. Department of Health and Human Services, Assistant Secretary for Planning and Evaluation, Office of Disability, Aging, and Long-Term Care Policy. (1992). Report from the Secretary's Task Force on Elder Abuse. Available at http://aspe.hhs.gov/daltcp/reports/elderab.htm. Last accessed November 23, 2011.

U.S. Department of Justice. (2000). Elder Justice: Medical Forensic Issues Relating to Elder Abuse and Neglect, Report Based on the Roundtable Discussion of Experts and Short Papers Submitted by Participants. Available at http://www.nij.gov/topics/crime/elder-abuse/roundtable/expert-testimony.htm. Last accessed November 23, 2011.

U.S. Department of Justice. (2000). Our Aging Population: Promoting Empowerment, Preventing Victimization, and Implementing Coordinated Interventions. Available at http://www.ojp.usdoj.gov/docs/ncj_186256.pdf. Last accessed November 23, 2011.

Weller, S. C., & Romney, A. K. (1988). Systematic data collection. Newbury Park, CA: Sage Publications.

2005 White House Conference on Aging, "Post-Event Summary Report, Think Tank on Elder Abuse and Neglect:  Promoting National and Global Awareness and Effective Action," (September 15, 2005), at http://www.whcoa.gov/about/des_events_reports/PER_CA_09_17_05.pdf.

Wiglesworth, A., Mosqueda, L., Mulnard, R., Liao, S., Gibbs, L., &Fitzgerald, W. (2010). Screening for Abuse and Neglect of People with Dementia. Journal of the American Geriatrics Society, 58(3), 493-500. doi: 10.1111/j.1532-5415.2010.02737.x

Rosalie S. Wolf, Research Agenda: A Research Agenda on Abuse of Older Persons and Adults with Disabilities (1999).

Erica Wood, The Availability and Utility of Interdisciplinary Data on Elder Abuse: A White Paper for the National Center on Elder Abuse (2006) at http://www.ncea.aoa.gov/NCEAroot/Main_Site/pdf/publication/WhitePaper060404.pdf.

1 Rosas, S. R., & Kane, M. (2012). Quality and rigor of the concept mapping methodology:  A pooled study analysis. Evaluation and Program Planning, 35(2), 236-245.

11