A Study of Reliability and Consequential Validity of a Mathematics Diagnostic Assessment System in Georgia

Forms and Documents

Document Name	Status
Diagnostic Assessment System in Georgia Focus Group and Surveys Form and Instruction	New
IRB Approval Letter 7-19-18.pdf Supplementary Document	2018-09-19
IKAN GLOSS OMB Clearance Part B Final.docx Supporting Statement B	2019-03-06
Appendix J - Details for Question A12 8-13-18.docx Supplementary Document	2018-09-19
IKAN GLOSS OMB Clearance Part A Final.docx Supporting Statement A	2019-03-06

IC Document Collections

IC ID	Document Title	Status
233088	Diagnostic Assessment System in Georgia Focus Group and Surveys Form and Instruction	New

ICR Details

OMB Control No: 1850-0946	ICR Reference No: 201809-1850-006
Status: Active	Previous ICR Reference No:
Agency/Subagency: ED/IES	Agency Tracking No: ED-2018-ICCD-0087
Title: A Study of Reliability and Consequential Validity of a Mathematics Diagnostic Assessment System in Georgia
Type of Information Collection: New collection (Request for a new OMB Control Number)	Common Form ICR:  No
Type of Review Request: Regular
OIRA Conclusion Action: Approved with change	Conclusion Date: 03/08/2019
	Date Received in OIRA: 10/31/2018
Terms of Clearance:

	Inventory as of this Action	Requested	Previously Approved
Expiration Date	03/31/2022	36 Months From Approved
Responses	44	0	0
Time Burden (Hours)	606	0	0
Cost Burden (Dollars)	0	0	0

---

Abstract: Several school districts in Georgia (including Forsyth and Jefferson Counties) currently use the Individual Knowledge Assessment of Number (IKAN; New Zealand Ministry of Education, 2011) and the Global Strategy Stage (GloSS; New Zealand Ministry of Education, 2012) as part of a diagnostic assessment system within their multi-tiered systems of support. The IKAN and GloSS assessments were designed for use in Grades K–8 (GaDOE, 2015). IKAN provides information on students’ number knowledge (that is, magnitude comparisons, knowledge of base 10 system and meaning of decimals and fractions), and the GloSS provides information on strategies students use when solving mathematical problems. When used together, the IKAN and GloSS assessments furnish teachers with information on how students solve mathematics problems and students’ understanding of whole and rational number concepts. Teachers can then use the information from the assessments to tailor their instruction to students’ levels of understanding and address problems that underlie lack of success with grade-level curriculum. The Georgia Department of Education (GaDOE) has received positive feedback through testimonials from district-level personnel and math coaches supporting the use of IKAN/GloSS. Yet, very limited psychometric data exists to support the use of these measures. GaDOE has not conducted reliability or validity studies using its student population. While many studies have been conducted in New Zealand by the Ministry of Education, participating Georgia school districts and the GaDOE are interested in psychometric data using teachers and students in their state in the context of their state system of standards, assessments, and accountability. Thus, through the Improving Mathematics Research Alliance, the GaDOE requested REL Southeast conduct a study to determine the reliability and validity of the IKAN/GloSS diagnostic assessments. The three research questions guiding the project relate to the inter-assessor reliability of the GloSS and IKAN assessments when administered by two assessors within a one-week period and the consequential validity of using IKAN and GloSS diagnostic assessments (that is, teachers’ perception of their instructional utility for providing intervention). Student data from the IKAN and GloSS assessments administered by teachers and mathematics coaches will be used to answer the two research questions related to the inter-assessor reliability studies.

---

Authorizing Statute(s): None

Citations for New Statutory Requirements: None

---

Associated Rulemaking Information
RIN:	Stage of Rulemaking:	Federal Register Citation:	Date:
	Not associated with rulemaking

---

Federal Register Notices & Comments
60-day Notice:	Federal Register Citation:	Citation Date:
	83 FR 42492	08/22/2018
30-day Notice:	Federal Register Citation:	Citation Date:
	83 FR 54729	10/31/2018
Did the Agency receive public comments on this ICR? No

---

Number of Information Collection (IC) in this ICR: 1

IC Title	Form No.	Form Name
Diagnostic Assessment System in Georgia Focus Group and Surveys	N/A, N/A, N/A	Focus Group Questions ,   IKAN/GloSS Sample Items ,   Teacher Consequential Validity Survey

---

ICR Summary of Burden

	Total Approved	Previously Approved	Change Due to New Statute	Change Due to Agency Discretion	Change Due to Adjustment in Estimate	Change Due to Potential Violation of the PRA
Annual Number of Responses	44	0	0	44	0	0
Annual Time Burden (Hours)	606	0	0	606	0	0
Annual Cost Burden (Dollars)	0	0	0	0	0	0

Burden increases because of Program Change due to Agency Discretion: Yes

Burden Increase Due to: Miscellaneous Actions

Burden decreases because of Program Change due to Agency Discretion: No

Burden Reduction Due to:

Short Statement: This is a new request - therefore all burden is new. This results in a program change increase in burden and responses of 606 hours and 44 responses respectively.

---

Annual Cost to Federal Government: $179,723

Does this IC contain surveys, censuses, or employ statistical methods? No

Does this ICR request any personally identifiable information (see OMB Circular No. A-130 for an explanation of this term)? Please consult with your agency's privacy program when making this determination.     Yes

Does this ICR include a form that requires a Privacy Act Statement (see 5 U.S.C. §552a(e)(3))? Please consult with your agency's privacy program when making this determination.     Yes

Is this ICR related to the Affordable Care Act [Pub. L. 111-148 & 111-152]? No

Is this ICR related to the Dodd-Frank Wall Street Reform and Consumer Protection Act, [Pub. L. 111-203]? No

Is this ICR related to the American Recovery and Reinvestment Act of 2009 (ARRA)? No

Is this ICR related to the Pandemic Response? Uncollected

---

Common Form ICR:  No

---

On behalf of this Federal agency, I certify that the collection of information encompassed by this request complies with 5 CFR 1320.9 and the related provisions of 5 CFR 1320.8(b)(3).

The following is a summary of the topics, regarding the proposed collection of information, that the certification covers:

(a) It is necessary for the proper performance of agency functions;

(b) It avoids unnecessary duplication;

(c) It reduces burden on small entities;

(d) It uses plain, coherent, and unambiguous language that is understandable to respondents;

(e) Its implementation will be consistent and compatible with current reporting and recordkeeping practices;

(f) It indicates the retention periods for recordkeeping requirements;

(g) It informs respondents of the information called for under 5 CFR 1320.8 (b)(3) about:

(i) Why the information is being collected;

(ii) Use of information;

(iii) Burden estimate;

(iv) Nature of response (voluntary, required for a benefit, or mandatory);

(v) Nature and extent of confidentiality; and

(vi) Need to display currently valid OMB control number;

(h) It was developed by an office that has planned and allocated resources for the efficient and effective management and use of the information to be collected.

(i) It uses effective and efficient statistical survey methodology (if applicable); and

(j) It makes appropriate use of information technology.

If you are unable to certify compliance with any of these provisions, identify the item by leaving the box unchecked and explain the reason in the Supporting Statement.

Certification Date: 10/31/2018

---