Overview
If you are completing an RFA for contract tutoring and wish to include Achieve in your program you may need to include a Validity Report for testing with Achieve and answer additional questions.
Article Sections
Validity Report
- Download Validity Report
- Link to Achieve Validity Report and Sample test papers on TutorFirst Website.
Q&A
If you are completing an RFA for contract tutoring and wish to include Achieve in your program you may need to answer additional questions.
The following responses from Achieve to some typical questions might prove useful.
Q: Provide evidence describing the technical qualities of the test. Evidence must include reliability and validity analyses or studies that determine whether the test meets its intended purpose.
A: Achieve is a criterion-reference test which meets the generally accepted standards of validity and reliability set forth in "Standards for Educational and Psychological Testing" (1999), published by The American Research Association. The test is mulitple-choice in nature and covers areas in Reading/Language Arts & Math for grades k-12. To ensure content validity, Achieve considered the amount of testing time and types of test items based on compatible standardized tests used in the state. In developing the test, a list of objectives on each unit was compiled to determine subject validity of these test items. A matrix was constructed linking objectives & test questions.
Q: At a minimum, content validity must be established. If there was a review of the test by a panel of experts on curriculum in the subject that the test is intended to measure, describe the procedures for review and the qualifications of the panel. If there was a study or analysis conducted to determine validity, describe what type of study or analysis was conducted and the results of the study or analysis.
A: The test was developed by a group of educational experts ranging from college professors to teachers. In selecting test items, consideration was given to test validity and reliability. The group of experts who wrote the test items examined the state curriculum adapted by the Department of Education and released items from previously used standardized tests that the state has used for assessing student achievement. Test items were written based on those considerations because the questions were appropriate for the target population, clearly worded, and contained appropriate vocabulary. Each item was field tested and carefully designed to align with state standards.
Q: At a minimum, test reliability must be established. If there was a study or analysis conducted to determine the type of study or analysis conducted (test-retest, internal consistency, etc.). Describe the results of the study or analysis.
A: To ensure the dependability, consistency, and trustworthiness of the scores, the assessment was used several times with the same sample population under the same treatment conditions. The reliability of the test was determined from the application of one of the Kuder-Richardson formulas, which are based on the proportion of persons correctly answering each item and the standard-deviation of the scores. Achieve scores were cross-referenced with the students' official standardized test scores. The results were analyzed by the statistical software, and the reliability coefficient was .81 for the Reading/Language Arts & .79 for Math.
Q: At a minimum, the test must have an accurate scoring system. Provide evidence that the test was appropriately calibrated. Describe measures taken to ensure the accuracy of the scoring system for the test. Describe the metric used to measure student performance. If a standard setting was conducted, describe the type of standard setting (Angoff, Modified Angoff, Bookmark, or Contrasting Groups) and the results.
A: Achieve scoring is Modified Angoff. An online grading rubric is used by educators/tutors for scoring the assessment. Each objective is broken down and scored based on the percentage of questions the student answers correctly. Depending on the percentage, the Achieve system breaks down mastery, non-mastery, or partial mastery for each skill level. Tutoring concentration should be for areas of non-mastery or partial mastery for each student.
Q: Indicate the test guidelines describing reasonable procedures recommended for Students Requiring Special Accommodations/Students with Disabilities.
A: Educators/teachers may use any of the guidelines based on their state's specific testing procedures for special needs students. These might include, but are not limited to: additional time needed for testing, enlarging the print, reading test questions to the student, converting the assessment to Braille, etc.