AUSTIN – A review of Texas’ student testing program found that the state is a leader in test security procedures but offered suggestions for enhancing the program.

The Texas Education Agency, which oversees the state’s student testing program, commissioned a review of its test security prevention and enforcement measures in January after allegations arose about “suspicious” scores at some schools.

The agency hired Dr. Greg Cizek, who is a professor of Educational Measurement and Evaluation at the University of North Carolina at Chapel Hill, to review the program.  An active researcher and writer, Cizek’s writings include Cheating on Tests: How to Do It, Detect It and Prevent It; Detecting and Preventing Classroom Cheating: Promoting Integrity in Educational Assessment; and Addressing Test Anxiety in a High Stakes Environment.  Cizek is also a member of the Texas Technical Advisory Committee, a team of national testing experts that provides advice to the Texas testing program.

In a report delivered to TEA, Cizek says: “…this review concludes that TEA is among the leaders in test security procedures for K-12 educational achievement testing programs.”  However, he offered a number of suggestions for enhancing the integrity of the program.

Commissioner of Education Shirley J. Neeley said, “It is comforting that a national testing expert has confirmed our belief that the security measures in force for the Texas assessment program are some of the best in the country.  We will consider all his recommendations carefully and have already taken action that addresses some of the issues raised in his report.”  For example, the number of employees that investigate allegations of cheating was increased from three to six staff positions recently, including creating a position called director of test security.

Some of Cizek’s findings and recommendations are based on the results of a survey of 539 educators who attended the Texas Assessment Conference in February.

The survey found that 96.7 percent of the respondents believe that educators in their district are “at least somewhat concerned, conscientious, or vigilant about test security.”

Security Policies and Documentation
 

            The survey and Cizek’s own research found that “security for the Texas student assessment program is supported by an abundance of documentation that provides clear and specific information to test users.”  Along with numerous documents, the agency also uses a trainer-of-trainer method to teach educators in the field about security provisions.
            “Overall, this review concludes that the policies and documentation are exemplary and would serve as good models.  They are well organized, clear, and thorough,” he said.
 

In-Process Procedures and Follow-Up Activities
 

      “It is apparent that attention to test security pervades test development, administration, and scoring activities for Texas assessments.   While under the control of TEA or its testing contractors, all secure test materials are maintained in such a way as to prevent unauthorized access or disclosure prior to or following administration of a test,” Cizek wrote.

      “In contrast to documentation and in-process procedures currently reviewed, follow-up activities conducted by the TEA are comparatively less well-developed and thorough.  For example, follow-up on reported test security violations does not appear to be timely or well-documented over time,” he found. 

Recommendations 

      Cizek’s suggestions for improving the testing program, include:

·        Developing model test security policies and procedures for district consideration/adoption.  Most polices are now focused on the state level but it will help promote even-handed enforcement at the district level if sample local policies and procedures were offered to them.

·        Providing more examples of inappropriate test administration behaviors to educators.

·        Enhancing the training provided to test coordinators by expanding the frequency, extent and audiences served by training sessions, as well as providing more training materials.  Currently, TEA typically trains lead educators who return to their districts and train colleagues.  This is known as a trainer-of-trainers model.

·        Considering expansion of the audiences solicited for reporting of test irregularities.  Cizek notes that parents, students and others, along with educators, may have information of cheating incidents.  TEA officials said currently most reports alleging cheating on state tests come from educators but some do come from other sources, such as parents.  The reports typically come to the agency via phone or written report.  Cizek suggested providing an incident form online that could be used to report a possible testing irregularity.  This spring, TEA created an email address to be used specifically for reporting testing problems.

·        Broadening involvement in on-site auditing of test preparation and administration activities.   He suggests having TEA staff develop a plan for identifying campuses or districts for audits; setting a number or percentage of on-site interviews that will be conducted; and creating procedures to be followed when conducting the audits.

·        Developing and distributing procedures for local districts to follow when investigating allegations of inappropriate testing behavior.  He also suggests creating methods to be used by districts when following up on questionable results.

·        Reviewing the effectiveness of current investigation policies.  Currently, districts are asked to investigate allegations first and then report their findings to TEA.  Cizek said educators at the local level probably “lack strong incentives for vigorously and objectively collecting information in the course of the preliminary investigations.”  He suggests developing a “triage” system that would trigger “external, independent review in specified situations.”

·        Developing plans for additional post-testing quality control analysis.  He suggests developing policies for the use of the erasure analysis, providing results of the analysis to districts and following-up on cases with excessive erasures after one year.  He also suggests creating an additional method for identifying unusual test results.  Whichever method is selected, Cizek says the procedures used and findings of the analysis should be reported annually.

·         Reviewing staffing levels and job qualifications in the assessment division’s Security Task Force (STF). 

·        Enhancing data gathering and reporting of Security Task Force activities, which would help spot trends and provide some deterrence to security breaches. 

·        Consider more vigorous and expeditious prosecution and resolution of cases in which testing irregularities or wrongdoing have been found. 

      Cizek concluded that, “To the extent that resources to address security concerns are limited, the TEA should consider enhancements in training, additional STF staffing, use of analytical tools, and improvements in documentation and dissemination of activities as the primary avenues for promoting the integrity and validity of scores of assessments.”

            The commissioner of education said: “Dr. Cizek offers many good suggestions for us to consider.  We have already begun to take some of the actions he suggests.

            “We have beefed up staffing in the unit that investigates allegations of cheating.  We are creating a database of testing irregularities that will help us spot trends and alert districts to areas of concern.  We are considering creating training modules for use by educators in the field and preparing a manual that pulls all the security-related provisions into one document. Our testing contractor has also hired Caveon, a leading test-security firm to monitor test data for irregularities,” Neeley said.

            “We want the integrity of our testing program to be unassailable.  We test almost 3 million students a year and each student takes two to four exams.  Despite having one of the largest testing programs in the country, we have a small number of cases of wrongdoing alleged, much less confirmed, each year. But we have zero tolerance for those who cheat on tests so we will take Dr. Cizek’s suggestions seriously as we look for ways to enhance our system,” Neeley said.

            The full Cizek report is available under the What’s New button at: http://www.tea.state.tx.us/student.assessment/index.html.