A Report to
Comptroller Carole Keeton Rylander
Public Education Integrity Task Force
In 1999, a handful of Texas public school districts were caught manipulating data on dropout rates and the state-mandated test, the Texas Assessment of Academic Skills (TAAS). The good news is Texas has one of the best and most comprehensive public school accountability systems in the nation; so the errors and manipulations were detected and the responsible individuals were held accountable. The bad news is the 1999 incidents raised a red flag in the minds of the public–are there inherent weaknesses in the system?
The Texas public school accountability system is based on regular assessments of academic skills and extensive data gathered from schools through the Public Education Information Management System (PEIMS). Based on the reported data, public schools receive an annual accountability rating, ranging from exemplary to low performing.
But improper and inaccurate reporting can skew the ratings. In 1999, Austin Independent School District administrators were under criminal investigation for tampering with TAAS documents. In Dallas and Houston, excessive erasures on the answer sheets were questioned. Ysleta ISD was suspected of providing inaccurate PEIMS data. In other districts, TEA investigated possible misclassification of special education students to boost performance ratings.
There also were reports of key student absences on test days—absences purposely arranged to affect overall test scores and thus a school’s rating. The number of schools involved represented a small percentage of those in the state, but enough to cause concern.
And, alarmingly, the system for data reporting and rating the performance of Texas public schools is supported by hardware and software more than 17 years old. In the Internet Age, the system runs on DOS—the technological equivalent of a Big Chief Tablet.
Under her mandated authority to examine state agencies and school districts for effectiveness and efficiency, Comptroller Carole Keeton Rylander created the Public Education Integrity Task Force—a diverse group of citizen volunteers from the private and public sectors.
After more than a year of study, the task force concluded the accountability system had not failed. There are some weaknesses in the system, but it is designed to be self-correcting. It is. The inaccurate data were detected, those responsible were held accountable and superintendents took steps to remedy the problems.
...but it can be better
The system works, but it can be better. The task force found significant obstacles to the compilation and reporting of accurate data—including unclear laws, rules and regulations surrounding the reporting process. Some of obstacles include the following:
- State law does not expressly authorize TEA to conduct on-site investigations of testing and reporting irregularities, or to impose sanctions for false or misleading data reporting. Also, TEA has a limited number of staff members to conduct investigations.
- Schools are not rated on the integrity of the data that is reported. There are no significant sanctions for unauthorized disclosure of TAAS data.
- The Commissioner of Education recommends the process and criteria by which teachers, administrators and counselors are evaluated, but the criteria do not include compliance with the state Education Code or TEA rules.
- State law requires cost ratios that compare administrative costs to instructional costs—but cost categories are not clearly defined. Administrative costs, for example, do not include principals’ salaries, and instructional costs include counselors even though they do not teach.
- School districts have difficulty keeping trained personnel who can properly submit data. The data entry clerks are among the lowest paid employees in a district and the turnover rate is high. Training is not always available and there is little consistency in the level of training and support the district receives from the service centers.
- State law requires the Commissioner of Education to develop rules that ensure the Public Education Information Management System (PEIMS) provide useful, accurate and timely information on student demographics and academic performance, personnel and school district finances. That has not been done.
- The technology that supports PEIMS is antiquated: more than 17 years old and DOS-based. It can’t keep up. What’s more, school districts use a variety of technologies—hardware and software— and some are much more sophisticated that others. Compatibility is a problem.
- The basic design and premise of the Public Education Information Management System has remained virtually unchanged since 1983. Data submitted today might not be available for public viewing for months, by which time the information is outdated. Corrections are difficult and cannot be made after certain critical points. And the process eats up the districts’ precious limited resources that could otherwise be used in the classroom.
Raise the bar
Public school accountability is a high stakes system that depends on collecting and reporting accurate data. When the data is poor, the playing field is not level. The task force made a number of recommendations to ensure everyone plays by the same rules.
The goal was to identify ways to ensure the accuracy of data submitted to the TEA and the integrity of accountability ratings. The task force also wanted to create incentives to encourage administrators, teachers and staff to adhere to the highest ethical standards in record keeping and reporting.
The task force recommended only minor changes to existing law to preserve the integrity of 1995 education reforms. The majority of suggested changes could be implemented within existing authority.
Some of the task force’s key recommendations are as follows:
- Tighten Controls and Step-up Enforcement. The TEA and the State Board of Educator Certification should increase audit and enforcement staff and the Legislature should make unauthorized distribution of TAAS tests a Class A misdemeanor.
- Administrative Data Reporting. To ensure that scarce education dollars are actually being used to educate students, the Legislature should refine the administrative cost ratio so that elected school board members, parents, teachers and other interested parties can easily see how much a district is spending on classroom instruction.
- Set Hardware and Software Standards. Technology has changed significantly since 1983, and TEA’s system is old and DOS-based. School districts use a variety of technologies, both hardware and software and some are much more sophisticated than others. Compatibility of software and hardware is contributing to the problem of bad data.
- Improve and Streamline the Entire Data Collection System for the State. The basic design and premise that rests at the heart of the accountability system has remained virtually unchanged since 1983. The entire system needs to be reassessed and improvements put in place that include a system of checks and balances that will readily identify errors or purposeful data manipulation.
- Reward Accurate Reporting. The Commissioner of Education already has authority to establish additional categories of awards, but the additional categories must be tied to a school’s involvement with paired, low-performing campuses. The Commissioner should be given greater flexibility to recognize data integrity.
- Discourage Excessive Exemptions and Absences. To discourage districts from exempting students from TAAS to achieve ratings of recognized or exemplary, districts should be precluded from those ratings if exemption rates exceed a specified percentage without verifiable reason. Consideration should also be given to identifying excessive rates of absenteeism on testing dates as a potential bar to higher ratings.
- Improve Training and Support. Data entry clerks are among the lowest paid employees in a district and the turnover rate is high. Training is not always available when it is needed, and there is little consistency in the level of training and support a district receives. The role of the Regional Education Service Center (RESC) should be evaluated to ensure that districts are receiving the support they need to gather and report data accurately.