Key Findings and Recommendations
The value of an effective system is that it fairly and equitably weeds out those individuals who try to circumvent the system from the vast majority of honest people who have dedicated their lives to educating our children. The system should be a means to an end, not an end in itself. And there should be no defense for grievous errors. The Task Force recommends enhancing and preserving local control by clarifying the roles and responsibilities of those involved in the process, improving training and support, adopting rules to improve the system and addressing the issues causing the problems so that the opportunity for error is minimized.
The recommendations are made in five categories: (1) tighten and strengthen the existing authority while preserving local control; (2) improve training and support; (3) formally adopt a PEIMS rule; (4) set hardware and software standards; and, (5) assess PEIMS for further improvements.
I. Tighten Controls and Step-up Enforcement To effectively tighten controls and step up enforcement, the Task Force recommends improvements to (1) special accreditation investigations, enforcement, rewards and sanctions; (2) appraisals; (3) self-reporting; and (4) administrative data reporting. A. Special Accreditation Investigations, Enforcement, Rewards and Sanctions Although statutory authority is implied under the Texas Education Codes Section 39.075, the law does not expressly authorize TEA to conduct on-site investigations to answer questions or investigate academic excellence indicators, dropout reporting, TAAS testing irregularities or PEIMS data reporting or impose sanctions for false or misleading data reporting. In 1999, after the Commissioner lowered Ysleta ISD's accreditation rating as a result of suspected inaccurate PEIMS data submissions, the district challenged the commissioner's authority to lower the accreditation rating in an administrative action. After individual campus ratings were restored by TEA, the action was ultimately dropped by the school district, thereby precluding any court ruling defining the commissioner's authority. The district retained an overall accreditation of "Unacceptable: Data Quality" for the 1998-99 academic year. Having express authority in law, including rulemaking, would reduce the likelihood of additional legal challenges. Section 39.074 authorizes the Commissioner to direct TEA to conduct on-site investigations of a district to obtain answers to questions concerning education programs required by federal law or funded by federal funds. Staff in TEA's Division of Audits and the State Board for Educator Certification (SBEC), however, are limited and tend to concentrate their efforts on high-dollar criminal investigations and cases involving physical harm to students. Compliance with the rules governing data collection and reporting can be improved through enhancing the special accreditation investigation authority of the Commissioner and increasing audit capability, including some provision for random sampling for errors.
With an eye toward providing parents with accurate and easily understandable information about the schools their children attend, the Commissioner needs to have clear authority to order a special accreditation investigation to be performed to investigate allegations of testing irregularities or inaccurate PEIMS data reporting. The legislature should also expressly authorize the Commissioner of Education to develop by rule additional accreditation ratings to be imposed based on the results of a special investigation. While this is implied in statute, the State Board of Education (SBOE) rule-making language in §TEC 39.072 contradicts all other accountability authority granted the Commissioner in the remainder of Chapter 39. If a school district has significant reporting errors, the only rating available currently is "academically unacceptable." A more appropriate rating may be "unacceptable for data quality" as the Commissioner imposed on two districts in 1999. Or if a district has been found to have significant financial mismanagement concerns, rather than lower the district's academic rating, it may be more appropriate to impose a rating of "unacceptable for financial management." Additional rating categories of this type need to be adopted formally as a Commissioner rule.
The Texas Education Code §§12.027(a)(3) and 12.115(a) state that charters can be revoked by the SBOE for violations of charter school laws as well as "other applicable federal or state law or rule." Section 12.104(b)(2)(K) makes sanctions by the Commissioner under Section 39.131, Education Code applicable to charters. Those sanctions are available following a special accreditation investigation under Section 39.075, which could include concerns about test security or data accuracy.
In addition, while the Texas Penal Code provides that tampering with a governmental record is an offense, there are no significant sanctions for unauthorized disclosure of a TAAS test. The TEA is currently required to release a TAAS test annually after the last date it has been administered for the year. However, sanctions should be available for premature release of a TAAS test to preserve the integrity of the test.
The Education Code also directs the SBOE to develop a plan for awarding schools that are rated as exemplary and recognized, but data integrity is not one of the criteria on which schools are rated. We believe the integrity of the criteria used should be unquestionable.
- The Texas Education Agency and the State Board for Educator Certification should increase audit and enforcement staff and resources to strengthen enforcement capability.
- The Commissioner of Education should, by rule, establish guidelines to positively recognize schools that have reported data accurately and authorize data integrity to be considered in determining awards for a school. The Commissioner already has authority to establish additional categories of awards under Section 39.092 of the Education Code, but the additional categories must be tied to a school's involvement with paired, low-performing campuses. The Commissioner should be given greater flexibility to recognize data integrity.
- The Commissioner should adopt through rulemaking additional accreditation rating categories (e.g., Unacceptable for Data Quality, Unacceptable for Financial Management) that can be imposed by the Commissioner based on the results of a special accreditation investigation. These additional ratings are already impliedly authorized by Section 39.072 of the Education Code, though an express authorization could be helpful.
- The Legislature should clarify in the Education Code that the Commissioner has explicit authority to conduct a special accreditation investigation and adjust performance ratings for violations of TAAS test security or for submitting false or misleading PEIMS data for both independent school districts and open-enrollment charter schools.
- The Commissioner should revise administrative rules to clarify prohibited conduct and sanctions that can be imposed.
- The Legislature should revise the Texas Penal Code to provide Class A misdemeanor sanctions for unauthorized distribution of TAAS tests.
- The Commissioner of Education should extend the model used in the 2000 rating cycle to preclude district ratings of recognized or exemplary for districts with exemptions from TAAS that exceed a specified percentage (in the absence of showing of a particular and unique concentration of students appropriately exempted in the district) and consider identifying excessive rates of absenteeism on testing dates as a potential bar to higher ratings.
- The Commissioner of Education should, by rule, clarify the responsibilities of districts and charter schools to ensure the integrity of data reported through PEIMS.
- The Texas Education Agency should emphasize the existing requirement in Education Code §42.008 for districts to include in their annual independent audits a review of the accuracy of fiscal information submitted through PEIMS.
- The Texas Education Agency and the Legislature should clarify that open-enrollment charters must comply with the reporting and compliance requirements for data and accountability expected of regular independent school districts.
B. Appraisals The Commissioner of Education recommends the process and criteria by which to appraise the performance of teachers, administrators and counselors. The commissioner's rules on administrators' appraisal criteria, however, do not include compliance with the Education Code or Commissioner or SBOE rules as an evaluated factor. Recommendations:
- Add an item to the administrators' performance evaluation that addresses "Accuracy and Integrity of Data Reported to TEA." This item could include reporting student retention data, PEIMS data reporting, TAAS testing results and TAAS security procedures. In addition, revise the appraisal criteria for "Management of administrative, fiscal and facilities functions" to include evaluating the superintendent and administrators with responsibility for data collection and submission based on how they manage and submit data to TEA in a timely fashion.
- Add to the appraisals of Regional Education Service Center administrators an item that would assess their ability to provide assistance to schools that would help ensure accurate PEIMS data with built-in rewards.
C. Self-Reporting The law requires teachers and administrators who have direct knowledge of data manipulation or testing violations to report those incidents to the appropriate authorities. The random survey of administrators and teachers (Appendix B) showed that more than 4 percent were aware of teachers who assist students during TAAS testing or who revise answer sheets. Yet, only a handful of incidents were reported to TEA. Administrators and teachers said they feared retaliation for reporting a fellow educator's misconduct. The Texas Whistleblower Act, Government Code §554.002, protects public employees from actions by a government entity to suspend or terminate the employment of an employee who reports misconduct. The Whistleblower Act does not protect an employee from a lawsuit filed by a fellow employee who may be implicated or embarrassed by the report of misconduct. Recommendations:
- The Texas Education Agency should expand toll-free lines so that school district administrators and other individuals can obtain quick responses to questions during TAAS administration. TEA's testing contractor currently maintains four toll-free lines at their Austin operations center to provide school districts assistance not only during test administration but also throughout the school year. This concept should be expanded upon and widely publicized.
- The contracted test administrator should be required to send each district a searchable Compact Disc containing test instructions and responses to frequently asked questions. This information should also be made available on TEA's website.
- The Texas Education Agency should implement a toll-free line that individuals can use to report test misconduct.
- The State Board of Education should clarify administrative rules to show that when a district or campus reports on its own data quality concerns, the Commissioner shall consider those self-reports as a positive mitigating factor when determining appropriate action.
- The Legislature should adopt a statute to provide immunity for someone who makes a good-faith report of misconduct (e.g., the person reporting the misconduct cannot be sued for libel or slander).
D. Administrative Data Reporting In response to rising school administration costs, the 1991 Texas Legislature imposed administrative cost ratios that compare administrative costs to instructional costs. For each school district, administrative costs are divided by instructional costs to calculate the district's administrative cost ratio. The administrative and instructional cost categories that constitute the ratio are not clearly defined in statute. Administrative costs do not include salaries of principals, although principals are clearly administrators. Instructional costs are defined to include counseling expenditures although counselors do not teach and often perform administrative duties such as course scheduling. The complex nature of the calculation makes it difficult for the general public to understand or verify. Recommendations:
- The Legislature should refine the administrative cost ratio to clearly show actual investments in classroom instruction.
- The new measures should be constructed to be simple and easily understood by elected school board members, parents, teachers and other interested parties.
II. Improve Training and Support School districts report difficulty keeping trained personnel who can properly submit data through PEIMS. Often the PEIMS data entry clerks are among the lowest paid employees in a district and the turnover rate is high. Training is not always available to handle mid-year hires and there is little consistency in the level of training and support a district receives. The role of the Regional Education Service Center (RESC) should be evaluated to ensure that districts are receiving the support they need to gather and report data accurately. Recommendations: The Legislature should authorize an independent third party to:
- evaluate the role of RESCs in the PEIMS data collection process;
- identify and recognize RESCs that are providing high quality support;
- evaluate RESCs' responsibility for training and assisting districts with PEIMS reporting and for overseeing data reporting to ensure data integrity;
- evaluate RESCs' responsibility for assessing data quality on an ongoing basis to determine districts' and charter schools' training needs; evaluate RESCs responsibility for providing consulting services on management reporting systems and for detecting data quality problems; and
- develop recommendations for enhancing the support provided to schools.
III. Formally Adopt PEIMS Rule The Texas Education Code states that the Commissioner of Education shall develop rules to ensure the PEIMS system provides useful, accurate and timely information on student demographics and academic performance, personnel and school district finances. This has not been done. Recommendations:
- The Commissioner of Education should adopt a rule that outlines school districts' PEIMS reporting requirements as outlined in the PEIMS Data Standards handbook or other TEA directives. (TEA is currently developing Commissioner's Rules for PEIMS and they are expected to be effective in February 2001.)
- The Commissioner should amend reporting requirement rules to make school districts responsible for accurately reporting PEIMS data.
- The Commissioner should, by rule, require a minimum level of training for PEIMS data entry operators and institute a review mechanism to ensure ongoing and uniform training.
IV. Set Hardware and Software Standards A system is only as good as the architecture that supports it. PEIMS and the technology that supports it are more than 17 years old. The system is antiquated. To ensure the integrity of the data and the accountability system that depends on that data, the infrastructure must be reviewed and ultimately replaced by advanced systems that are highly reliable. Technology has changed significantly since 1983. TEA's system is old and DOS-based. School districts use a variety of technologies, both hardware and software and some are much more sophisticated than others. Compatibility of software and hardware has become an increasing problem. Recommendations:
- The Commissioner of Education should appoint a representative group of individuals from various sized districts, vendors, TEA, the Telecommunications Infrastructure Fund (TIF) Board, the Department of Information Resources (DIR) and the General Services Commission to establish administrative technology standards.
- This committee should seek to set standards that address both TEA's and the districts' needs and capabilities. For example, the proposed software validation process should be designed to provide districts and campuses a set of specific guidelines for selecting quality software that will facilitate the PEIMS collection process as well as the district's day-to-day administrative needs. When the committee's work is complete, the Commissioner should adopt these standards and publish them on the agency Web site.
- The Department of Information Resources should certify vendor products that meet the established standards, but school districts should not be required to buy those products.
- Districts would not be required to change hardware or software, but as districts buy new equipment or software, the standards could be used for Requests for Proposals.
- The Legislature should amend the Texas Utilities Code to authorize the Telecommunications Infrastructure Fund Board to consider as a factor in the board's decision to award TIF grants to public schools whether proposed purchases of administrative hardware and software by public schools meet adopted TEA standards.
V. Assess Public Education Information Management System (PEIMS) for Improvements Many of the laws, rules and regulations for collecting and submitting school district data have evolved over the years to strengthen the state accountability system. What has remained virtually unchanged since 1983 is the basic design and premise that rests at the heart of the accountability system. The system is designed to produce information as a snapshot in time. Data submitted today might not be available for public viewing for months, by which time the information is outdated. This lapse in time makes it difficult for districts to use comparative information from other districts for effective planning and management purposes. Corrections are difficult and cannot be made after certain critical points. The complexity and voluminous nature of the data gathered is of constant concern to districts that must divert limited resources to data gathering that could otherwise be used in the classroom. Recommendations:
- The Texas Education Agency should hire an independent consulting firm to work with TEA, school districts, Regional Education Service Centers and legislative representatives to assess the entire PEIMS system structure, data elements and data collection and submission techniques for possible enhancements. The computer architecture should be upgraded if feasible.
- The Commissioner of Education should direct periodic reviews of PEIMS similar to the Sunset Commission reviews currently performed for state agencies and programs. The Commissioner already has statutory authority to conduct these reviews.
- The independent contractor should ensure that any recommended new or modified system is simple, easy to understand and use and has the ability to reduce the administrative and financial burden that the current system places on school districts and maintains data integrity and quality.
- The independent contractor should ensure that any proposed system has a system of checks and balances that will readily identify errors or purposeful data manipulation. Blind protocols could be used at random to accomplish this task.