Resource Center

• by José Cardenas, Ed.D. • IDRA Newsletter • October 1998 • 

In the next few months, I will probably be called upon to present testimony in court concerning the use of the Texas Assessment of Academic Skills (TAAS) test as a valid measure for high school graduation. A suit filed by the Mexican American Legal Defense and Educational Fund (MALDEF) is challenging state regulations that prescribe that regardless of student performance in the schools, successful performance on the TAAS is required for the granting of a high school diploma.

Being called upon as an “expert” witness demands that I review my knowledge and experience in this area. On the basis of my past expert testimony in almost 100 court cases, I can expect two hours on the hot seat in direct testimony, followed by at least another two hours of cross examination.

What do my 48 years as an educator involved in local, state, national and international education provide me in preparation for this testimony?

The first question that must be addressed is: What constitutes good performance in school? The obvious answer is: Meeting the purposes and goals of the school. Though this may sound simplistic, the sequential question (What are the purposes and goals of the school?) is much more complicated.

My involvement in Cahill vs. Robinson, a New Jersey school finance court case, led me to pursue this issue. The New Jersey constitutional provision for a “thorough and efficient” system of education brought up the question of what a “thorough and efficient” system of education is.

After much soul searching, I advised the plaintiffs to stay away from determining what was “thorough and efficient” and to take a pragmatic approach in declaring that whatever “thorough and efficient” was, low wealth school districts were entitled to it. Unfortunately, my advice was not heeded, and the New Jersey courts spent years arguing about the purposes of education, a question that has not been resolved in more than 2,000 years.

In the legal challenge to the TAAS, the court will address the question of “performance.” Good performance is determined by success in meeting the goals of education. What are the goals of education? I believe there is consensus that the general goal of education is to bring the student to “maximum self-realization.”

Maximum self-realization means bringing the student to the apex of his or her potential in physical, intellectual, social, emotional and spiritual development.

Determining to what extent this goal is met presents some formidable problems. What is the student’s potential in each of the five areas? Who determines what that potential is? How is it to be measured? How is the determination that the potential has been met to come about?

In educational measurement there is little relationship between the determination of a student’s potential and the extent that it has been realized as a result of schooling. In at least four of the five areas of development, there has been little effort in even addressing the areas. Educational evaluation has mostly focused on the acquisition of skills and knowledge that is only indirectly related to the intellectual development of the child.

For a significant portion of students, the goals of education have been modified for social purposes into a substitute goal of having the student adhere to social conventions of behavior, be able to do simple academic tasks, such as reading, writing and arithmetic, and be able to follow orders. Not an impressive list of objectives in a democratic society.

Regardless of the limited idealistic or realistic goals of education being used, measurement of the attainment of these goals is grossly inadequate. In 1982, Texas legislated determinations of teacher pay on the basis of performance. Ironically, Texas adopted such a policy at the same time that other states were already dropping similar policies. Basing teacher pay on teacher performance is idiotic if it is not known what constitutes satisfactory teacher performance or how it is to be measured. At best, satisfactory teacher performance was based on providing a quiet, orderly environment in which it was assumed maximum learning could take place.

Achievement Measures

There has been considerable shifting of emphasis on what constitutes a good effort on the part of the system. During my own tenure as an educator, I have seen the emphasis shift among input (the resources provided), process (the use of resources) and output (the results of the process).

Input evaluation focuses on the provision of adequate resources for the instructional program. As a teacher and administrator, I participated in the accreditation process where the determination of appropriate instruction was indicated by an analysis of school inputs. The number of teaching hours, teaching subjects, library books, teacher credentials and other pre-instruction factors were the determinants of adequacy.

Even then, the determinants of input adequacy were not applied as absolutes since the accrediting agency, usually the Texas Education Agency (TEA), made a generous allowance in consideration of the great variance in resources available to different school districts in keeping with the inequitable system of school finance. It is inconceivable that the revelation that over 50 percent of the teachers in the Edgewood Independent School District (ISD) could not meet the minimum standards for certification by the state would lead to loss of accreditation. On the other hand, should the Alamo Heights ISD in another part of San Antonio provide a teaching force where 50 percent of the staff were ineligible for certification, certification would have been immediately revoked.

There was much flexibility in determining the minimum effort in the inputs of education. In general, what was considered inadequate inputs was dependent on the type of student being taught. What was sauce for the goose in hundreds of poverty school districts like Edgewood was not sauce for the gander in hundreds of high wealth schools districts.

In time, the focus in evaluation shifted from input to process. This transition was exacerbated by the emerging educational, social and legal questions being raised about the gross disparities in input. It is embarrassing to hold school systems responsible for realistic input when the state system of school finance makes no pretense of providing equitable or equal resources for acquiring these inputs.

Process evaluation based on an analysis of teacher-pupil interaction turned out to be no better than the input evaluation. Observation of instruction could not lead to valid evaluation if there was a large disparity in opinions as to what constitutes desirable and adequate instruction. Is the very well prepared lecture of a loquacious teacher adequate if the students in the class do not have an inkling of what the teacher is talking about? In many cases, observations in the classroom shifted from the teacher and student interaction, to the environment of the interactions. Were the bulletin boards attractive? Were the students orderly? How high was the noise level in the classroom? On more than one occasion I participated in evaluation training aimed at learning to distinguish between “productive” noise and “non-productive” noise in the classroom.

Determination of quality of instruction was so blurred that when legislation resulting from the Perot task force established pay incentives for “master” teachers, many school districts augmented the state incentive pay for master teachers with local incentive pay for the rest of the staff, so that the school systems reverted to a single salary pay schedule. Process evaluation reverted to input evaluation with master status being determined by teacher training, advanced degrees, years of experience or seniority without any objective observation of the teaching-learning process.

Then emphasis in evaluation shifted from process to output. The basic tenet of the current system is that input and process are relatively meaningless. What is important is the amount that the student learns. This is grossly unfair to the student. The student may have some impact on what is learned, but the student has no impact on input (the adequacy of resources) nor on process (the quality of instruction).

Determining a measure of output, or educational results, proved difficult since standardized achievement tests were based on a national assumption of what students are supposed to be taught in school. This lead to the development of an achievement test in Texas based on what students are supposed to be taught in Texas schools. The initial state examination was based solely on reading, writing and arithmetic, possibly assuming that these were the only subjects worth teaching but more probably that these were the only subjects in which learning could be measured. Over the years, learning areas in the state test have been expanded, most noticeably the inclusion of science and social studies in the current TAAS. But no attempt has been made to determine the parameters of education, let alone how success in all fields is to be measured.

Conducting evaluation solely on the basis of output measures in a limited number of fields led to the development of a test-driven curriculum. As the results of student performance in the limited fields were presented in the local and state media, educational instruction focused on the material expected to be tested by the state. The new slogan for Texas schools seemed to be, “If it is not on the TAAS, it is not worth teaching.” Schools striving to improve in the limited areas measured by the TAAS gave small consideration to music, art, physical education, socialization, civic responsibility or any other area not included in the state test. Concepts of maximum realization in physical, emotional, social and spiritual development may still be a part of the school goals, but they are seldom formally addressed in instruction.

Even intellectual development appears to have suffered under the current output evaluation. The early state tests addressed the lower forms of learning, so that the higher forms of learning were sacrificed for factual knowledge sure to be found in the state exam. Teachers focused on “who discovered what in what year,” rather than addressing the application of the social sciences to the solution of present social problems.

Recent revisions in the sate test have attempted to include the measurement of higher forms of learning, although an extensive attempt to do so is difficult, not only because the measurement of factual information is still around, but because the measurement of the higher orders of learning is difficult to accomplish in a multiple choice test item.

Since neither input, process nor output have proven to be adequate in evaluating student-teacher performance, where should the focus be placed? The obvious answer is the distribution of evaluation among all three. None of the three can be utilized without consideration of the other two. Past and present failures in evaluation cannot be attributed to the use of any of the three phases. The failure can be attributed to the focus on one of the phases to the exclusion of the other two.

Accountability

Determinations of student, teacher and system performance serve little purpose if inadequate performance is not to be addressed. Accountability is the determination of who is responsible for performance and what is to be done about it. Accountability should be closely related to the reward and punishment system of the operation.

The determination of accountability in education has always been a difficult concept. The teaching-learning process is one that conceivably can be controlled by both participants, the teacher and the student. This accounts for the rare incidence of educational malpractice litigation. The current focus on output (student performance) with little concern for input and process makes the question of accountability even more complex.

The present focus on output evaluation makes accountability a growing issue, particularly since education has become more of an imperative and the number of different, atypical and hard to teach students continues to grow. School systems and professional organizations have developed a line of defense that assumes that resources are adequate, everything done by the school is proper, and if a student fails to learn, the student and only the student must be held accountable. This may not be too different than the position taken in other fields such as medicine, except that the field of medicine has a scientific set of inputs and procedures to validate medical performance. Not only does education not have such a set of validating inputs and procedures, the shift of emphasis to outputs will preclude their development.

Educators are not prone to give attention to educational inputs and processes when it has been so easy to use the students as the scapegoat for educational failure.

The TAAS and Accountability

The use of the TAAS as a state administered achievement test to measure performance has led to the onus for unsuccessful performance being placed on the student. This was not unexpected. Minority groups and advocates of atypical studies fought hard against the implementation of such a test. A few others bought the concept that once the test was implemented it would be simple to determine unproductive school personnel and either retrain or replace them. This has seldom been the case.

It can be argued that there has been some amount of accountability as a result of the TAAS testing. This may be so, but the amount of accountability has been very limited. It is true that TAAS data has been used to identify underperforming schools, but the practice has had two severe limitations.

First, identification of underperforming schools was initially based on massive underperformance. When TAAS accountability was initiated, a poor performing school was one in which more than 80 percent of the students performed disastrously on one or more sections of the three-area test. I see no great development of educational insights when it is realized that less than 20 percent of a class, a school or a district cannot perform at the lowest level of the TAAS. This is no great breakthrough since performance on all other standardized tests that have been a requirement for decades in the accreditation process already showed such dismal performance.

Second, there have never been any individual sanctions associated with poor performance, nor is there any likelihood that such sanctions are forthcoming. In recent years there has been some unfavorable publicity about poor performance, although the bad publicity is usually directed at “the school” rather than at the staff.

Individual accountability has seldom occurred, and when it has, the penalty has consisted of teacher and administrator transfers, rather than dismissal. Transfers of poorly performing staff is a dysfunctional educational response. At best it is hoped that the incompetence of a large number of staff can be hidden by placement in other school settings so that the collective incompetence is not so visible.

If anything, TAAS accountability has led to the better performing schools becoming even better. In too few instances have the poor performing schools made a drastic improvement. Even in the few school districts where this has happened, it has been at the expense of outstanding administrators under severe harassment for “moving too fast,” “upsetting school staff,” “making drastic changes” and “traumatizing the community.”

The TAAS and Graduation

In spite of the limitations of the TAAS, some school systems have found it a convenient way of coping with other problems. A few school districts adopted the TAAS as a requirement for promotion. With complete disregard for the adequacy of school inputs and processes, and an equal disregard for characteristics and needs of children, educators implemented an accountability system that placed the onus for retention-in-grade entirely upon the student and the student’s performance on the TAAS. Years of educational research, warnings by test publishers and the advice of educators were set aside by defaulting on professional responsibility and allowing the standardized test to make critical educational decisions instead of using extensive information on what is best for the individual student.

The state did the same thing in requiring successful performance on the TAAS as a condition for graduation. Accountability is based solely on one output measure, student performance, without consideration of input and process.

It is impossible to produce valid measurement of student performance without considering input. Testing consists of a sampling of items taken from the curriculum presented to the student. It is assumed that the only variable on which performance is to be based is whether the student mastered the content and can respond positively to the sampling comprising the test. Any measurement based on content not available to the student is an invalid measurement of student performance. The use of the TAAS as a determinant of student performance must be preceded by assurance that the sampling making up the content of the test is consistent with the content of the curriculum afforded the student.

In Texas, making such an assumption is ridiculous. During the recent litigation on school finance there was an abundance of testimony presented by plaintiffs and accepted by the courts that low wealth schools were unable to provide course work that is used in the TAAS. Analysis of the post-Edgewood system of school finance indicates that although substantial improvement has been made, there are still wide disparities in resources available to low and high wealth schools. It is dysfunctional to measure student competence in a subject that was not taught during the years that the student was enrolled in school.

Advocates of student accountability by the use of the TAAS argue that course offerings are immaterial in determining student performance. The state has developed a minimum standard and each student is expected to meet that standard. This argument is ridiculous. Students denied diplomas because of their inability to pass the TAAS are being held accountable for the insufficiency of inputs in the Texas educational system.

Graduation

Should Texas continue to use the TAAS test as a criterion for high school graduation? The answer is an emphatic no! The following reasons demand that students who have completed graduation requirements be allowed to graduate regardless of performance on the TAAS.

• The test was not intended to be used as a measure of the completeness or adequacy of an educational program. It is poorly representative of the complicated comprehensive educational program, and test results must be combined with other factors to determine satisfactory performance.
• The TAAS does not have inherent validity – no test does. The validity of a test is determined by the specific situation in which it is used. Extensive factors influence the results of the test other than student proficiency. Student apprehensiveness, physical or mental disturbances, the environment in which the test is administered, distractions and many other factors may influence student performance.
• The TAAS test, and other psychological measurements, should not be used as a sole criterion for determining success or graduation. In past court cases governing the use of a test as a sole criterion, Texas has argued that a specific test is not a sole criterion if other requirements must also be met. The common use of the term “sole criterion” in educational literature denotes any criterion as “sole” if it is used in determining a decision regardless of what other criteria must be met. Since the TAAS precludes graduation and the awarding of a diploma regardless of other criteria, it is a sole criterion. A student with perfect attendance, completing all required courses, having the prescribed number of electives, making no grade lower than an “A” in high school, and attaining a perfect score on the Scholastic Aptitude Test (SAT) cannot graduate and receive a diploma if the TAAS score is below a prescribed level. The TAAS becomes a sole criterion for graduation.
• The TAAS is not a valid measure since there is no way that the content of the test can be adjusted to be representative of the curriculum experienced by the student. The types, amounts and qualities of school instruction for more than 3 million students are too varied to be represented in a standardized test.
• Considering the limitation of the TAAS and the inability to determine who is responsible for poor performance, the penalty on the student is too severe and too long-lasting. Students having met all graduation requirements other than passing the TAAS may spend the rest of their lives in a form of limbo. Retaking the test is not a solution, especially when a feasible avenue for remediation is difficult, if not impossible.
• The brunt of the penalty for any lack of achievement is borne by the student. Neither adequacy of input nor process figures in the determination of student accountability.

It is regrettable that advocates of students must once again resort to the courts to protect students from unfair and prejudicial educational practice.

Dr. José A. Cárdenas is the founder and director emeritus of IDRA. Comments and questions may be sent to him via e-mail at feedback@idra.org.

[©1998, IDRA. This article originally appeared in the October 1998 IDRA Newsletter by the Intercultural Development Research Association. Permission to reproduce this article is granted provided the article is reprinted in its entirety and proper credit is given to IDRA and the author.]