For further information please refer to Renzulli, J. S., & Reis, S. M. (1997). The schoolwide enrichment model: A how-to guide for educational excellence (2nd Ed.). Mansfield Center, CT: Creative Learning Press.
The Student Product Assessment Form
The Student Product Assessment Form (SPAF) was the result of a comprehensive instrument development research project (Reis, 1981) that was directed toward establishing the reliability and validity of this instrument and assessing the quality of products that were produced by various groups of students participating in programs for advanced ability students (click to see form). The validity and reliability of SPAF were established through a year long series of studies, using a technique developed by Ebel (1951). Levels of agreement among raters on individual items of the scale ranged from 86.4 percent to 100 percent. By having a group of raters assess the same set of products on two occasions, with a period of time between ratings, we established a reliability coefficient of .96 for the instrument. Information about the reliability of this instrument should be brought to the attention of decision makers in order to establish the credibility of an educator’s approach to the evaluation of student products. In other words, when questions about “hard data” and objectivity are raised, the fact that educators are using a research based instrument of proven value will help to overcome many of the concerns that traditionally are raised about the merits of various approaches to evaluation.
The instrument is composed of fifteen items designed to assess both individual aspects, as well as overall excellence of products. Each item reports a single characteristic on which raters should focus their attention. Items 1 through 8 are divided into three related parts:
- The Key Concept. This concept is always presented first and is printed in large type. It should serve to focus the rater’s attention on the main idea or characteristic being evaluated.
- The Item Description. Following the Key Concept are one or more descriptive statements about how the characteristic might be reflected in the student’s product.
- Examples. In order to help clarify the meaning of the items, an actual example of a student’s work is provided. These examples are intended to elaborate upon the meaning of both the Key Concept and the Item Description. The examples are presented after each item description.
Item 9 contains seven different components and details an overall assessment of the product. When completing the ratings for this assessment of a student’s product, raters attempt to evaluate the product in terms of their own values and certain characteristics that indicate the quality, esthetics, utility, and function of the overall contribution. In other words, raters are encouraged to consider the product as a whole by using their own judgment and relying upon their own guided subjective opinions when completing this component.
The results of product assessment should be summarized in the main body of an evaluation report. When this approach is used it is important to make the readers aware that the individual assessment forms, Management Plans, and actual products are available for their review. It is not necessary to evaluate every product for a formal evaluation. A stratified random sample (by grade level and various areas of student interest) can be used to provide a fair picture of the types of work that are being pursued in the special program. Whenever random samples are used, it is important to secure agreement (from boards or funding agencies) about sample sizes prior to deciding the actual number of products to be rated. It is also important to describe in detail exactly how a truly random and unbiased approach will be used to select products for rating.
Sharing the Student Product Assessment Form With Talent Pool Students
An almost universal characteristic of students of all ages is a desire to know how they will be evaluated or “graded.” We would like to begin by saying that we strongly discourage the formal grading of Type III endeavors. No letter grade, number or percent can accurately reflect the comprehensive types of knowledge, creativity and task commitment that are developed within the context of a Type III Enrichment endeavor. At the same time, however, evaluation and feedback are an important part in any educational experience and students should be familiar with evaluation procedures from the outset.
The best way to help students gain an appreciation for the ways in which their work will be evaluated is to conduct a series of orientation sessions organized around SPAF. Two or three examples of completed student products that highlight varying levels of quality on the respective scales from the SPAF instrument will help students gain an appreciation for both the factors involved in the assessment, as well as examples of the manifestation of each factor. In many ways these sessions represent an excellent way to teach students about the nature of a Type III Enrichment project and the difference between a traditional “report” on one hand and a first-hand investigative activity on the other. Over a period of several years, it would be a good idea to collect various examples of student products that highlight outstanding levels of accomplishment on one or more of the SPAF scales.
“Data” From the Management Plan
There are several important types of evaluation information that can be derived directly from an analysis of several Management Plans. Each of these types of information is consistent with the objectives for Type III Enrichment set forth on the summary sheet of this chapter. For example, one objective calls attention to student involvement in various interdisciplinary studies. By simply tallying the numbers of check marks in the “General Areas of Study” section from several Management Plans teachers can provide some factual (and even statistical) information about the variety of disciplines that can be found in Type III Enrichment projects. Similarly, the same objective refers to advanced levels of knowledge and methodology used within particular disciplines. Information relating to this objective may be obtained by analyzing several Management Plans for books and/or resource persons that students ordinarily would not come into contact with in regular curricular activities or through the use of ordinary school textbooks or library materials. Categorical tallies of intended audiences, products and outlets will help to highlight the ways in which students are achieving the objectives set forth for Type III Enrichment. Even the objective related to task commitment can be documented by simply presenting data about the average length of time that students spend on their Type III projects. This information coupled with Management Plans, notes and rough drafts of student work, Student Product Assessment Forms, the products themselves, and perhaps examples of sophisticated resources such as college level books, esoteric scientific equipment, computer software, etc. will provide a comprehensive and impressive array of evaluation information.
Additional Procedures for Evaluating Type III Enrichment
On the pages that follow, we have included additional forms and procedures that have been used to evaluate Type III Enrichment and provide parents with feedback about student involvement in other types of enrichment (see Figures 83-86). The information on these forms may be modified or combined in whatever ways teachers feel will be most effective in analyzing the ways in which their own program implements the Type III dimension of the SEM. Teachers also might want to make use of one or a combination of the numerous evaluation instruments that are included in the appendices of A Guidebook for Evaluation Programs for Gifted and Talented (Renzulli, 1975). This guidebook includes more than thirty instruments that have been specifically designed to evaluate various aspects of programs for advanced ability students. Taken collectively, the instruments can serve as an excellent reservoir of items from which teachers can select evaluation procedures that are most relevant to the Type III Enrichment dimension of their program, as well as other aspects of their overall programming efforts. As is always the case with any evaluation effort, the most important consideration is the degree to which evaluation instruments and procedures reflect the extent and quality of services that are unique to any given program or dimension thereof. For this reason, we recommend that teachers begin developing evaluation procedures by exploring a wide variety of available instruments and making selections based on the individual needs and points of focus in their own program.
References