A Multi Criteria System for the Identification of High Achieving and Creative/Productive Giftedness

Joseph S. Renzulli
University of Connecticut

Amy H. Gaesser
Purdue University

It is better to have imprecise answers to the right questions than precise answers to the wrong questions.
Donald Campbell
American Statistician

 

Major Considerations for the Development of Gifted Program Identification Systems

As scientific study advances our understanding of how human potential develops over the course of a lifetime, the systems created to enhance that potential (i.e. the education system) should also change to reflect contemporary theories. In the field of gifted education during the past several decades, research has supported a broadened, expanded conception of giftedness (Gardner, 1983; Renzulli, 1978; Sternberg, 1985). A thorough review of this research is beyond the scope of this article, (see Dai, 2010; Sternberg & Davidson, 2005), but to simplify a complex and active debate, very few researchers and theorists continue to accept an isolated IQ or achievement test score as a valid measure of a child’s capacity for producing notable accomplishments over the course of the lifetime. This does not mean that IQ or achievement scores should not be included as one of a number of criteria, only that they should not form the entire basis for decision making in identification for gifted and enrichment programs. In this article, we discuss the critical issue of having a cohesive relationship between the identification process and education programming for high ability students. Additionally, we review the Renzulli Identification System for Gifted Programming Services (RIS/GPS)—a comprehensive, evidence-based, and effective program that achieves this congruency. When we change to a true multiple criteria identification system, such as RIS/GPS, more services can be given to develop excellence in a greater number of students in an equitable and theoretically justified manner.

Districts just starting to develop gifted and talented programs and those with programs already in place both benefit from considering how to analyze the appropriateness of identification systems designed to select their students. The following questions constitute a starting point for reflection on the practical, political, and psychometric complexities of the issue:

  1. Will this identification system be applicable to diverse school populations and groups of students that have been traditionally underrepresented in programs for the gifted?
  2. How will we “label” students identified for these programs?
  3. Will the system be economical in terms of the personnel time, group and individual testing costs, and other resources necessary to identify our students?
  4. How much individual testing by school psychology staff will be required?
  5. Will the system be flexible enough to accommodate talent potentials across different domains such as music, art, drama, technology, and other non-verbal or mathematical talent areas?
  6. Will it be flexible enough to make changes if student performance warrants a reexamination of selection or rejection decisions?
  7. How will the system fit in with regulations of state departments of education (especially in those cases where some level of financial reimbursement is provided by state agencies for each identified gifted student)?
  8. How will the system help us avoid parental dissatisfaction or legal challenges?

Additionally, in any plan to identify gifted and talented students, six important considerations should be kept in mind:

Consideration 1: There is No Such Thing as a Perfect Identification System! There is no perfect way to identify who is or is not gifted, just as there is no single best way to develop giftedness and/or talent potentials. Every identification system is a “trade off” between the instruments and criteria selected, the ways we make decisions about any and all types of information we collect, and how much weight we give each type of information in the decision making process. The first and most important decision that should be made regarding practical procedures for identification is the conception or definition of giftedness adopted by a particular school system. In some cases, state regulations mandate the definitions that must guide identification and the number or proportion of students that can be served. There are programs, however, where additional students with high potential may be served if supported by local funds; and in such cases, this group may be designated by a label that is different from the state certified group designated as “the gifted” (e.g., Talent Pool, Advanced Learners, High Potential). Local circumstances notwithstanding, the conception or definition issue should be consistent with the types of services for which students are being identified (see Consideration 6 below). Any number of excellent resources exist that decision makers can consult in order to reach agreement on a conception/definition decision. A selected bibliography of the best resources to guide in this decision-making process is presented in Appendix A. It is recommended that decision makers examine and discuss these references to reach consensus before selecting or designing an identification system.

Consideration 2: The Objective vs. Subjective Trade-Off. The most frequently used type of identification information is tests of cognitive ability and/or academic achievement. These types of tests are considered objective because they rely on student performance rather than the judgment of others. Some people might question the objectivity of these tests because the decision to use them is, in and of itself, a subjective act or they have concerns about whether or not a one-hour “glimpse” into a young person’s overall potential can be considered an accurate appraisal of a student’s total capacity for high-level performance. Almost all other criteria (e.g., teacher, parent, peer, or self ratings, portfolio or writing sample assessments, or grades earned in school subjects) are considered to be subjective as their use implies personal judgments that may be open to personal bias, an idiosyncratic view of giftedness, or inconsistent grading standards. These types of criteria enable us to see other signs of potential, such as motivation, creativity, leadership and executive functions (initiation, execution, and completion of tasks), or intense interest in a topic not reflected in more objective cognitive ability tests. If we view some of these non-cognitive skills as important, then we need to examine the degree to which we are willing to make trade-offs between objective and subjective information.

Consideration 3: People—Not Instruments—Make Decisions. Regardless of the number or types of instruments used in a multi-criteria identification system, instruments only provide selected sources of information, instruments do not make decisions! These team members (e.g. teachers, program coordinators, school psychologists, district liaisons) may need different levels of orientation and training to become well-informed evaluators. Protocols for resolving differences of opinion that will invariably emerge can be structured in advance, reducing the need for ad hoc solutions to team member disagreements. How much “weight” will be given to the various instruments or decision-making criteria should also be determined before implementing the identification system. For example, if a decision is made to use three cognitive ability measures (e.g., aptitude test, achievement test, and course grades1), and only one measure of creativity (e.g., a creativity test or a teacher rating), there will be triple weighting of cognitive ability and single weighting of the creativity criterion. The relative emphasis on different sources of information should be aligned with the overall intent of the program. This consideration is important in both the design of the identification system and the interpretation of the information provided to the committee who will review students’ records and subsequently make decisions.

Consideration 4: Avoid The Multiple Criteria Smokescreen. Most identification systems utilize a traditional nomination/screening/selection approach, and at least part of any multiple criteria screening process is usually based on non-test information (e.g., teacher nominations and/or ratings). A problem arises, however, if the nomination or screening process only determines which students will be eligible to take an individual IQ test or a more advanced cognitive ability test. In such cases, a teacher nomination or high ratings is only used as a “ticket” to take the individual or group ability test. The test remains the ultimate “gatekeeper” for which students enter or do not enter the program, as the score is still ultimately the deciding factor. Any highly positive attributes that might have been the basis for a teacher nomination, or favorable information discovered in the screening process, are totally ignored when it comes to the final selection decision. The danger here is, of course, that we may be systematically excluding high potential students from culturally diverse backgrounds or students who have shown signs of high potential in other than the high verbal, mathematical, or analytic skills measured by standardized tests. What appears to be a multiple criteria approach ends up being a smokescreen for a more traditional cut-off score approach.

Consideration 5: What Will We Call Selected Students? A fifth consideration emerges from some of the considerations discussed above and relates to the degree of specificity that we are attempting to achieve in the identification process. The tradition has been simply to label all selected students as “the gifted;” thereby relegating all others to a non-gifted category. In recent years, however, a large body of research has argued very forcefully against such a broad stroke labeling process (Frasier & Passow, 1995; Gardner, 1983; Renzulli & Reis, 1997, 2014; Sternberg, 1985; Winner 1996) and in some cases recommendations have been made to do away with any labeling altogether (Borland, 2004). A more current trend is to document specific student strengths by preparing an electronic multiple criteria profile (Field, 2009; Renzulli & Reis, 1997, 2014). This strength-based profile can be used for making more personalized decisions about the types of resources and activities recommended for talent development.

Of course, labeling of any kind is always a controversial issue. In recent years, an approach that has gained in popularity is to label the service rather than the student (Renzulli & Reis, 1994, 1997, 2014). For example, in a school utilizing the Schoolwide Enrichment Model, a special service offered to all students called an “enrichment cluster” enabled any interested students to participate in a class entitled Statistical Techniques for Young Researchers. This class was specifically designed for upper elementary students with strong aptitudes and interests in mathematics. Students—without needing to be labeled themselves—could benefit from material that was much more advanced than the math being covered in their sixth, seventh, and eighth grade math classes.

Another example of a labeled service is Curriculum Compacting (Reis & Purcell, 1993; Reis & Renzulli, 2005; Reis, Westberg, Kulikowich, & Purcell, 1998), which is a within-the-regular-classroom process that teachers use for students who have already mastered the concepts and skills to be taught in a given unit of instruction, and/or who are capable of covering the regular material at a faster pace and higher level of comprehension than their classmates. This process involves specific procedures for identifying particular strength areas, documenting these competencies in a systematic fashion, and providing advanced level enrichment and/or acceleration opportunities with the time gained from eliminating already mastered material.

Consideration 6: The Relationship between Identification and Programming. Our final consideration addresses the congruence between the criteria used in the identification process and the goals and types of services that constitute the daily activities of students in a special program. Congruence between identification and programming is so important that it might be viewed as “the golden rule” of gifted education! For example, identification for advanced courses in a content area such as math is best accomplished through math testing, examination of previous math grades, teacher recommendations or ratings on mathematical skills, and perhaps even estimates of a student’s motivation to work hard in math. A problem arises, however, when we expect an “all purpose” gifted program to develop strengths that are unique to each child. If a general gifted program has a curriculum, or if individual teachers in the program decide most of the activities (e.g., the teacher’s favorite Rain Forest Unit or play production), then little room exists for variations in students’ interests, learning styles, or preferred modes of expression. In other words, the materials covered in the general gifted program may be different from the regular curriculum, but the prescriptive nature of what is to be learned uses essentially the same approach to teaching used in regular classrooms. Therefore, a related decision in developing an identification system is the selection of a pedagogical programming model that will be used to guide direct and indirect services to students regardless of how they are grouped or organized for special program services. In this case, we are not discussing organizational models, but rather what the teaching/learning process looks like within any predetermined organizational arrangement.

Again, there are numerous programming models recommended for serving this population, and these programming models can be divided into two categories. Organizational or administrative models address how we group students and move them from one activity to another (e.g., full-time classes, pull out programs, centers where students go for a given period of time each week, regular class inclusion approaches, to mention only a few). Theoretical or pedagogical models focus on the kind and quality of learning experiences that are offered within any grouping or organizational arrangement. The Enrichment Triad Model (Reis & Renzulli, 2003; Renzulli, 1977, 1988), the Autonomous Learner Model (Betts, 2009), and a variety of acceleration, problem-based learning, and Socratic reasoning approaches are examples of theoretical or pedagogical models. An excellent resource for examining the range of programming options can be found in Systems and Models for Developing Programs for the Gifted and Talented (Renzulli, Gubbins, McMillen, Eckhart, & Little, 2009).

By way of summary here, the six considerations discussed above point out the “landscape” surrounding the always complicated and frequently controversial topic of identifying gifted and talented students for services. This discussion of the issues will not provide ready-made answers to the many challenges of identification system design, but it does provide an understanding of some historically encountered problems that may be helpful in avoiding the pitfalls faced by so many persons who have set out on the journey of creating an efficient, effective, and equitable plan for identification. Following, we present one such identification model.

The Three-Ring Conception of Giftedness

The Three-Ring Conception of Giftedness (Renzulli, 1978, 1986, 2005; see Figure 1) was purposefully designed for a programming model that develops both academic or high achieving and creative-productive types of giftedness (Renzulli & Reis, 1994, 1997, 2014) as both of these types of giftedness are important and often interact, and both should be encouraged in special programs. This identification model is supported by decades of research that indicates three interlocking clusters of ability that characterize this group (Baum, Renzulli, & Hébert, 1994; Gubbins, 1995; Reis & Renzulli, 1982). The name derives from the conceptual framework of the theory – namely, these three interacting clusters of traits 1) Above Average but not necessarily superior ability as measured by cognitive ability and achievement tests, 2) Task Commitment, and 3) Creativity, and their relationship with general and specific areas of human performance.

3-RingWithHoundstooth
The hounds tooth background represents personality and environment factors that give rise to the three clusters of traits.
Figure 1. The three-ring conception of giftedness.
 
Perhaps the most salient aspect of this theory is that it is the interaction among these clusters of traits brought to bear upon a particular problem situation and/or performance area that creates the conditions for the creative productive process to begin. Additionally, these clusters of traits emerge in certain people, at certain times, and under certain circumstances. The Enrichment Triad Model (Reis & Renzulli, 2003; Renzulli, 1977, 1988) is the compatible learning theory from which we promote educational circumstances that create the conditions for stimulating interaction between and among the three rings as described below.

Above Average Ability encompasses both general (e.g., verbal and numerical reasoning, spatial relations, memory) and specific (e.g., chemistry, ballet, musical composition, experimental design) performance areas and is the most constant of the rings. That is, any student’s performance within the parameters of this ring is minimally variable, as it is linked most closely with traditional cognitive/intellectual traits. The reason that this ring makes reference to “above average ability” (as opposed to, for example, “the top 5%” or “exceptional ability”) derives from research suggesting that, beyond a certain level of cognitive ability, real-world achievement is less dependent upon ever increasing performance on skills assessment than upon other personal and dispositional factors (e.g., Task Commitment and Creativity; Renzulli, 1978, 1986, 2005). The limitations of intelligence tests and innumerable aptitude and achievement tests that are used to identify candidates for “gifted programs.”

Task Commitment represents a non-intellective cluster of traits found consistently in creative productive individuals (e.g., perseverance, determination, will power, positive energy), and what some contemporary authors (Duckworth, 2009) are currently calling “grit.” It may best be summarized as a focused or refined form of motivation—energy brought to bear on a particular problem or specific performance area. The significance of this cluster of traits in any definition of giftedness derives from myriad research studies, as well as autobiographical sketches of creative productive individuals. Simply stated, one of the primary ingredients for success among persons who have made important contributions to their respective performance areas is their ability to immerse themselves fully in a problem or area for an extended period of time and to persevere even in the face of obstacles that may inhibit others.

Creativity is that cluster of traits that encompasses curiosity, originality, ingenuity, and a willingness to challenge convention and tradition. For example, there have been many gifted scientists throughout history, but the scientists whose work we revere, whose names have remained recognizable in scholarly communities and among the general public, are those scientists who used their creativity to envision, analyze, and ultimately help resolve scientific questions in new, original ways.

A frequently raised question is: Must Creativity and Task Commitment be present in order for a person to be considered “gifted?” In the study of human abilities, traditionally measured achievement tends to remain constant over time (indeed, this is the reason for the high reliability of cognitive ability and achievement tests). Task Commitment and Creativity, on the other hand, are not always present or absent; rather, they come and go within certain contexts and circumstances that are the result of experiences and effective teaching that promotes these traits. Creativity and Task Commitment, unlike traditionally measured academic achievement traits included in the Above Average Ability circle, are developmental and therefore subject to the kinds of experiences provided for both young people and adults. They are the result of the kinds of opportunities, resources, and encouragement that are provided to spark a creative idea or develop the motivation that causes a person or group to want to follow through on the idea.

In many cases, Creativity and Task Commitment “feed” upon one another. For example, a person notices something happening that draws his or her attention to a problem that needs to be addressed (e.g., bullying in their school). They become interested and develop the Task Commitment to do something about it. They may then begin to explore various creative ways to start an awareness campaign, do a questionnaire study about bullying, obtain a video to be shown to the students in their school, or prepare some posters or discussion groups that address this issue.

The reciprocal relationship between Creativity and Task Commitment may also work in the opposite direction. A group of students may, for example, have a creative idea about raising money to buy school supplies for poor children by baking and selling cookies. They must now develop their Task Commitment to actually get the job done. Task Commitment requires the time, energy, and the organizational and management skills necessary for their creative idea to become a reality.

The Three-Ring Conception of Giftedness is based on an overlap and interaction between and among these three clusters of traits that create the conditions for developing and applying gifted behaviors. Giftedness is not viewed as an absolute or fixed state of being (i.e., “…you have it or you don’t have it”). Rather, it is viewed as a developmental set of behaviors that can be applied to problem solving situations. Varying kinds and degrees of gifted behaviors can be developed and displayed in certain people, at certain times, and under certain circumstances. In a certain sense, we might view the most important role of teachers is to provide young people with the opportunities, resources, and encouragement to generate creative ideas and the skills necessary to follow through on their ideas. In other words, in the Above Average Ability population, our most important goal is to create the Creativity and Task Commitment traits specified in the Three-Ring Conception of Giftedness and to bring the circles together in order for gifted behaviors to coalesce and make something happen. This identification model is a cornerstone of the Renzulli Identification System for Gifted Program Services (RIS/GPS; Renzulli & Reis, 2012).

To best support effective implementation of gifted programming, there should be congruence between the criteria used in the identification process and the goals and types of services that constitute the day-to-day gifted program’s activities in which students will be involved, as well as a link between a broad range of services and teaching practices that are specifically designed to develop a variety of talents in young people. Another critical consideration as outlined above is our firm belief that we should label the services rather than the students, enabling teachers to document specific strengths and use this information to make decisions about the types of activities and the levels of challenge that should be made available.

The RIS/GPS incorporates these important factors. It recognizes students with undiscovered potential and provides opportunities to develop their talents through an integrated continuum of special services, allowing for the identification of students who would benefit from services that recognize both academic and creative-productive giftedness. A key feature within this identification system is the formation of a Talent Pool that includes students who have been identified by both test and non-test criteria. The system includes students who earn high scores on traditional measures, but also leaves room for students who show their potentials in other ways or those who have high academic potential but underachieve in school.

In districts where this system has been implemented, students, parents, teachers, and administrators have expressed high degrees of satisfaction with this approach. By eliminating many of the problems usually associated with the identification of gifted students, we gain support from teachers and administrators, and by expanding services to students below the top few percentile levels usually admitted into special programs and those students who gain entrance by non-test criteria, we eliminate sometimes justifiable criticism of persons about entrance being denied to students who are in need of special opportunities, resources, and encouragement. This identification system is not as tidy as using cut off scores, but it is a more flexible approach to identifying and serving young people with great potential.

Implementation: The Nuts and Bolts of the RIS/GPS

Now that we have reviewed the research, presented the evidence, and introduced the key considerations, we hope that you agree that a multiple criteria identification system is preferable to a traditional system. What is next? The following section will outline a pragmatic approach to implementing such an identification system. The diagram (Figure 2) below forms the basis for the step-by-step process to select students for services based on multiple sources. After following the steps in the RIS/GPS, identification team members can assemble a “Talent Pool” comprised of the students who have been identified through multiple ability/achievement scores, teacher ratings, parent ratings, peer ratings, and self-nominations.

Identification_System
Figure 2. The Renzulli identification system for gifted program services (RIS/GPS)
 
Decisions about Talent Pool Size
Deciding on the size of the talent pool is a function of two major decisions. The first is the number of special program personnel assigned to the program and the number of students that these personnel can provide adequate services to each week in such a manner that it makes a difference in the accomplishment of program goals. The second decision is the nature and extent of an expanded range of services that will be made available to targeted students by classroom teachers (e.g., Curriculum Compacting, Enrichment Clusters, Mentorship Programs for advanced students). Services such as Robotics Club, History Day Competition, Math League, Music, Art, and Drama clubs, or any other organized interest-based grouping that focuses on a specific talent area falls within the scope of most special program goals. These types of opportunities reflect a total school talent development perspective, and they are especially valuable for a student or small group that has a high degree of potential, but only in a particular area of interest. It is important to convey to parents that this expanded range of services is, in fact, part of the special program opportunities that fall under the purview of the gifted program.

This second decision about an expanded range of services also has implications for special program administrative personnel. If we expect classroom teachers to participate in the services mentioned above, and if we hope to offer a robust range of extra-curricular activities geared toward talent development, it is essential to have a program coordinator that plans and “grows” such services, monitors the effectiveness of the services, maintains student records, and communicates talent development progress with parents. All teachers involved in the expanded range of services should believe they are an integral part of the program rather than a random provider of an extracurricular activity. They should be aware of the program mission and goals, participate in staff development that focuses on talent development, and attend “gifted program” meetings. Their accomplishments should be described in program brochures, reported in program announcements and newsletters, and recognized in special events about program activities. The program coordinator helps to create an expanded range of services that are an essential part of a total talent development program.

The RIS/GPS respects and includes students who earn high scores on traditional measures of cognitive ability, but a major variation from traditional identification practices is that this system “leaves some room” in the Talent Pool for students who show their potentials in other ways. The percentage of total students in the Talent Pool and the corresponding proportions of students identified through test and non-test criteria can and should be modified based on the resources and goals of the individual program involved.

Steps in Forming the Talent Pool

A team of school personnel including teacher(s) of the gifted, classroom teachers, administrators, and pupil personnel specialists (e.g., counselor, school psychologist, social worker) should be responsible for managing the Talent Pool selection process. This group can be thought of as the Review and Selection Team. Any and all information related to the selection process should be made available to all members of the team and a case study approach should be used to review each set of student records. On some occasions, it will be necessary to seek supplementary information about a student and to request that non-team members meet with the team to provide supplementary information. It is important for all persons on the team (and parents and the general faculty as well) to understand that instruments provide information but people make decisions! A multiple criteria approach means that simply setting arbitrary cut-off points or adding up points from various instruments cannot make decisions. Informed human judgment is crucial for an identification system that: (1) seeks to develop diverse talent potentials in diverse segments of the school population; and (2) is geared toward services that place a premium on developing creative productivity rather than merely advanced lesson learning.

Step 1: Academic Performance and Test Score Nominations
Academic performance based on end-of-year grades for the past two years and the most recent total verbal and total numerical scores from district-wide achievement tests are the first two criteria used in forming the Talent Pool. In a 15 percent Talent Pool example, students who score at or above the 92nd percentile on either verbal or numerical sections of the achievement test should automatically be placed in the Talent Pool. In schools that serve diverse populations, it is also recommended that a non-verbal cognitive ability test be used in addition to standard achievement tests or aptitude tests.

A very big caution, however, is in order here! There is a good deal of controversy about the effectiveness of non-verbal tests for increasing the proportion of minority students in programs for the gifted (Lohman, 2005; Naglieri & Ford, 2003, 2005). Until more definitive studies are conducted, we should treat non-verbal test scores as another piece of information in the overall decision-making process rather than a substitute for regular cognitive ability tests and school performance.

Lohman (2005) further argues that comparisons should only be made between students who share similar learning opportunities or background characteristics. It is for this reason that this identification system recommends the use of local norms (i.e., calculated by school and grade level). Our goal is to identify the most promising students in each school and at each grade level who are the best candidates for supplementary services. Since we are not admitting students from other school districts or states, it does not make sense to engage in national comparisons! The use of national norms invariably results in the under representation of minorities and students whose potentials may be manifested in non-traditional ways.

Students who score below the 92nd percentile, but who have demonstrated “straight A” academic performance in their end-of-year grades should also be considered eligible for gifted program services unless the selection team notes unusual discrepancies between test scores and grades. Or there may be cases where high-scoring students do not have high grades due to underachievement or personal or social issues. In such cases, before determining which services are appropriate, additional individual assessment and record review should be carried out to determine if factors such as underachievement, a learning disability, personal or family problems, or difficulty with timed group tests is giving an inaccurate picture of the student’s potential. Individual intelligence tests administered by a qualified examiner are needed when discrepancy information is found in the types of assessment mentioned above. This approach will help to control the expensive and time-consuming use of individualized testing, thereby meeting the economy goal of this identification system.

Scores from the most recent regularly administered standardized achievement or aptitude test can be used for this purpose; however, we recommend that admission to the Talent Pool be granted on the basis of either a high verbal or a high mathematics score. This approach will enable students who are high in verbal or mathematical ability (but not necessarily both) to gain admission. Programs that focus on special talent areas such as music, art, drama, or leadership should use non-test criteria (see Step 2) as major indicators of Above Average Ability in a particular talent area. In a similar fashion, whenever test scores are not available, or we have some question as to their validity, the non-test criteria recommended in the following steps should be used. This approach is especially important when considering primary age students, disadvantaged populations, or culturally and linguistically different groups.

The conclusion of Step 1 should be the creation of a list of names with an approximately equivalent number of students selected from each grade level. Through team discussions and negotiations, this list should represent approximately one-half of the predetermined number of “slots” in the Talent Pool.

Step 2: Teacher Nominations
If we were using nothing but test scores to identify a 15 percent Talent Pool, the task would be ever so simple. Any child who scores above the 85th percentile (using local norms) would be placed in the Talent Pool. In this identification system, however, we have made a commitment to “leave some room” for students whose potentials may not be reflected in standardized tests. This approach guarantees that all traditionally bright youngsters will automatically be selected, and they will account for approximately 50 percent of our Talent Pool. This process also guarantees admission to bright underachievers.

In order to minimize paperwork on the parts of classroom teachers, the first activity in Step 2 is to provide classroom teachers with a list of the names of students from their class who have already been selected for the Talent Pool in Step 1. After being provided with a brief training activity on the use of teacher rating forms, teachers are asked to complete ratings on students other than those already selected in Step 1 whom they might consider for admission to the Talent Pool. In other words, teachers should be informed about all students who have gained entrance through test score nominations, so that they will not have to complete ratings for students who have already been admitted. Step 2 allows teachers to nominate students who display characteristics that are not easily determined by tests (e.g., high levels of creativity, task commitment, unusual interests, talents, or special areas of superior performance or potential).

The instrument recommended for teacher ratings is the Scales for Rating the Behavioral Characteristics of Superior Students (SRBCSS; Renzulli, Smith, White, Callahan, Hartman, & Westberg, 2002). These scales are the most thoroughly researched and widely used teacher-rating instrument in the world (Renzulli, Seigle, Reis, Gavin, & Systma Reed, 2009). The scales are now available from Routledge (https://www.routledge.com) in an online version, which allows for ease of rating, and more importantly (because this system recommends the use of local norms), the online version automatically calculates local norms as well as individual student profiles.

Most schools use the three main scales corresponding to the Three-Ring Conception of Giftedness (i.e., Learning, Motivation, and Creativity); however, additional scales are available for programs seeking ratings for special areas of talent or for nominating students who might be the best candidates for categorical programs, such a Future Problem Solving, Web Quest, or MathCounts. In such cases, one or a combination of the following SRBCSS scales might be used: Leadership, Reading, Mathematics, Science, Technology, Music, Art, Drama, Communication; Precision, Communication; Expressive, and Planning. Once again, local norms based on school and grade level ratings are used rather than state, regional, or national norms; and each scale is considered a categorical data point. In other words, scores from the scales should never be added together or averaged.

With the exception of teachers who are over-nominators or under-nominators, nominations from teachers who have received training in this process are accepted into the Talent Pool on a par value with test score nominations. We do not refer to students nominated by test scores as the “truly gifted,” and the students nominated by teachers as the moderately or potentially gifted. Nor do we make any distinctions in the opportunities, resources, or services provided, other than the normal individualization that should be a part of any program that attempts to meet unique needs and potentials. Special programs should first and foremost respect and reflect the individual characteristics that brought students to our attention in the first place.

In cases of teachers who are over-nominators, the selection team can and should request that teachers rank order their nominations for review (i.e., place the scales in a pile from high to low) and return them to the selection team. Procedures for dealing with under-nominators or non-nominators will be described in Step 4.

Step 3: Alternate Pathways
Most schools using this identification system make use of test scores and teacher nominations, and in most cases, the majority of the Talent Pool will come from these two criteria. Alternate pathways are optional, locally determined by individual schools, and pursued in varying degrees by individual school districts. Alternate pathways generally include parent nominations, peer nominations, self-nominations, specialized tests (e.g., creative writing, spatial or mechanical ability), product evaluations, or virtually any other procedure that might lead to initial consideration by a selection team. A large number of instruments for gathering alternate pathway information are available in the identification literature. A good source of information about traditional testing instruments can be found in Assessment of Children: Cognitive Applications (Sattler, 2001); and reviews of instruments specifically related to gifted programs can be found in Instruments Used in the Identification of Gifted and Talented Students (Callahan, Hunsaker, Adams, Moore, & Bland, 1995). It is, of course, important and ethically responsible for teachers to make use of the findings resulting from all procedures and the identification information should always be shared with classroom teachers and periodically monitored to determine if appropriate attention is given to information about special interests or activities.

The major difference between alternate pathways on one hand (Step 3), and test score and teacher nomination on the other (Steps 1 and 2), is that alternate pathways are not automatic. In other words, students nominated through one or more alternate pathways will become the subjects of a case study by the Review and Selection Team, after which a selection decision will be made. In most cases the team carries out a case study that includes examination of all previous school records, interviews with students, teachers, and parents, and the administration of individual assessments (as needed) that may be recommended by the team. In some cases, students recommended on the basis of one or more alternate pathways can be placed in the Talent Pool on a trial basis.

A local planning committee or the Review and Selection Team should make decisions about which alternative pathways might be used. Some consideration should also be given to variations in grade level. For example, self-nomination is more appropriate for students who may be considering advanced classes at the secondary level. Peer nomination is particularly useful for program services that focus on particular talent areas such as technology, music, or drama; and students themselves are sometimes better at revealing which students have natural or “street smart” leadership potential.

Step 4: Special Nominations (Safety Valve No. 1)
Special nominations represent the first of two “safety valves” in this identification system. This procedure involves preparing grade level lists of all students who have been nominated through one of the procedures in Steps 1 through 3 and circulating these lists to all previous year teachers. The directions sent with the lists are as follow:

“These lists contain the names of all students who have been nominated for the Talent Pool for the forthcoming year. Will you please review the lists and send us the names of any students you have previously taught that are not on the lists, but that you think should be considered for Talent Pool membership.”

Teachers should not be required to give a reason for their special nominations at this time. Busy schedules may discourage teachers from preparing justifications “on the spot.” A later meeting or request that teachers complete a set of rating scales can also help to insure that invitations for special nominations are not ignored by busy teachers.

This procedure allows previous-year teachers to nominate students who have not been recommended by their present teacher, and it also allows gifted education teachers to make recommendations based on their own previous experience with students who have already been in the Talent Pool, or students they may have encountered as part of enrichment experiences that have been offered in regular classrooms. This process also allows special topic teachers (e.g., music, art, physical education) or teachers who have had responsibilities for special programs (e.g., Future Problem Solving, National History Day, etc.) to have opportunities for input into the nomination process. These teachers often observe students in non-traditional learning environments, and therefore they are excellent talent scouts for a variety of creative, practical, and motivational strengths. Faculty orientation about such opportunities is, of course, very important for gaining such input.

The Special Nomination step allows for a final review of the total school population, and is designed to circumvent the opinions of present year teachers who may not have an appreciation for the abilities, styles, or even the personality of a particular student. This one last “sweep” through the population also helps to pick up students that may have “turned-off” to school or developed patterns of underachievement as a result of personal or family problems. This step also helps to overcome the general biases of any given teacher who is an under-nominator or a non-nominator. As with the case of alternate pathways, special nominations are not automatic. Rather, a case study is carried out and the final decision rests with the Selection Team.

Step 5: Notification and Orientation of Parents
A letter of notification and a comprehensive description of the program should be forwarded to the parents of all Talent Pool students indicating that their youngster has been placed in the Talent Pool for the year. The letter does not indicate that a child has been certified as “gifted,” but rather explains the nature of the program and extends an invitation to parents for an orientation meeting. At this meeting, a description of the Three-Ring Conception of Giftedness should be provided, as well as an explanation of the differences between “high achieving giftedness” and “creative productive giftedness.” It is important to emphasize that both types of giftedness are important and will be addressed in the program. What should also be emphasized is that creative productive giftedness is the type that represents the way that the larger society has recognized persons of significant accomplishment (Treffinger & Renzulli, 1986).

The meeting with parents should also provide an explanation of all program policies, procedures, and activities. Parents are informed about how admission to the Talent Pool is determined; that selection is carried out on an annual basis, and that changes in Talent Pool membership might take place during the year as a result of evaluations of student participation and progress. Parents are also invited to make individual appointments whenever they feel additional information about the program in general, or their own child, is required. A similar orientation session should be provided for students, with emphasis once again being placed on the services and activities being provided. Parents are not told that their children are “the gifted,” but through a discussion of the Three-Ring Conception and the procedures for developing general and specific potentials, they come to understand that the development of gifted behaviors is a program goal, as well as part of their own responsibility. Schools using the Renzulli Learning System (RLS) should provide parents with a copy of their child’s electronically generated profile.

Step 6: Action Information Nominations (Safety Valve No. 2)
In spite of our best efforts, this system will occasionally overlook highly creative students or students talented in a specific area, who, for one reason or another, are not selected (but should have been) for Talent Pool membership. To help overcome this problem, a process called Action Information Nomination is used and all teachers are provided with an orientation related to spotting unusually favorable high-interest topics in the regular curriculum.

Action information can best be defined as the dynamic interactions that occur when a student becomes extremely interested in or excited about a particular topic, area of study, issue, idea or event that takes place in school or the non-school environment. It is derived from the concept of performance-based assessment, and it serves as the second safety valve in this identification system. The transmission of an Action Information Message (AIM; see Appendix B) does not mean that a student will automatically be placed in the Talent Pool. It does, however, serve as the basis for a careful review of the situation to determine if any types of special services are warranted. AIMs are also used within Talent Pool settings (i.e., pull-out groups, advanced classes, cluster groups) to make determinations about the pursuit of individual or small group investigations (Type III Enrichment in the Triad Model). In order for the Special Nomination process to work effectively, all school personnel should be provided with an orientation to “talent spotting” situations where the initiation and transmission of an AIM may be warranted. Transmission to the Review and Selection Team or to someone in the school and/or community that might provide guidance, serve as a mentor, or help the student to follow up in his or her area of interest are obligations that accompany the use of AIMs in our effort to leave no stone unturned is helping young people develop their potential talents. In programs based on the Schoolwide Enrichment Model (Renzulli & Reis, 1997, 2014), we also provide a wide variety of in-class enrichment experiences that might result in recommendations for special services through the Action Information process.

Processing Identification Information: Keeping it Organized and Communication-Friendly

Despite our initial admonitions against emphasizing administrative “tidiness” at the expense of multiple sources of data identifying young people’s talents, it is nonetheless important to keep all sources organized in a coherent manner that enhances communication among stakeholders. We recommend placing a summary sheet (Figure 3) at the very top of each student’s file. This allows a concise condensation of the multiple measures used in the identification process that is clearly visible to anyone who accesses the information.

Another possible way to summarize multiple criteria into a meaningful format for decision-making is to use the following steps, developed by Lohman and Renzulli (2007) This process incorporates verbal, quantitative, and non-verbal CogAT scores, math and reading achievement scores, and SRBCSS Learning Ability, Creativity, and Motivation scales in the review and selection process.

Renzulli Identification System: Information Summary Form

 
Name: ______________________________________________________     Date: __________________
School: _____________________________________________________     Grade: _________________
 
I. Academic Performance
     A. Academic Test Scores (Most Recent Test Scores)

 
Test
Date
Raw Score
Grade Equiv.
Local %ile
Verbal
 
 
 
 
 
Numerical
 
 
 
 
 
Non-Verbal
 
 
 
 
 


 
     B. End of Year Grades for Past 2 Years

Subject
Year 1
Year 2
Subject
Year 1
Year 2
Reading
 
 
Music
 
 
Mathematics
 
 
Art
 
 
Lang Arts/English
 
 
Foreign Lang
 
 
Social Studies
 
 
Other: __________________________________
 
 
Science
 
 
Other: __________________________________
 
 


 
II. Teacher Ratings [Scales for Rating Behavioral Characteristics of Superior Students (SRBCSS)]

Scale
Score
Group Mean
Scale
Score
Group Mean
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 


 
III. Alternative Pathways

 
Scale
Summary of Strengths
Parent Rating
 
 
Peer Rating
 
 
Product Rating
 
 

 
IV. Special Nominations
 
Teacher: _____________________________________________________
Grade: _______________________________________________________

Attach a brief description from the nominating teacher about why this student was nominated and enter the SRBCSS ratings in Part II above.
 

Figure 3. Identification summary sheet
 
In closing we would again point out that simplistic single-score identification systems cannot provide us with the rich information necessary in making decisions on how to best provide services to develop children’s unique talents and gifts. Choosing to implement a multiple criteria identification system harnesses the best theoretical evidence about talent development across the lifespan. It also provides avenues for traditionally under-represented student populations to participate in special programming, thus enhancing social equity. This article details how implementing such a system is not only desirable, but practically feasible as well. As educators move to the implementation stage of any decision-making innovation process, we hope this article has provided a practical roadmap as well as resources to guide a successful implementation of a flexible and fair identification system. We believe that the focus of tradition and expediency that has characterized gifted program identification must give way to expanded conceptions and innovative approaches to identification. These expanded approaches may not be as “tidy” or expedient as past practices, but they will help our field fulfill its promise of developing outstanding talent in more young people and increasing society’s reservoir of creative and productive adults.

References
Baum, S. M., Renzulli, J. S., & Hébert, T. P. (1994). Reversing underachievement: Stories of success. Educational Leadership, 52(3), 48-52.
Betts, G. T., & Kercher, J. J. (2009). The Autonomous Learner Model. In J. S. Renzulli, E. J. Gubbins, K. S. McMillen, R. D. Eckert, and C. A. Little (Eds.), Systems and models for developing programs for the gifted and talented (2nd ed., pp. 49-103). Mansfield, CT: Creative Learning Press.
Borland, J. (2004). Issues and practices in the identification and education of gifted students from under-represented groups (RM04186). Storrs: University of Connecticut, The National Research Center of the Gifted and Talented.
Callahan, C., Hunsaker, S., Adams, C., Moore, S., & Bland, L. (1995). Instruments used in the identification of gifted and talented students (Research Monograph 95130). Storrs: University of Connecticut, The National Research Center on the Gifted and Talented.
Dai, D. Y. (2010). The nature and nurture of giftedness: A new framework for understanding gifted education. New York, NY: Teachers College Press.
Delcourt, M., Dewey, C., & Goldberg, M. (2007). Cognitive and affective learning outcomes of gifted elementary school students. Gifted Child Quarterly, 51, 359-381.
Field, G. B. (2009). The effects of using Renzulli Learning on student achievement: An investigation of internet technology on reading fluency, comprehension, and social studies. International Journal of Emerging Technology, 4, 29-39.
Frasier, M. G., & Passow, A. H. (1995). A review of assessment issues in gifted education and their implications for identifying gifted minority students (Research Monograph 95204). Storrs: University of Connecticut, The National Research Center on the Gifted and Talented.
Gardner, H. (1983). Frames of mind. New York, NY: Basic Books.
Gubbins, E. J. (Ed.) (1995). Research related to the enrichment triad model (RM95212). Storrs: University of Connecticut, The National Research Center on the Gifted and Talented. http://nrcgt.uconn.edu/research-based_resources/gubbins
Lohman, D. F. (2005). Review of the Naglieri and Ford (2003): Does the Naglieri Nonverbal Ability Test identify equal proportions of high-scoring White, Black, and Hispanic Students. Gifted Child Quarterly, 49, 19-28.
Lohman, D. F. & Renzulli, J. (2007). A simple procedure for combining ability test scores, achievement test scores, and teacher ratings to identify academically talented children. Retrieved from http://faculty.education.uiowa.edu/docs/dlohman/Lohman_Renzulli_ID_system.pdf
Naglieri, J., & Ford, D. (2003). Addressing underrepresentation of gifted minority students using the Naglieri Nonverbal Ability Test (NNAT). Gifted Child Quarterly, 47, 155-161.
Naglieri, J., & Ford, D. (2005). Increasing minority children’s participation in gifted classes using the NNAT: A Response to Lohman. Gifted Child Quarterly , 49, 29-36.
Reis, S. M., & Purcell, J. H. (1993). An analysis of content elimination and strategies used by elementary classroom teachers in the curriculum compacting process. Journal for the Education of the Gifted, 16(2), 147-170.
Reis, S. M., & Renzulli, J. S. (1982). A case for the broadened conception of giftedness. Phi Delta Kappan, 64, 619-620.
Reis, S. M., & Renzulli, J. S. (2003). Research related to the Schoolwide Enrichment Triad Model. Gifted Education International, 18(1), 15-40.
Reis, S. M., & Renzulli, J. S. (2005). Curriculum compacting: An easy start to differentiating for high potential students (The Practical Strategies Series in Gifted Education), (F. A. Karnes, & K. R. Stephens, Eds.). Waco, TX: Prufrock Press.
Reis, S. M., Westberg, K. L., Kulikowich, J. M., & Purcell, J. H. (1998). Curriculum compacting and achievement test scores: What does the research say? Gifted Child Quarterly, 42, 123-129.
Renzulli, J. S. (1976). The enrichment triad model: A guide for developing defensible programs for the gifted and talented. Gifted Child Quarterly, 20, 303-326.
Renzulli, J. S. (1977). The Enrichment Triad Model: A guide for developing definsible program for the gifted and talented. Mansfield: Creative Learning Press.
Renzulli, J. S. (1978). What makes giftedness? Re-examining a definition. Phi Delta Kappan , 60, 180-184.
Renzulli, J. S. (1981). Action information message. Mansfield Center, CT: Creative Learning Press.
Renzulli, J. S. (1986). The three-ring conception of giftedness: A developmental model for creative productivity. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (pp. 332-357). New York, NY: Cambridge University Press.
Renzulli, J. S. (Ed.). (1988). Technical report of research studies related to the enrichment triad/revolving door model (3rd ed.). Storrs: University of Connecticut.
Renzulli, J. S. (2005). The three-ring conception of giftedness: A developmental model for promoting creative productivity. In R. J. Sternberg & J. Davidson (Eds.), Conceptions of giftedness (2nd ed., pp. 217-245). Boston, MA: Cambridge University Press.
Renzulli, J. S., Gubbins, E. J., McMillen, K. S., Eckhart, R. D., & Little, C. A. (Eds.). (2009). Systems and model for developing programs for the gifted and talented (2nd ed.). Mansfield, CT: Creative Learning Press.
Renzulli, J. S., & Reis, S. M. (1994). Research related to the Schoolwide Enrichment Triad Model. Gifted Child Quarterly, 38, 7-20.
Renzulli, J. S., & Reis, S. M. (1997). The Schoolwide Enrichment Model: A how-to guide for educational excellence (2nd ed.). Mansfield, CT: Creative Learning Press.
Renzulli, J. S., & Reis, S. M. (2012). Defensible and do-able: A practical, multiple criteria gifted program identification system. In Hunsaker, S. L. (Ed.). Identification: The theory and practice of identifying students for gifted and talented education services. Mansfield Center, CT: Creative Learning Press.
Renzulli, J. S., & Reis, S. M. (2014). The Schoolwide Enrichment Model: A how-to guide for educational excellence (3rd ed.) Waco, TX: Prufrock Press.
Renzulli, J. S., Siegle, D., Reis, S. M., Gavin, M. K., & Sytsma Reed, R. E. (2009). An investigation of the reliability and factor structure of four new Scales for Rating the Behavioral Characteristics of Superior Students. Journal of Advanced Academics, 21, 84-108.
Renzulli, J. S., Smith, L. H., White, A. J., Callahan, C. M., Hartman, R. K., & Westberg, K. W. (2002). Scales for Rating the Behavioral Characteristics of Superior Students. Technical and administration manual (rev. ed.). Mansfield, CT: Creative Learning Press.
Renzulli, J. S., Smith, L. H., White, A. J., Callahan, C. M., Hartman, R. K., Westberg, K. W., . . . & Sytsma Reed, R. E. (2010). Scales for Rating the Behavioral Characteristics of Superior Student: Technical and administration manual (3rd ed.). Mansfield Center, CT: Creative Learning Press.
Sattler, J. (2001). Assessment of children: Cognitive applications (4th ed.). Austin, TX: Jerome M. Sattler Publisher.
Sternberg, R. J. (1985). Beyond IQ: A triarchic theory of human intelligence. Cambridge, England: Cambridge University Press.
Sternberg, R. J., & Davidson, J. E. (Eds.). (2005). Conceptions of giftedness (2nd ed.). Cambridge, England: Cambridge University Press.
Treffinger, D., & Renzulli, J. S. (1986). Giftedness as potential for creative productivity: Transcending IQ scores. Roeper Review , 8, 150-154.
Winner, E. (1996). Gifted children: Myths and realities. New York, NY: Basic Books.

  

Appendix A – A Bibliography of Resources for Conceptions and Definition of Giftedness and Talent Development
Borland, J. H. (2005). Gifted education without gifted children: The case for no conception of giftedness. In R. J. Sternberg and J. E. Davidson (Eds.), Conceptions of giftedness (2nd ed., pp. 21-52). New York, NY: Cambridge University Press.
Brody, L. E. & Stanley, J. C. (2005). Youths who reason exceptionally well mathematically and. or verbally: Using the MVT: D4 model to develop their talents. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (2nd ed., pp. 21-52). New York, NY: Cambridge University Press.
Feldman, D. H. & Benjamin, A. C. (1986). Giftedness as the developmentalist sees it. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (pp. 285-305). New York, NY: Cambridge University Press.
Field, G. B. (2009). The effects of the use of Renzulli learning on student achievement in reading comprehension, reading fluency, social studies, and science. International Journal of Emerging Technologies in Learning, 4(1), 23-28.
Gagné, F. (2005). From gifts to talents: The DMGT as a developmental model. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (2nd ed., pp. 98-119). New York, NY: Cambridge University Press.
Renzulli, J. S. (2005). The three-ring conception of giftedness: A developmental model for promoting creative productivity. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (2nd ed., pp. 246-279). New York, NY: Cambridge University Press.
Renzulli, J. S., & Delcourt, M. A. B. (1986). The legacy and logic of research on the identification of gifted persons. Gifted Child Quarterly, 30, 20-23.
Robinson, N. M. (2005). In defense of a psychometric approach to the definition of academic giftedness. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (2nd ed., pp. 280-294). New York, NY: Cambridge University Press.
Stanley, J. C. & Benbow, C. P. (1986). Youths who reason exceptionally well mathematically. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (pp. 361-387). New York, NY: Cambridge University Press.
Sternberg, R. J. (1986). A triarchic theory of intellectual giftedness. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (pp. 223-243). New York, NY: Cambridge University Press.
Tannenbaum, A. J. (1986). Giftedness: A psychosocial approach. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (pp. 21-52). New York, NY: Cambridge University Press.
Walters, J. & Gardner, H. (1986). The crystallizing experience: Discovering an intellectual gift. In R. J. Sternberg & J. E. Davidson (Eds.), Conceptions of giftedness (pp. 306-331). New York, NY: Cambridge University Press.

 

Appendix B – Action Information Message (AIM)