Teacher Evaluation

The Diane Rehm show 2nd hour today was about teacher evaluation, centering mostly around the research of economists Raj Chetty and John N. Friedman of Harvard and Jonah E. Rockoff (show guest) from Columbia. They measure how much teachers add value by comparing teacher's students' test scores with their incoming test scores. Their conclusion is that the top 5% of teachers in this value added measure have dramatic impacts on their students' lives. They have a website with much more info about their research.

Good teachers are good for students. Huh. Who knew?

Seriously, though, of course what they are claiming that they can now identify these teachers using much-maligned test scores.  I am not familiar enough with their research to rip it to shreds professionally, but critical thinking would seem to raise several problems quickly.  Where these discussions always seem to go is that now we know whom to fire. Finally! I love (sarcasm) how in a discussion about dramatic student improvement, there is no thought given to teacher improvement. Of course, it's not really a growth mindset in terms of the students, either, but rather an idea that they are in the control of these good or bad teachers. It inspired this cartoon, based on my favorite Sydney Harris cartoon.

I went immediately from there to a mentoring meeting, where after a brief discussion of international humor, we began talking about how we self-evaluate our teaching. Here's what a half-hour led to:

We noticed that the majority of these were student outcomes, instead of teacher behavior. The question arose: do we assess these characteristics that we care the most about?

Furthermore we came up with some (also student-centered) monitoring questions for ourselves:

• Are the students equipped for the activity?
• Do the students know what they are hoping to achieve/where they are hoping to get to?
• Are students getting more independent?
• What can the students do?

The big question was raised: how do we know if these things are really about effective teaching or are they just what we want?

I'm not suggesting that this, finally, is the teacher evaluation system for which we have been looking. Instead, I want to suggest that this kind of teacher conversation is what will lead to better teaching.  Follow it with a chance to see each other teach, and talk about these criteria based on common experience. In other words, improvement in teaching is much like learning in any area.

Student Teacher Self-Evaluation

In seminar today, we tackled the idea of evaluation.  We contrasted the traditional evaluation of sorting (these students can, these students can't) with Vygotsky's (what can students do, what are they trying to do, what can they not do yet).  Dave shared sample rubrics that his wife developed with her first grade students.  (See them in this post.)

We asked the students to develop three panel rubrics in terms of Jim Knight's framework for coaching: What does effective teaching look like in these areas?

Classroom Management :-) :-| :-( Supporting learners to find meaning in math Select activities learners would enjoy Give candy to get learners to participate Clear and consistent routine for students Inconsistent routine or unclear to students No routine for students	Content :-) :-| :-( Relational Understanding with learners in mind Relational Understanding Instrumental Understanding
Instruction :-) :-| :-( Empowering Learners Uncovering Content Covering Content	Assessment :-) :-| :-( Implementing a variety of assessment approaches and monitoring effectiveness Recognizing that I need to know more Typical summative assessments without a purpose

The student teachers started off by focusing on the smiley faces, mostly on Classroom Management (the first one on the board).  Dave filled in the above as a demonstration off the top of his head.  And we realized that this was going way beyond today.  It was feeling more like a semester long project, as we collectively learn more about being effective teachers.  One TA later shared that she felt mostly she knew about the frowny faces.  Didn't know what she wanted to be yet, but had some ideas what she didn't want to be.  Makes sense to me.

I'll continue to share as these develop.  It would be terrific if any readers might share share elements of what would be on their rubric.

Photo credits: From Flickr - bryangeek, rutty, jamiemw

Self-Assessment for Teachers

Merry Christmas to all!  (I offer these greetings sincerely and in peace, not trying to bludgen anyone with it.)

As my colleague David Coffey and I were planning for the winter student teachers, we were discussing ways to involve the student teachers in self-assessment.  Dave is quite the assessment guru, and we want to have them help develop the rubric, but at the same time support them with what we might develop.

Kathy Coffey, local first grade teacher par excellence, develops these amazing rubrics with her students.  If first graders can, I figure novice teachers have a shot.  (If only Kathy was teaching them...)

Of course, how to evaluate teaching is a thorny enough process that we are having a national hissy fit over it as a country. 

The best resources we've found so far are the NCTM professional standards (going on 20 years old at this point; need a membership to read the full thing, unfortunately), and the research Rebecca Walker noticed in JRME this past semester.  I wanted to share this stuff just to share, but also in hopes of feedback.  How do you evaluate other teachers, novice teachers, or support new teachers in self-evaluation?

Professional Standards: Evaluation
The Process of Evaluation

Standard 1: The Evaluation Cycle
Standard 2: Teachers as Participants in Evaluation
Standard 3: Sources of Information

The Foci of Evaluation

Standard 4: Mathematical Concepts, Procedures, and Connections
Assessment of the teaching of mathematical concepts, procedures, and connections should provide evidence that the teacher

• demonstrates a sound knowledge of mathematical concepts and procedures;
• represents mathematics as a network of interconnected concepts and procedures;
• emphasizes connections between mathematics and other disciplines and connections to daily living;
• engages students in tasks that promote the understanding of mathematical concepts, procedures, and connections;
• engages students in mathematical discourse that extends their understanding of mathematical concepts, procedures, and connections.

Standard 5: Mathematics as Problem Solving, Reasoning, and Communication
Assessment of teaching mathematics as a process involving problem solving, reasoning, and communication should provide evidence that the teacher-

• models and emphasizes aspects of problem solving, including formulating and posing problems, solving problems using different strategies, verifying and interpreting results, and generalizing solutions;
• demonstrates and emphasizes the role of mathematical reasoning;
• models and emphasizes mathematical communication using written, oral, and visual forms;
• engages students in tasks that involve problem solving, reasoning, and communication;
• engages students in mathematical discourse that extends their understanding of problem solving and their capacity to reason and communicate mathematically.

Standard 6: Promoting Mathematical Disposition
Assessment of a teacher's fostering of students' mathematical dispositions should provide evidence that the teacher-

• models a disposition to do mathematics;
• demonstrates the value of mathematics as a way of thinking and its application in other disciplines and in society;
• promotes students' confidence, flexibility, perseverance, curiosity, and inventiveness in doing mathematics through the use of appropriate tasks and by engaging students in mathematical discourse.

Standard 7: Assessing Students' Understanding of Mathematics
Assessing the means by which a teacher assesses students' understanding of mathematics should provide evidence that the teacher-

• uses a variety of assessment methods to determine students' understanding of mathematics;
• matches assessment methods with the developmental level, the mathematical maturity, and the cultural background of the student;
• aligns assessment methods with what is taught and how it is taught;
• analyzes individual students' understanding of, and disposition to do, mathematics so that information about their mathematical development can be provided to the students, their parents, and pertinent school personnel;
• bases instruction on information obtained from assessing students' understanding of, and disposition to do, mathematics.

Standard 8: Learning Environments
Assessment of the teacher's ability to create a learning environment that fosters the development of each students' mathematical power should provide evidence that the teacher-

• conveys the notion that mathematics is a subject to be explored and created both individually and in collaboration with others;
• respects students and their ideas and encourages curiosity and spontaneity;
• encourages students to draw and validate their own conclusions;
• selects tasks that allow students to construct new meaning by building on and extending their prior knowledge;
• makes appropriate use of available resources;
• respects and responds to students' diverse interests and linguistic, cultural, and socioeconomic backgrounds in designing mathematical tasks;
• affirms and encourages full participation and continued study of mathematics by all students. 

Japanese Principles of Instruction
“Are There Any Places That Students Use Their Heads? Principles Of High-Quality Japanese Mathematics Instruction,” Corey, Peterson, Lewis and Bukarau, JRME, v41, n5, 2010

Original

1. The Intellectual Engagement Principle: High-quality mathematics instruction intellectually engages students with important mathematics.
2. The Goal Principle: An ideal lesson is guided by an explicit and specific set of goals that address student motivation, student performance, and student understanding.
3. The Flow Principle: The flow of an ideal lesson is built from a question or a problem that students view as being problematic. As students intellectually engage in the problem, building on their previous knowledge, the students are supported in learning the lesson’s big mathematical idea.
4. The Unit Principle: A lesson is created in the framework of past and future lessons, particularly between lessons in a unit but also between units and grade levels. The lessons in a unit help students progress to ways of thinking, writing, and representing mathematics evident in the discipline of mathematics.
5. Adaptive Instruction Principle: High-quality instruction adapts so that all students are engaged in mathematical work that appropriately challenges their current understanding.
6. The Preparation Principle: High-quality instruction requires a well-thought-out, detailed lesson plan that addresses the previous five principles and interconnects them in a coherent lesson.

Draft version of this for our student teachers:

1. The Engagement Principle: math instruction should intellectually engage students with important mathematics that is relevant to their purposes.
2. The Goal Principle: A lesson is more effective when guided by an explicit and specific set of goals that address student motivation, performance, and understanding. Examples include: students’ progress to ways of thinking, writing, and representing mathematics the way professionals do.
3. The Problem Principle: A lesson is more effective when built from a question that students view as being problematic. As students build on their previous knowledge by means of the problem, the students are supported in learning the lesson's big mathematical idea.
4. The Connection Principle: Math instruction is more effective when units are a framework of connected lessons; ideally also with connections amongst units and even between grade levels.
5. The Support Principle: High-quality instruction adapts so that all students are engaged in mathematical work that appropriately challenges their current understanding.
6. The Assessment Principle: High-quality instruction includes gathering and analysis of data that measures student progress towards objectives, and use of that data in planning and adjusting instruction. An ultimate goal is that students will be equipped to self-assess and self-correct.
7. The Reflection Principle: High-quality instruction requires a well-thought-out, detailed lesson that addresses the previous principles and interconnects them in a coherent lesson.

As questions for starting conversations:

1. Why is this math content important? For what is it needed? Why is it worthwhile?
2. What do you want students to understand from this lesson? What should they be able to do? How will it help students progress to ways of thinking, writing, and representing mathematics like mathematicians?
3. Is this a problem or an exercise? What makes it problematic? What’s the hook of the problem?
4. How does this connect to what they’ve done before in this unit? In this year? In previous years? How will it connect going forward?
5. Are all students ready for this task? What kind of support do they need, as a group or as individuals? How will you tell?
6. What do students understand? How do you know? How can they get from what they know to what they need to know?
7. Does this lesson make sense as a whole? How would you or a student summarize it or reflect back on it?

billsophoto @ Flickr

Self Evaluation
As we think about the novice teachers self-evaluating, one of the relevant frameworks seems to be the Levels of Transfer from Joyce and Showers.  It's hard to use that in a grade situation because students want to jump to Executive Use, which is not how that works.  (Here's a nice blogpost by a teacher using them for self-evaluation; also a nice introduction to them.)



So how would this work for a self-evaluation rubric?

As teachers develop, they attend to different aspects of the profession:

• Focus on self:  What am I doing?  How did I do? 
• Focus on content:  What am I covering? What's a good activity or explanation for this?
• Focus on student performance:  What was the average?  What grades are they getting?
• Focus on student learning: What do they understand? What support do they need to do this?

What I like about this progression is that it's not from good to bad.  They are all aspects of teaching which are necessary to contemplate.  I think as teachers become more experienced, they progress through the levels more quickly and holistically.  I think novice teachers could recognize where they are at with respect to different aspects of teaching.  "In my planning I'm thinking about student learning, but in instruction I'm still concentrating on myself."

So how do you self-evaluate?  If you're a teacher educator, how do you assess your novices' teaching?  Do you have ways of supporting your teachers in self-evaluation?