Visible Learning- Effect Size

Effect Size and Effective Techniques

Welcome to my second post on the book "Visible Learning" by John Hattie. In my previous post I discussed the practices of uninspired (bad?) teachers. This post is less on the generalities of teaching, and more on how this book measures effective teaching.

This book quotes effect size in just about every chapter. On the surface, it is pretty easy to decipher what it is about. The higher the effect size, the better the technique/item/etc. is. At the outset, they establish 0.20 as the minimum effect size that is "worth it" and 0.40 as a full year's worth of learning. A few examples of higher effect size influencers are "Teacher Student Relationships" (0.72), "Teaching Better meta-cognitive strategies (0.69), and "Teaching Study Skills (0.63). A few low level influences included "Home-Schooling" (0.16), "Open VS Traditional Learning Spaces" (0.01), and "Individualized instruction" (0.23).

You may not be surprised to learn that I quickly grew frustrated with the lack of clarity on what exactly effect size was. I made it through 96 pages before I broke down and looked it up in the appendix. Here is is

Do a pre-test, Do a post test.
Take the class average for the two and find the difference (post-pre)
Divide that by the average of the standard deviations for the two tests
Boom- Effect size.
So if you improve the scores a great deal by your technique, and it worked equally for all students (they were all brought up, so there was a low standard deviation), then by this book it was deemed effective.

The book later recommends teachers employ this formula and technique with their classes to determine how effective the teaching techniques are. I don't think this is a bad idea per-say, but it does leave a few large-ish holes in the overall view of education.

1. It doesn't measure long term retention
2. It only views the class as a whole. So if half the class gets 100% after the lesson, but the other half was not paying attention and fails, the technique was a failure
3. Just because it is effective, does not mean it is necessarily good for students.

I don't think any of those three reasons is enough to say "don't do this", but I feel it is important for me to keep it in mind. Numbers can be a tempting mistress to math and science teachers.

Individual effect sizes are possible (post-pre test)/class standard deviation

From videos of John Hattie, I've learned the following:

• Classes should have clear learning intentions
• Classes should have challenging success criteria (for both struggling, and accelerated students)
• Provide feed back
• Learn Visibly myself
• Provide an opportunity for students to determine their own progress
• Classes should trust the teacher so they are comfortable saying "I don't know"

I do my best to make my class comfortable, and convey care for students, but students almost never say "I don't know" in my class. Am I alone? How Can I improve this? Can I improve this?

Visible Learning: Inspired Vs Uninspired Teachers

Visible Learning: Inspired Vs Uninspired Teachers

I recently obtained a copy of  "Visible Learning for Teachers" by John Hattie. Unlike several other education books out there, Visible Learning for Teachers uses research and data to examine the effectiveness of teaching strategies.  For instance, yes homework will help students marks, but is the effort worth it for all parties involved (spoiler alert: the answer is no). 

The book starts off with some general boiler plate information, but one section has caught my interest the most. It is titled "Visible Learning- Checklist for Inspired and Passionate Teaching", and it has a list of professional development criteria that has been shown to lend itself best to "inspired" teaching. I don't know if inspired is really the best term, so I just mentally translate it to "good" teaching. 

In the discussion after this list, there is a rather important string of sentences discussing what inspired teachers do NOT do.  I think I am a rather far distance from "inspired", but I think I can certainly avoid being "uninspired". Here is a list of "uninspired" teaching practices

Italics added for my own emphasis

• Use grading as punishment
• conflate behavioral and academic performance
• elevate quiet compliance over academic work*
• excessively use worksheets
• have low expectations and defend low-quality learning as "doing your best" **
• evaluate their impact by compliance, covering the curriculum, conceiving explanations as to why they have little to no impact on their students
• prefer perfection in homework over risk-taking that involves mistakes

*I'm not too sure what they are getting at here. Why is there a comparison between compliance and academic work, it seems like they are comparing apples and oranges.

Where I am falling short is the excuse making. I know there are students that are struggling in my class, but because I don't know what to do about it (beyond what I have already done), I make excuses. "They are not really trying" , "They don't study", "They did not have the skills required for the course coming into it". While these things may be true, they are not terribly constructive. It doesn't matter why the students are not improving, it only matters that they are not improving. I will admit, it is unlikely that they have made zero improvement, but with the constraints of the public education system as they stand, the improvement made is not sufficient to allow them to complete the course. 

So what is an aspiring inspired teacher to do? Keep reading the book I suppose!

-Mr. H

The power of the baseline

The Power of the Baseline

TL;DR: Establishing a starting point for students' abilities with the curricular competencies (a base line), is the first step to not only assessing what they've learned, but also the first step to growing their skills in your curricular area(s). 

If you are reading this in the hopes of a post regarding the righteous baselines of The Red Hot Chili Peppers, you are going to be disappointing. The title was misleading, I'm sorry. However, if you are tuned in for a post regarding assessment practices with the curricular competencies and some standards based assessment, then boy do I have a treat for you!

Some background information to start. When the new curriculum for BC came out,  I wasn't a big fan. It taught too little, there wasn't enough detail, the content was ambiguous, overall I thought it was  'bleh!'. As I've worked through the new mandate from the ministry of education, I've grown to appreciate it. The following are my reasons for the turn around.

1. Memorization is not Science. While it is true that you need to know things in order to practice science, it is not true that the focus should be memorizing facts. 
2. I have been teaching the scientific method for years, and this curriculum is heavily influenced by it
3. I am a huge nerd and this allows me the freedom to go into the areas both my students and I find interesting. 

The only issue I've had (up until recently) was how to assess the curricular competencies (CC). Neither the district I work for, nor the ministry of education itself, were of any help here. So this is what I've come up with...

Step 1: Baseline assessment (experiment like gyrocopter lab, non-curricular, but opportunity to explore the CC

Step 2-5: Graduated Release of the Inquiry process (focus on how it relates to the CC) 

Step 6: Open inquiry

Steps 1-5 are almost entirely student assessed with teacher support, step 6 is an interview based assessment with the questions supplied in advance of the interview. 

Assessment is done with a single point rubric (what students are working towards) with a growth mindset emphasis, that students are not expected to exceed expectations in all categories (or any really for that matter) but rather are expected to address where they are, and how they can improve. The goal, I tell them, is to fully meet expectations for all of these items by the time you graduate high school. 

The beginning of the year, which previously was reserved for "teaching the scientific method" is now "introducing the CC". Throughout the year, I have students do labs/inquiry projects with decreasing involvement from myself. The first project I guide the CC heavily (direct questions, check lists, etc) but as we progress through the curriculum, I provide fewer and fewer scaffolds, and allow them to be independent with the CC more and more. My hope is that they can reflect on their CC and see how their skills have developed over the course of the year. Below is what it looks like in general for science 10.

Intro lab: Galileo's rolling ball experiments- how does the height of a ramp and slope of a ramp impact the final velocity of a ball. - students self assess on CC

Start the question journal: Students are informed that they will learn a lot of interesting things this year, many of which are half-truths. There will be a lot of room for questions, and those questions are encouraged. They are asked to keep a journal of any questions that come up for them, and time is given at the end of most classes for students to reflect and record these questions. 

First guided lab: Foam insulation and marble roller-coaster. - Students build a roller coaster out of foam insulation, marbles and tape. They first model it using a phet "physics skate park" and I ask them questions directed to address ALL the CC directly (major headings, but not the sub-headings)

Second Guided activity:- Students measuring pH of soil sample. I provide a back story that describes how I have begun the inquiry process, students complete the process, then reflect on any CC that have grown. 

Question journal check in : Class discussion regarding the questions they've developed and how they can be moved to an inquiry question.

Third Guided activity: - Students address the question "If the Canadian space agency wants to colonize a new solar system, what star should they target" - students develop criteria, I give resources for students to develop understanding of star-cycles to help them frame their questions further. Students then present their information as a bidding war to the agency as to why their solar system is the best choice. 

First unguided activity: - Students are asked to investigate the questions they've had throughout the year. This is a formally assessed project, and students will engage in a double blind assessment of the draft product

Fourth guided activity: - Are Mr. Smith's children his own? Students use their understanding of basic Mendelian inheritance to develop and ask questions to explore the problem. *Special focus on ethics in this section*

Final inquiry project: Students are told to "mine" their journals for a question to research. They are expected to go through the CC for guidance on what their research should include. The final assessment will be in the form of an interview, where they look at where they started off in their CC and make claims on their growth, using their learning portfolio (Freshgrade) and their final inquiry project as evidence of that growth. 

All of this is done with the typical classroom activities which will develop their CC as well as their curricular content skills (and core competencies of course), but the steps described above are typical classroom activities that have been adapted for the purposes of illustrating their CC. 

Summer Reading: Part 3 of ? - The Value of Time - Deep Thinking requires Deep Knowledge - Wait time

Summer Reading: Part 3 of ? - The Value of Time 

TL;DR: Students miss school because the time spent in it, is not accurately valued by students. In their minds, minutes and even hours can be wasted with no adverse consequences. One way to change this mind set is to treat every second in your class like gold. This gives your students more time for learning.  

I remember when a misadventure in elementary school left me with three days at home to think about what I had done. I don't know much about other 11 year olds, but I know I wasn't exactly the "self-reflection" type. I was a ninja turtles type, so that's what I did. I watched TV, I read comics, and I played video games. Oh sure, there was homework assigned as punishment, but that took me 15 min and then I was free to do what I liked. To make matters worse, I completed two weeks worth of work in the three days off. This was likely the most damaging event in all my schooling. Essentially, I was given very clear evidence that my time in class was a waste. This attitude lasted until my first year of university, and very nearly cost me my place there. I can't help but wonder, are any other students getting this same evidence anywhere?

1) What do we want!? Student accountability for their own learning! When do we want it!? When parents stop sabotaging our attempts at instilling it!

So cheeky title aside, the area I teach has a fair number of students whos' families hail from lands far from where they currently call home. Normally this is a fantastic addition to the classroom because it provides me a variety of viewpoints and opinions to draw from. All too frequently however, it presents a significant problem for myself and my students. Visiting the homeland. Any time we have more than a few days off, I lose at least one of my 150 students to visiting a far flung land. Not for days, but for weeks, and occasionally months. On this, I am torn.

On one hand, it is an opportunity of a lifetime. I grew up not knowing my heritage (not that it is all that exotic) so for my students to experience a new culture, and to be able to relate that to where they came from? That is something that I couldn't give them in a lifetime of classes. However, that isn't what I have been tasked with teaching them. I have a detailed set of learning outcomes that I have to assess at the end of the year. If a student misses a month of school, what are the chances that they will achieve all of the learning outcomes demanded by the ministry of education? What am I to do with a student that is gone that long. As previously indicated, I am very much on the fence. Theoretically, students are intended to get a form filled out for any extended absences, I could simply refuse to sign it until a student has shown me they have the minimum skills / knowledge from the unit/s they will be missing. Unfortunately, the vast majority of the time these students don't get the forms signed, and if they actually do bring you the form, it is considered a fait accompli. I don't want to be the road block stopping them from seeing where they came from, but I also don't want to be complicit in them sacrificing their education. I am often confused as to what the parents are thinking when they book these expensive and extended vacations. I have a strong suspicion they don't value my class time. That might be my fault...

2) Time use: Waste Not Want Not

Hands up if you have ever had this experience. You have given the students a task to complete, most of the students remained on topic and are now finished whatever it is you have assigned. There is five minutes left in class, and one student starts the avalanche of "pack up time". With an efficiency typically reserved for military parades, all your students have packed up and are now fully, and completely off task, regardless of the completion level of their respective assignments.  Not a big deal right? 5 minutes? 

5 min / class  X 180 classes per year = 900 min. That is 15 hours of lost time, or in other words, 11 full classes missed. 

I routinely communicate this to my students, but unsurprisingly, I get few results. Here is my theory on why. 

Some students will complete assignments fairly quickly, it is just the nature of the game. Often, I don't have meaningful work ready for those students. By the time I drag out something for them to do, their lack of attention / activity has spread like wildfire and wild horses couldn't drag their attention back to whatever it was I wanted them to do. 

Doug Lemov in his book Teach Like a Champion has a fairly simple solution. Always have deep and meaningful work in your pocket (literally or figuratively) The moment a student completes an assignment, you pull out something new for them to do. Taking it a step further, I would prefer to have an ongoing, weekly assignment, that students would work on in their spare time. There is no waste on transitions, no confusion, and no opportunity for tom foolery. 

This communicates to students that your time, and their time is valuable, and not to be wasted. Sometimes enforcing the "work to the bell or past it" mentality is exhausting, but anything less communicates to students that the time in class in not of value. I suspect this bleeds into their parents' opinions of school and is a contributing factor in why they think they can pull their child out for a month and have no consequence. 

In my next post:
3) Deep thinking requires deep knowledge
4) Wait time, and why do we ask for hands?

The Best Self-Assessment Questions for Encouraging a Growth Mindset

The Best Self-Assessment Questions for Encouraging a Growth Mindset



A Growth mindset (GM) is one of the pillars to improvement.  While this article doesn't speak to what a GM is, I think it is pretty clear as to how the questions presented would be valuable questions for students  to ask. In fact, they aren't bad questions for educators to ask of themselves as well!

Summer Reading- Post 2 of ? : "To be a Scientist, you must always read"

Summer Reading- Post 2 of ? :

 "To be a Scientist, you must always read"

TL;DR - Science teachers should get students to read more scientific articles / information because it is their job, and what is best for kids. Reading aloud is one technique that I would like to use more, both to break down the culture of fear surrounding being wrong, and to add life and excitement to material that may otherwise seem listless or boring to some students. 

     In his book "Teach Like A Champion" Doug Lemov discusses techniques to optimize your students' classroom experience. One such technique that caught my eye was "Control The Game". Upon first reading the title, I assume this was going to be something about classical classroom management. That it was going to be some psychobabble regarding being the dominant presence in the room, or some such other thing. Imagine my surprise when right from the byline it states "Ask students to read aloud frequently, but mange the process to ensure expressiveness, accountability, and engagement."

     I have a confession to make. I am a big. fat. phony. At least that is how I feel after reading this chapter. I have long taken up the cause of the beleaguered Humanities teachers. A bit of back story first perhaps? 

1. One of the most important skills a child develops is how to read. The vast majority of research available indicates that the best way to get better at reading, is to read more.
   
2. Many science teachers recognize this, and bemoan their students inabilities to read and comprehend scientific information. Furthermore, they often level criticism at their Humanities counterparts. "Why don't you ask them to read more scientific information" they often lament.
    
3. It is not the job of a Humanities teacher to teach the skills of a science class.
   
4. It is; however, the job of a science teacher to teach the skills of a science class

--- Q.E.D.---

5. It is my bloody job to teach students how to bloody well read scientific information!

     So that sounds great and everything. But here is the problem. You see, nobody ever taught me how to read scientific information. It was something that I just picked up over time, and honestly, I am not entirely sure exactly what I do when I read scientific papers. So here is where the hypocrisy comes in... I haven't quite been doing that, actually to be fair, I haven't been doing that at all in my Jr. Classes. Do I ask my Jr. Science classes to read from the text book? Sure, sometimes. Do I bring in the occasional article? Yeah. But with my Jr. Science classes 1, I have been seriously lacking. 

     Back to the book. Lemov recommends reading early, reading often, and reading out loud. Furthermore, he suggests not just reading out loud, but reading with emphasis and emotion. I can't tell you the last time I had a student read out loud, let alone with emphasis or emotion! He continues in the chapter to further refine the "read out loud" theme. I'm sure you had a similar experience to mine. A teacher would start with one student, then predictably move around the class, each student reading one paragraph. You would look around the class, figure out what paragraph was yours, then promptly tune out. For that reason (tuning out), he proposes keeping students' on their toes by mixing how much they read, who reads, and obviously what they read. This connects with the technique mentioned in my previous post regarding a culture of wrong, and regarding mistakes a learning opportunity, rather than something to be embarrassed about. In addition, not accepting "I don't know" or "I don't want to read" as an answer. Even if they just read a sentence, or the teacher reads the majority of the material, but they pause and make the students read out important words. 

     I remember during my time in  high school, there was nothing worse than having to read out-loud, a new word, and then stumble and fumble over it. Part of the dread of reading out loud came from the fear of failure, or being wrong. If I can reduce that fear, maybe I can reduce some of the anxiety around reading aloud  and thereby introduce more meaning to the deluge of scientific reading I will be throwing at them ;) .  Furthermore, by reading the material out-loud I can bring more meaning and life to it. If a student reads something in a monotone voice, I can re-read it with emphasis on the parts that I found interesting or surprising, or I can ask them to re-read it doing the same. On occasion, it has been pointed out to me, not unkindly, that I am a bit of a nerd. That is to say, I am very passionate about topics that one is typically not passionate about. It is that passion that may be one of my greatest strengths as a teacher. What better way to try to get my students to match that passion then with matching my volume and cadence with reading a particularly juicy sentence out loud?

     What do you think? Maybe you disagree and you think it is the English teachers' job to teach students how to read different types of literature (science included). Maybe you think asking students to read aloud is cruel and unusual punishment. Maybe you didn't like how I switched between out-loud and aloud (I checked, they both suffice). You know where to comment. 

1- I am being a bit harsh on myself here. I think I do a much better job with my Sr. Science classes. I typically get them to keep a pdf I found regarding reading scientific articles, and I have a small selection of primary research articles for ecology and a few other topics. I also pepper in a liberal number of articles from "IFL Science" 

Summer Reading: Teach Like a Champ - Post 1 of ?

Summer Reading: Teach Like a Champ - Post 1 of ?

In this post: Checking for understanding, The culture of "wrong", and don't take "I don't know" for an answer.

In the dying days of the school year, and in a desperate attempt to maintain my motivation going into the summer, I decided to order a book of chapters.ca (No I won't call it indigo). I was torn between "Teach Like a Champion" and "Teach Like a Pirate". I was leaning heavily towards Pirate (for obvious reasons) but after learning that the latter book did not involve swords, parrots, or booty, I decided on "Teach like a Champion" by Doug Lemov [From here on out abbreviated TLC... don't go chasing water falls]

The book begins with some useful preamble about he purposes of the book, but in an effort to avoid stepping on the authors toes, I am just going to skip to the good stuff. The vast majority of this book involves techniques, so I will provide my take on some of the 62 techniques (Yes, you read that right, 62!)

The first six groups deal with checking for understanding, abbreviated as CFU by the author. This posed a bit of a problem for me, working as a lifeguard and taking quite a few molecular biology courses, CFU typically stands for Colony Forming Units and is indicative of how much feces is in the water.  But I digress.....

When Doug Lemov speaks of CFU, he isn't talking about at the end of a unit for the purposes of a report card, but rather activities at the beginning, middle, and end of a lesson to determine A) When it is time to move on and B) Where you need to focus some more attention. While reading, this is where I started to get really interested. I will be teaching math for the first time in a while starting in September, and all of this clicked well with how I was envisioning my math class. (These techniques will also work well with my Jr. and Sr. Science classes as well, and he provides examples on how to use them with an English class to boot!)

Unfortunately this first unit is also where I start to get a bit worried. Throughout TLC Lemov provides suggests that involve a great deal of preparation. He suggests having everything in a lesson planed out, from the questions you will ask, to the answers you expect to get from the students, and how you will address what you think the students will do incorrectly. At times it reminds me of some mad game of chess. That degree of planning might be obtainable for me after five of six times teaching a course, but I don't know if our curriculum will stay the same for long enough to make that possible (kidding....mostly) Here is my takeaway (I'm ignoring the parts that I don't agree with, or feel like I wouldn't have time for).

1. Ask probing questions in class.
      e.g. "Where is the energy transformation when a car stops? What else has a similar                       transformation? How do you know"
   These questions should be answered in full sentences so that someone walking by the room and only hearing the student answer, would know what was going on (more on that later)1. This ensures that the student DOES understand what is going on, and it isn't a fluke, and it makes sure that the other students understand it now as well (even if they didn't before) 
2.   Ask for volunteers infrequently. If you are asking questions of the class to assess understanding, and the same three students are answering your questions, you are really only assessing the understanding of those three students. Relying on the same few students also allows the remainder of the herd to relax and take a mental break during these question times. Not exactly what a teacher is looking for!
3. Don't take "I don't know" for an answer. Those herd members that want the rest know that if you just feign ignorance enough times, you will get that rest. Don't let them!
   E.g "Markus, Where would you find the largest biomass in an ecosystem""I don't know""Lets figure it out then. Where should we start to find the answer?""I don't know""What are the options""I don't know""Okay, Bobby Brownose, where would we find the LEAST biomass" "The apex predators""Correct Bobby, Frieda Cantfocus, why do you think Bobby said that""Because they rely on the next level down for energy, so if there were more of them, they would starve""I think you have the answer, but I'm not sure. Can you give that answer again, but convince me you understand it, maybe using the correct terms for everything""Apex pedators eat primary and secondary predators, so if there were more biomass with the apex predators, they would starve"
   "Much better Frieda! Now Markus, with that information can you predict where we would find the MOST biomass and why?
   This exchange would probably take 3 min, but would give a pretty clear indication of the general level of class understanding, but only if you used targeted questioning. You have to know who to ask that is least likely to understanding, most likely, and equally as likely. This requires you to know your class, but that is a post for another day (Attachment Theory etc) 
4. Change the emotion behind being incorrect. "Being wrong is embarrassing. It grates. It is to be avoided at all costs!" at least that is how I felt as a student. It wasn't until university that I realized I learned more with I got something wrong then when I got it right (sometimes by accident) The class (and maybe school) mentality has to shift. "Growth mindset" gets thrown around a lot. But that is what is called for by this book. 
5. Standardize your written CFU. Create a template to put your questions in, and have students write the answers in the same location with their work in another, laid out in a particular manner. This allows you to circulate and quickly understand what students understand and what they don't. Furthermore, he suggests you should have these problems all worked out in advance along with likely areas of misunderstanding. When you circulate, he suggests you write down the names of those students that had those misunderstandings. This along with the culture of "wrong is good" can help to shape your class discussions or single out students for more help at lunch/after school.
   
   I feel like # 5 is the tip that would help me the most in my Math classes, but I haven't taught them enough to anticipate the misunderstandings. That is where planning with a more experienced teacher would be great, unfortunately there is vanishingly little time in the school year to do things like that. I find this to be the most heart breaking because it is where "what is good for kids" does not seem  possible to me (sparing giving up my every waking moment) 

1- Using "academic language" is the only way to ensure students speak the language of academia and their points are taken seriously. Saying " the thing does stuff and that shows that process" will convince nobody of anything