Easy or hard?

A x A = BC

BC x BC = DEC

Which digit does each letter represent?


Level two GCSE problem.

Technology, Environment, Mathematics & Science (TEMS) Symposium Abstract

Done, and dusted.  Sorry about the quality - but if you are interested here is how the PhD is shaping up two years in.

How to set up a system of observing tutors that is safe and effective

A previous post raised a range of issues that relate to effectively implementing and evaluating L&N provision.  This post is designed for the person in a professional development role tasked with overseeing the implementation of ELN within a medium to large organisation.  A position I have held.

This position is tough as the role requires bridging two different worlds.  On the one hand, management, and on the other, tutors.  As such the role requires that you take on-board the unique stresses and demands of each domain without ever being a complete part of one or the other. This is partly what makes this role essential in an organisation as it enables a unique perspective on the demands of the roles while avoiding the extremes of both.

One of the key tasks for the person in this role is to observe and provide professional feedback to tutors and work toward meeting performance outcomes set by the organisation.

Beginning tutor observations

Tutor observations are essential to improving organisational performance.  They are also a key tool for cultivating an internal culture of constant improvement.  Yet, as a tutor there is simply NOTHING more intrusive and threatening than being observed.  The solution is to set things right at the beginning and stick to it.  I will describe some key ways to do this below.

Selection of the observer

Managers take note:  The person you select for this role (and observations) must be an experienced, trusted and respected staff member.  If this is not the case, abandon the whole idea now.  If you are not 100% certain you have the right person, have the tutors select their own person from the staff.  Or if it looks like the whole thing is too difficult, distribute the task to all the staff and work through every staff member.  That is, every tutor will have a go at observing other tutors and giving feedback. This has proven to be effective in the past.

Change the notion of 'observing' 

 Assuming you are this person, here are some ideas to help reduce tutor concerns regarding the observations.  First, change the connotation many tutors have with observation.  That is, the top down quality control view.  Observations are done to provide another perspective on tutoring practice in order to develop skill, not to check up on how tutors teach.  What helps is to create a divide between the findings and management.  There is an easy way to do this.

1.  Have a meeting with the tutors and tell them what is happening.  Make sure you tell them that this is to improve performance and provide positive feedback.  Note, that they won't care what it is for, they will likely simply view it as threatening.  Don't say anything else -  just ask for their concerns. Then just listen.  

2.  Address those concerns.  Most will relate to security and vulnerability, and the possibility of bad performance reviews.  You are adding to their workload and stress.  You MUST address these concerns. Here is how:  

Confidentiality

1. All the observation feedback must be confidential and anonymous.  That is, the feedback will be only between the tutor and observer.  

2. The feedback from all observations will be thematised and summarised into a report that management will see.  This report will present themes that CANNOT be traced back to any one tutor.  It is broad and not specific to single tutor performance.

3. This report will be sent to all the tutors first and if they have any concerns they can talk to you privately before you send the report to management.  Give them all time to read it.  Make any changes the tutors want.    

4.  Positive feedback.  In my experience most tutors expect this to be an unpleasent experience- it should be incredibly encouraging. Make it encouraging and rewarding for the tutor.  Give great feedback that specifically identifies the things that are really good at.  How they handled difficult learners, how they introduced certain content, their rapport with learners.

As for constructive feedback, simply ask them.  "What do you think you could have done to improve the lesson?".  Let them talk.  Give no constructive feedback.  None. Nadda.  Thank them and reiterate the positive feedback.  Promise to send them their feedback form and make sure you do within one day.

At the next observation, however (one, two,  months  later) during the feedback, read back to them what they said last time about room for improvement, and ask them how they addressed what they had mentioned.  Let them talk.  People will quickly realise what's happening.  Ask them again what areas they would like to improve and then repeat again in a month or two.

The simple fact that you have put in a system that gets tutors  to state where they may need to improve, and that it will be regularly talked about, will get tutors taking action. And... tutors will begin to enjoy this process - once they realise it is safe, and not a system to beat them with.  It is a time of validation and reflection.  There is no stick.

What you are trying to develop is a culture of constant improvement in which tutors do not fall into ruts and can make change.

Use the second themed report to show that tutors had identified and acted on key areas.  Ask them for feedback and then send it to management.  The first three reports will not really any use for real measurement, but they are a way of implementing a process that serves the tutors and the organisation.

In time the reports will indicate key areas that might require training or PD.  Themes will emerge (say classroom management). Once you have a pattern you might decide to bring in a classroom management specialist.    It is also a great way to remind management of what is happening in classrooms.  They will like this because once the tutors trust the process and don't dress up the results then management have an extra mode of communication to inform their thinking.

In my experience, observing tutors is wonderful for boosting morale and getting a buzz going.  Once you have told tutors it is happening do it quickly, with a day or two.  Otherwise they worry and they shouldn't.  Make the experience positive.  They will tell the others and word will spread that it is good.

Last thing:  we all know that we prepare better for the lesson we are being observed on and hence it does not reflect a 'real' session.  That's okay.  You see, hopefully, the extra time the tutors put into that lesson will ensure a good class, and generate new ideas and behaviours.  It will pay off.

The inquiry 

McDonell:  I'm going to be asking you questions pertaining to two different but related areas.  The first is to gain clarifications regarding your organisations' definition of embedded literacy and numeracy and the process you took to verify that this aligned with your funders' definition.  Second, what actions you took regarding your quality control measures for your organisations' embedding of literacy and numeracy into your level one and two programmes.

CEO: Understood.

McDonell:  Please explain what your Organisations' understanding of embedded literacy and numeracy is?

CEO:  We have experts who are well grounded in the details.  My role does not require in-depth knowledge of ELN, only that systems are in place to ensure it is.  My understanding of ELN is that literacy and numeracy delivery is integrated into programme delivery.

McDonell: Are you aware that the TEC had released (some time ago) a document that defines its' high level expectations for embedded literacy and numeracy?

CEO:  I was not.

McDonell:  What efforts did your organisation take to ensure it's understanding of ELN was correct to ensure compliance with funding criteria that you were receiving?  

CEO:  We have specialist staff members who's role it is to stay updated.

McDonell:  And it was these, or this, staff members role to ensure the entire organisations' level one and two programmes were aware of the definition and were compliant with it?

CEO:  Yes.

McDonell:  And you assume your staff are aware of the definition, and subsequent expectation for their provision of ELN?

CEO:  Yes.

McDonell:  Well, we are compiling their responses to a questionnaire, and interviews,  as we speak.  We can review the findings tomorrow.  For now, let's continue.

McDonell:  Can you please explain your organisations' process for determining the quality of your embedded literacy and numeracy provision, or even if it was occurring?

CEO: We acquisitioned a staff member to implement ELN across the organisation and ensure tutors were embedding literacy and numeracy.

McDonell:  Can you explain to the house the criteria for selection of the staff member for the position?

CEO: The individual had experience with ELN and was an experienced staff member

McDonell: In what way?

CEO: Sorry?

McDonell: In what way were they experienced with ELN?

CEO:  Ah, they had delivered literacy and numeracy.  And they had been to a range of workshops.

McDonell: In what capacity had they delivered ELN?

CEO: I'm not sure I understand the question?

McDonell:  How do you know they were qualif... I'll rephrase.  What were your organisations' appointment criteria for this position?  For example what qualifications did the position require?

CEO:  I'm not sure.

McDonell:  Can you at least tell us what the position description defined as qualified?

CEO:  I can't.

McDonell:  So your appointment of the position went to an individual that did not meet any criteria other than having experience with literacy and numeracy that cannot be defined?  We don't really know what they did do we?  Or their quality? You see the problem don't you?  How do we know they didn't get their previous job based on having experience somewhere else?  We really need some criteria of their expertise that is not simply holding a role.

CEO:  They are a trusted staff member.

McDonell:  Now this individual was tasked with determining whether literacy and numeracy was being embedded and whether it was of  a sufficient quality to warrant accepting funding for it?

CEO:  They were tasked with making sure it was occurring.

McDonell: Not quality?

CEO:  Well yes, PD was provided to tutors.  Tutors were trained.

McDonell:  And the outcome of that training was?

CEO: To embed literacy!

McDonell:  Yes, and I am asking you what internal systems were in place to ensure this was in fact happening and that the quality was of a sufficient level to warrant receiving funding.  Did your staff member report to the organisation on the quality of provision?

CEO:  We use the TECs' Assessment Tool to determine the effectiveness of ELN provision.

McDonell: So the staff member had no mechanism to report back on quality?  Did the staff member actually review quality?

CEO:  As I said we have an extensive assessment process.

McDonell:  So I will assume that despite vying for and receiving funding that required quality ELN provision you had no internal mechanism for determining whether or not literacy and numeracy was being embedded in programmes, or that the quality was sufficient, other than your Assessment Tool results?

CEO: Correct:

McDonell: So there were no tutor observations, no interviews, no student interviews, nor internal reviews?  How do you know your tutors' were making any change to their practice, let alone doing a good job?

CEO:  I trusted my staff member to tell management if it required attention.

McDonell: But you will admit no formal mechanism existed?  Despite an extensive shift by the NZQA toward internal evaluation systems?  Out of interest did the staff member ever give an indication that tutors were not meeting the requirements of the funding?

CEO:  No.

McDonell:  That seems extraordinary given your present situation.  Surely you see that you would not be here today if things were operating as you presumed?  I'll move on.  The data from your Assessment Tool results have been largely static over the last four years and suggests that you have failed to improve the quality of your provision each year. What does this suggest to you?

CEO: Actually our results show that many learners do improve their literacy and numeracy.

McDonell:  I'm not talking about improvement within a given year.  I'm talking about a trend of improvement.  One year compared to the next.  Yearly improvements suggest your organisation is improving.  Your results do not.

CEO:  That is due to a range of issues in the sector such as the nature of the learners' skills, attrition rates, transience, social issues and a raft of ever changing TEC funding criteria.

McDonell:  Do you think that it is reasonable that a tutors' performance will improve the longer they are in the job?

CEO:  For the reasons I mentioned earlier, not necessarily.

McDonell:  In every other educational domain learner outcomes improve as tutors gain experience. Please explain to the house why this is not true in your organisation?

CEO: Tutors are busy, they have demanding students, the schools have failed and we have to meet unreasonable outcomes to continue in business.

McDonell:  Then it would seem reasonable that your organisation should have exceptional evaluation and training systems in place - particularly in funding sensitive areas such as embedded literacy and numeracy?

CEO:  We do have evaluation systems.

McDonell:  You made the statement that the assessment tool is your primary measure of ELN?  Two questions.  What other measures are you using to determine the quality of ELN?  And second, did the results of the assessment tool suggest in any way  that the PD you speak of improved learner outcomes?

CEO: In answer to the first, as I just said, no, the Assessment Tool is our measure.  As to the second improving tutor performance takes time.  We need time to bring the staff up to speed, and we are doing this.

McDonell:  The truth is though, isn't it, that because you haven't implemented systems to inform you of the internal provision of ELN that you don't actually know the answer to either of those two questions?

CEO: Not at this moment, no.

McDonell:  Tell me if you disagree with my summary.  You selected a staff member based on a trivial notion of experience.  No criteria existed for this role despite this role overseeing significant funding streams deemed priority areas by the Government.  Second, no formal mechanism existed for this individual to feed back concerns to management regarding quality.  I suspect the individual was left without support or direction from management.  Third, this role did not include a quality control aspect in any 'real' sense. For example, the person did not observe tutors and evaluate performance in any way.  There were no tutor observations, no interviews, no learner interviews, no learner feedback mechanisms regarding literacy and numeracy provision.  If this did occur, it happened in a laissier-fare way.  Your organisations' sole method of evaluating whether tutors are embedding L&N, and the quality of it, is the Assessment Tool results.  Yet these results have not been interpreted in any meaningful way.

I fail to see how you would become aware if no real change occurred in any class.  Would you say you have been negligent in your approach to embedding literacy and numeracy?

CEO: No

McDonell: Oh boy.  Okay, the next questions will relate to your knowledge of your tutors' practice, in regard to ELN.

Invisible numbers, ratios and the eternal cement/concrete conundrum: Musings (part one)

Travelling across the country the last few weeks as a part of the workshop series that I have helped deliver, I have encountered many interesting, strange, and useful things.  The next few posts are a few musings about various things I have learned/observed.

I'll start with the least interesting and move to the weird and wonderful.

First, I learned quite a bit about how adults approach ratios.  A key finding is that adults are largely unaware of the need to identify the 'relationship' that exists between two or more quantities.  I have begun using the term 'invisible number' to create some interest about it and basically 'pizzazz' it up a bit.   

Did you know that being able to recognize the invisible number is a key part of being a 'proportional reasoner'?  Being a proportional reasoner is highly desired as it allows you to do a whole lot of things you otherwise would struggle with.  

Lamon (2006) has estimated that 90% of adults are NOT proportional reasoners despite this being a year seven requirement.  That's right - you should have learned this when you were about 12!  Not being so essentially means that you will be forced to learn algebra by memory, not as a meaningful system, and subsequently struggle and perhaps decide that you don't much like it and would rather play with a ball.  

Below is a power point I used to help make the point.  The participants were asked to make various ratios using Postit notes.  I showed them the following and asked them to model it with the Postits and discuss where else these ratio may apply.

Then I increase the quantities on one side, while removing the other.  They had to determine the amount necessary to add or subtract to ensure the correct ratio.  For example in the Power point below you need to work out how many  Postits would be needed on the right side to complete the ratio.

If the ratio of tan to grey Positis is 2:3 and you have 4 tan Postits, how many grey ones will you have?

Or, imagine you are a caterer.  You are providing one savory and three sandwiches for each person.  The ratio is 1:3.  If you have two savories, how many sandwiches will you need?

See below for the answer.  The tutors would have discussed this and modeled it with real Postits on their desks.

After a while we discussed the 'invisible number'.  The invisible number is the relationship between the two quantities.  However it became just as useful to talk about the invisible number as 'any' relationship (the mathy's out there will know this as invariant relationships).  In the case of the ratio above (2:3) the invisible number is the relationship between the 2 and the 4 tan Postits.  This relationship is double or multiplied by two.  Once this number is identified we simply applied it to the other side.  3 doubled is six.

For the 1:3, we simply doubled our amount, hence double the other side also.

Eventually we ended up here.

Proportional reason according to Lamon (2005, p.4) includes "detecting, expressing, analyzing, explaining and providing evidence in support of assertions".  The tutoring workforce need to work on this to be sure learners are able to develop these skills.

I'm not explaining it too well but the 'invisible number' idea has legs.  I'm going to develop the idea and phrase some more.  Lets see where this takes us.

My next musing will be about the concrete/cement paradox that you never knew about or cared about.  Until now...

Self-regulated learning

The single biggest factor in learner success is a learners ability to self-regulate their learning.  It is the exact opposite to 'learned helplessness' that you should read about here.  Learn about it, understand it, and you will become a better educator.

Of course, self-regulated learning (SRL) is not easy to develop in learners particularly learners who have been beaten around by the educational sector.  Also SRL can be broken down into many sub-skills and dispositions that are generally developed separately but used cohesively.

I used to think that SRL was the secret to developing the potential of adult learners with problematic learning histories.  I used to think it was the answer to the literacy and numeracy problem.  No more embedded ELN, no more literacy or numeracy provision - just SRL.

I now realise that sitting below SRL are belief systems that dictate SRL - beliefs are the mainspring of downstream effects.

As such the form below is no good on its own, even with an extensive PD package that was almost developed and ready to go.

Moving on.

One of the problems with mathematics teachers and resources

I'm playing the devils advocate here - so bear with me.


I've been looking at the age problems used in all algebra courses.

For example: Mary is three times older that Dave.  In 12 years Dave will be one year younger than twice Mary's age.  How old are they both?

I know someone is going to ask, 'why is this useful?  Couldn't we just ask them how old they are?'  And they will have made a reasonable point.

So I Googled, "Why are algebraic age problems useful?" and got:

The purpose of age problems is to determine the age of the people in the equation.

Really?  So the purpose of driving is to drive?  The purpose of eating is to eat?  The purpose of learning to count is so you can count?  Crikey.

Mathematics educators need to provide FIRST and foremost, the reason that this is useful and give practical examples.  The point of learning is not to help facilitate more learning.  Okay, in some cases it may be (developing automaticity for example) but not in regard to reasoning.

But what's the big picture baby?  What's the end game?  Why should I learn algebra?  When will I ever (!!!!) need to calculate someones age based on the crazy data provided?

Now, I can answer the 'why do I need algebra?' question as there are solid reasons (another post). But I'm still struggling with the application of the age problems.

Someone help me.  Give me a real example of the usefulness of the type of mathematics used in age problems in the real world.  Please.

Thoughts on numeracy 'in the real world'

Several weeks ago I asked this question: How do you write one-half as a ratio?  It started a few people journeying into the world of fractions and ratios.

Although it appears easy, it isn't.  Have a think about it, and most of all, 'prove' your answer, no matter how sure you are or how simple you think it is (no, it is not 1:2).

I have been delivering workshops and I have asked all the participants across the country  the same question.

They all all struggle initially, and finally crack it.  But here is where its interesting.  They all think its easy and yell out the 'answer' 1:2.  It isn't 1:2 and its isn't 2:1.

Here is the second interesting thing.  Before I give the question the groups tell me how important ratios are and how they use them almost every day in real world tasks.

They must be wrong, because they all hold the wrong beliefs around ratios, regarding how they work and what they represent. They do not have a conceptual understanding of ratios and therefore have not been directly using them in any of the examples given.

Are we as tutors in danger of mythologizing some aspects of numeracy?  I believe we are.

We definitely need strong numeracy skills, and more than ever, but do we really recognize where it is hiding in our lives?

Food for thought.

Work!!!

The posts have been a little slow as of late because I am currently co-delivering a series of workshops across the country.

So far we have been to Dunedin, Invercargill, Christchurch and Auckland.  What a great group of tutors in each of those places.

There are some fantastic people doing some fantastic things out there.