The inquiry 

McDonell:  I'm going to be asking you questions pertaining to two different but related areas.  The first is to gain clarifications regarding your organisations' definition of embedded literacy and numeracy and the process you took to verify that this aligned with your funders' definition.  Second, what actions you took regarding your quality control measures for your organisations' embedding of literacy and numeracy into your level one and two programmes.

CEO: Understood.

McDonell:  Please explain what your Organisations' understanding of embedded literacy and numeracy is?

CEO:  We have experts who are well grounded in the details.  My role does not require in-depth knowledge of ELN, only that systems are in place to ensure it is.  My understanding of ELN is that literacy and numeracy delivery is integrated into programme delivery.

McDonell: Are you aware that the TEC had released (some time ago) a document that defines its' high level expectations for embedded literacy and numeracy?

CEO:  I was not.

McDonell:  What efforts did your organisation take to ensure it's understanding of ELN was correct to ensure compliance with funding criteria that you were receiving?  

CEO:  We have specialist staff members who's role it is to stay updated.

McDonell:  And it was these, or this, staff members role to ensure the entire organisations' level one and two programmes were aware of the definition and were compliant with it?

CEO:  Yes.

McDonell:  And you assume your staff are aware of the definition, and subsequent expectation for their provision of ELN?

CEO:  Yes.

McDonell:  Well, we are compiling their responses to a questionnaire, and interviews,  as we speak.  We can review the findings tomorrow.  For now, let's continue.

McDonell:  Can you please explain your organisations' process for determining the quality of your embedded literacy and numeracy provision, or even if it was occurring?

CEO: We acquisitioned a staff member to implement ELN across the organisation and ensure tutors were embedding literacy and numeracy.

McDonell:  Can you explain to the house the criteria for selection of the staff member for the position?

CEO: The individual had experience with ELN and was an experienced staff member

McDonell: In what way?

CEO: Sorry?

McDonell: In what way were they experienced with ELN?

CEO:  Ah, they had delivered literacy and numeracy.  And they had been to a range of workshops.

McDonell: In what capacity had they delivered ELN?

CEO: I'm not sure I understand the question?

McDonell:  How do you know they were qualif... I'll rephrase.  What were your organisations' appointment criteria for this position?  For example what qualifications did the position require?

CEO:  I'm not sure.

McDonell:  Can you at least tell us what the position description defined as qualified?

CEO:  I can't.

McDonell:  So your appointment of the position went to an individual that did not meet any criteria other than having experience with literacy and numeracy that cannot be defined?  We don't really know what they did do we?  Or their quality? You see the problem don't you?  How do we know they didn't get their previous job based on having experience somewhere else?  We really need some criteria of their expertise that is not simply holding a role.

CEO:  They are a trusted staff member.

McDonell:  Now this individual was tasked with determining whether literacy and numeracy was being embedded and whether it was of  a sufficient quality to warrant accepting funding for it?

CEO:  They were tasked with making sure it was occurring.

McDonell: Not quality?

CEO:  Well yes, PD was provided to tutors.  Tutors were trained.

McDonell:  And the outcome of that training was?

CEO: To embed literacy!

McDonell:  Yes, and I am asking you what internal systems were in place to ensure this was in fact happening and that the quality was of a sufficient level to warrant receiving funding.  Did your staff member report to the organisation on the quality of provision?

CEO:  We use the TECs' Assessment Tool to determine the effectiveness of ELN provision.

McDonell: So the staff member had no mechanism to report back on quality?  Did the staff member actually review quality?

CEO:  As I said we have an extensive assessment process.

McDonell:  So I will assume that despite vying for and receiving funding that required quality ELN provision you had no internal mechanism for determining whether or not literacy and numeracy was being embedded in programmes, or that the quality was sufficient, other than your Assessment Tool results?

CEO: Correct:

McDonell: So there were no tutor observations, no interviews, no student interviews, nor internal reviews?  How do you know your tutors' were making any change to their practice, let alone doing a good job?

CEO:  I trusted my staff member to tell management if it required attention.

McDonell: But you will admit no formal mechanism existed?  Despite an extensive shift by the NZQA toward internal evaluation systems?  Out of interest did the staff member ever give an indication that tutors were not meeting the requirements of the funding?

CEO:  No.

McDonell:  That seems extraordinary given your present situation.  Surely you see that you would not be here today if things were operating as you presumed?  I'll move on.  The data from your Assessment Tool results have been largely static over the last four years and suggests that you have failed to improve the quality of your provision each year. What does this suggest to you?

CEO: Actually our results show that many learners do improve their literacy and numeracy.

McDonell:  I'm not talking about improvement within a given year.  I'm talking about a trend of improvement.  One year compared to the next.  Yearly improvements suggest your organisation is improving.  Your results do not.

CEO:  That is due to a range of issues in the sector such as the nature of the learners' skills, attrition rates, transience, social issues and a raft of ever changing TEC funding criteria.

McDonell:  Do you think that it is reasonable that a tutors' performance will improve the longer they are in the job?

CEO:  For the reasons I mentioned earlier, not necessarily.

McDonell:  In every other educational domain learner outcomes improve as tutors gain experience. Please explain to the house why this is not true in your organisation?

CEO: Tutors are busy, they have demanding students, the schools have failed and we have to meet unreasonable outcomes to continue in business.

McDonell:  Then it would seem reasonable that your organisation should have exceptional evaluation and training systems in place - particularly in funding sensitive areas such as embedded literacy and numeracy?

CEO:  We do have evaluation systems.

McDonell:  You made the statement that the assessment tool is your primary measure of ELN?  Two questions.  What other measures are you using to determine the quality of ELN?  And second, did the results of the assessment tool suggest in any way  that the PD you speak of improved learner outcomes?

CEO: In answer to the first, as I just said, no, the Assessment Tool is our measure.  As to the second improving tutor performance takes time.  We need time to bring the staff up to speed, and we are doing this.

McDonell:  The truth is though, isn't it, that because you haven't implemented systems to inform you of the internal provision of ELN that you don't actually know the answer to either of those two questions?

CEO: Not at this moment, no.

McDonell:  Tell me if you disagree with my summary.  You selected a staff member based on a trivial notion of experience.  No criteria existed for this role despite this role overseeing significant funding streams deemed priority areas by the Government.  Second, no formal mechanism existed for this individual to feed back concerns to management regarding quality.  I suspect the individual was left without support or direction from management.  Third, this role did not include a quality control aspect in any 'real' sense. For example, the person did not observe tutors and evaluate performance in any way.  There were no tutor observations, no interviews, no learner interviews, no learner feedback mechanisms regarding literacy and numeracy provision.  If this did occur, it happened in a laissier-fare way.  Your organisations' sole method of evaluating whether tutors are embedding L&N, and the quality of it, is the Assessment Tool results.  Yet these results have not been interpreted in any meaningful way.

I fail to see how you would become aware if no real change occurred in any class.  Would you say you have been negligent in your approach to embedding literacy and numeracy?

CEO: No

McDonell: Oh boy.  Okay, the next questions will relate to your knowledge of your tutors' practice, in regard to ELN.