Unit and Teaching Evaluation System

Staff can access UTEI Reports Here

The current UTEI forms are available here:

• UTEI form
• UTEI form (Simplified Chinese)
• UTEI Supplementary Form
• Instructions

These PDF forms should be completed with the unit and lecturer details and then printed on an SOE laser printer to ensure accurate result scanning.

An instrument such as the UTEI has two principle purposes. The first is diagnostic, enabling staff to identify the strengths and weaknesses in the unit and their teaching. The second is that of monitoring units and their teaching over time.

The Unit and Teaching Evaluation Instrument (UTEI) is a standardised three-part questionnaire (ie no choice of items) that provides for

• unit evaluation of each unit, and within the unit,
• lecturer evaluation of each lecturer, and
• tutor evaluation of each tutor.

Thus the UTEI separates the teaching of each unit into two generic components:

• lecturing - organising the content and providing class instruction (eg. making clear the standard of the work expected, making the content interesting and engaging etc.);
• tutoring - conducts classes (eg. encouraging and supporting student learning, making clear what students are expected to do and learn etc.) but also assesses student work providing feedback and guidance.

Each staff member teaching a unit is evaluated either as a lecturer only, a tutor only, or both a lecturer and a tutor.

By providing student feedback data on individual units and teachers, the UTEI is an important quality tool enabling University staff to monitor and evaluate the effectiveness of the teaching and learning across the University.

It is University UTEI policy that the UTEI be administered across the entire unit each time the unit is conducted. Thus the UTEI is to be administered in each teaching period, evaluating each unit and each staff member teaching it, irrespective of when the unit is offered, how the unit is taught or where the unit is taught. Policy also specifies that the purpose of student evaluations of units and teaching is to review the effectiveness of the environment for teaching and learning in the University.

The UTEI unit items, lecturer items and tutor items each comprise:

a set of positively worded core items that use the standard five-point Likert agreement scale:

two open-ended questions inviting written feedback from the student.

The UTEI questionnaire comes in two paper-based forms:

• standard form which is always used for evaluating the unit, and is sufficient for classes where individual students have no more than one lecturer and one tutor during the semester for the unit; and the
• supplementary form which is used when individual students have had more than one lecturer and/or tutor during the semester for the unit.

Each form is a single page, two-sided A4 response sheet, with sections as follows.

Students in a unit complete the UTEI questionnaire by completing the unit section together with a section for each tutor and a section for each lecturer they had during the semester.

The method of collecting the UTEI data must be sufficiently flexible to cover the variety of class schedules used, yet be cost-effective. All UTEI data is currently collected using one of three possible options.

1. Standard Collection: centrally organised using non-ECU staff to collect the bulk of the data during the standard collection fortnight where this is cost effective.
2. Standard Collection with Preference: as for 1 but with a date/time preference.
3. Non-standard Collection: School/Faculty organised and conducted, coordinated by the School/Faculty UTEI Coordinator, where it is inconvenient or impossible for the data to be collected via 1 or 2 above.
4. Online: with the School/Faculty informing students when and how they may respond online to the UTEI questionnaire.

The standard collection fortnight comprises the semester weeks 11 and 12 of the 13-week (Pattern 1) University timetable. Collecting UTEI data online assists with the paper-based data collection process, particularly from offshore classes and students studying externally or in fully online mode.

Variability in how the UTEI is administered can affect the quality of the data collected. The validity and reliability of the data are improved when procedures for collecting the data are standardised and meet the following administration standards.

The Evaluation Officer's Manual contains all the information necessary to administer the UTEI questionnaire to the required standards. A PowerPoint summary of the key points from this manual may also be useful for training new staff as UTEI data collectors.

UTEI data is both:

• quantitative - in the computer-generated reports, and
• qualitative - in the information contained in the student responses to the open-ended questions.

The UTEI system generates unit, lecturer and tutor reports at each of the reporting levels, individual, School and University. Thus there are:

• Individual Evaluation Reports - for the staff teaching the unit;
• School Evaluation Reports - for the Head of School;
• University Evaluation Reports - for senior University staff and the University community.

All University UTEI unit, lecturer and tutor reports from 2002 semester two to date, are publicly available.

Click here to access University UTEI reports

From 2004 semester two, University teaching staff access their UTEI reports from their desktop computers via the ECU Web Portal at: portal.ecu.edu.au

UTEI reports available depends on the staff member's teaching role. As a Lecturer/Tutor, each staff member has access to the three public University reports, their own individual:

• Unit Evaluation Reports for the campus (or campuses) taught;
• Lecturer Evaluation Reports;
• Tutor Evaluation Reports;

and their School Unit Evaluation Report.

Unit Coordinators and Course Coordinators have access to all individual UTEI reports, unit, lecturer and tutor, in their respective areas of responsibility. Heads of School, listed as School Coordinators in the online UTEI system, have access to the five School UTEI reports. The five School UTEI Reports

Prior to 2004 semester two, all UTEI reports were available from a website via a School-specific password-protected directory.

Click here to access UTEI reports from 2002 semester two to 2004 semester one. You will need the School password.

The UTEI data is used in different ways by ECU staff at different levels of responsibility. Individual academic staff need to use UTEI data to:

• to identify strengths and weaknesses in their units and teaching, in order to make improvements;
• to report to the Head of School as part of M4P;
• for probation, contract renewal and promotional purposes.

When reading an individual UTEI unit lecturer or tutor report, first compare each individual scale value at the bottom of the report with the with the corresponding value for the unit, School and Faculty. This comparison locates each scale evaluation as above or below that for the unit, School and Faculty. For more detail on each scale, scan the numeric data for the core items corresponding to the scale at the top of the report. High mean and the percentage agreement item and scale scores identify strengths, low mean and the percentage agreement item and scale scores identify weakness and hence the opportunities for improvement.

Relate the numeric information in the UTEI report to the qualitative information. Qualitative information, in the responses of students to the open-ended questions, also contains information useful for diagnosing strengths and weaknesses in the unit or its teaching.

Use this qualitative information to validate the information in the UTEI report and to provide quotes that can be used in a report to give understanding.

The Academic Promotion Policy specifies that applicants must provide:

• Evaluations of a unit for which the applicant was the Unit Coordinator and the applicant as a lecturer and/or tutor for at least three (3) unit offerings, in accordance with Academic Board expected standard.
• Evidence of critical reflection identifying strengths, weaknesses and changes, in on-going development and improvement of teaching and learning:

This requires staff to use the UTEI data, identifying strengths and weaknesses and changes, in an on-going improvement cycle.

Unit and Course Coordinators, Heads of Schools and Executive Deans of Faculties need to monitor and evaluate the quality of teaching and learning within their respective areas of responsibility. These staff need to use UTEI data to:

• demonstrate the process and results of such evaluations to their Line Managers as part of the M4P process;
• assure the quality of teaching and learning for self-review and academic audit purposes, particularly for Faculty Annual Reviews and Area of Scholarship Reviews.

It is sometimes argued that UTEI data is not a valid source of information about units and teaching. Students respond anonymously on the basis of what they think and feel about their learning experiences. Despite this, students do a pretty good job and provide staff with a valuable source of data for reflecting on each unit and its teaching.

The UTEI is the third centralised instrument that the University has used for student feedback of units and teaching. Please note that data generated with the UTEI is not compatible with either of the former two instruments.

The relevant details of each instrument are as follows.