Key Performance Indicators (KPIs)

At the end of 2002, ECU updated its previous Strategic Plan, and produced a new ECU Strategic Plan 2003-2007 . While the University mission remains the same, the new Strategic Plan is underpinned by five Strategic Priorities:

1. Enhancing Teaching, Learning and Research

2. Engaging with the Professions and Professional Life

3. Building Partnerships, Pathways and Precincts

4. Improving Outcomes for Students and Staff

5. Strengthening Enterprise and the Resource Base

From these Strategic Priorities, a number of Effectiveness and Efficiency Indicators have been condensed from the University's full set of KPI's. These selected KPI's, detailed below, are also generally consistent with those used elsewhere within the Higher Education Sector, and focus on the University's core business (teaching, learning and research), and core external stakeholders (students).

Effectiveness Indicators:

1. Enhancing Teaching, Learning and Research

ECU is focused on

“ the education of learners for the knowledge based service professions, and will be recognised for the quality of its teaching and its learning opportunities in those fields. ECU is a teaching and research university, and research will inform its teaching. ECU will be sought out to participate in research projects and productive partnerships. This will be because of the University's excellence in collaboration and the delivery of research outcomes that contribute to policy development and improved practice in the professional fields of business, human services and education, and to advances in health, information and communication technologies and the environment.” ECU Strategic Plan 2003 – 2007, page 9

Key Performance Indicators related to these outcomes are –

• Retention Rate
• Course Satisfaction
• Quality of Teaching
• Research Funding
  

1. Retention Rate

The Retention Rate is used to indicate how effective the University is at retaining its students from one year to the next. Many factors have impact on a student's decision to remain at a university. These include university-controlled factors such as course content, teaching styles, learning support, and institutional infrastructure. They also include factors such as individual expectations, financial and social circumstances over which the university has little or no influence. Nevertheless, this indicator is commonly used to show an institution's ability to retain its students.

Student Retention has been a high priority for the University's teaching and learning activities and is a focus of the Teaching and Learning Management Plan. Edith Cowan University has maintained relatively strong performance in its retention rate for 2002. The 2002 result is a further improvement on the 2001 result and consolidates a period of improvement in this area. While there are many factors contributing to performance against this indicator, the result is consistent with the University's expectations of strong performance resulting from continued commitment to Service, Professionalism and Enterprise . This is a lagging indicator, reflecting the delay in its production by DEST. Results for 2002 are the latest available for the current reporting period.

Source: DEST Higher Education Statistics Collection Return, Edith Cowan University.
Definition: The percentage of students who commence a Bachelor Pass course in a given year and either complete, defer or are still enrolled in the same or an alternative course one year later .

 2. Course Satisfaction

Course Satisfaction is used to indicate how satisfied students are with the education and service provided, given their expectations and experience of their course. This indicator is derived from the results of the Course Experience Questionnaire (CEQ), which is conducted annually by the Graduate Careers Council of Australia (GCCA). The number of 2002 Bachelor Pass graduates surveyed in total during 2003 was 3125, of whom 1493 responded for a 48 percent response rate. This is a lagging indicator, reflecting the delay in its production by GCCA. Results for 2002 are the latest available for the current reporting period.

The University has continued its strong performance in this indicator and is on par with the sector. While there has been a decrease of 2.2 percentage points on Course Satisfaction between 1999 and 2000, this follows an increase of 3.4 percentage points in the previous year and the current level is still well above the result for 1998. Since 2000, the University has increased the level of Course Satisfaction each year.

The Graduate Careers Council of Australia states that comparison of CEQ results between disciplines and universities is not recommended. This reflects the different student cohorts, subject matter, and professional requirements that make such comparisons problematic. The Graduate Careers Council of Australia suggest that bona fide comparisons can only be made between like disciplines across institutions using trends over time.

Source: National Course Experience Questionnaire results 1998-2002 graduates.
Definition: The percentage of Bachelor Pass students who ‘broadly agree' (score #3 or #4 or #5 on a five-point Likert scale) with the overall satisfaction statement from the Annual Course Experience Questionnaire (CEQ).

3. Quality of Teaching

Quality of Teaching is defined as the mean of responses to Items 1, 3, 10, 15, 16, and 27 from the annual Course Experience Questionnaire (CEQ) for Bachelor Pass graduates, run as a national survey by the Graduate Careers Council of Australia (GCCA).

The Items are –

• Item 10 – The teaching staff of this course motivated me to do my best work
• Item 1 – The staff put a lot of time into commenting on my work
• Item 27 – The staff made a real effort to understand difficulties I might be having with my work
• Item 3 – The teaching staff normally gave me helpful feedback on how I was going
• Item 15 – My lecturers were extremely good at explaining things
• Item 16 – The teaching staff worked hard to make their subjects interesting

The mean has a possible range of –100 to +100. This measure is more specifically related to teaching than Course Satisfaction , which draws only on the CEQ's Overall Satisfaction scale. A score of –100 (an average score of 1 on 5 point Likert Scale) translates to “Strongly Disagree”, 0 (an average score of 3 on a 5 point Likert Scale) translates to “Neutral” and +100 (an average score of 5 on a 5 point Likert Scale) translates to “Strongly Agree. This is a lagging indicator, reflecting the delay in its production by GCCA. Results for 2002 are the latest available for the current reporting period.

The University's results have shown an upward trend for all years other than 2000, where the measure dropped to 9 from 12 in 1999, before increasing to 14 in 2001. ECU's performance sees it ranked at around the mid-point of Australian universities on this measure.

Source: National Course Experience Questionnaire results 1998-2002 graduates

4. Research Funding

Research Funding obtained during a prescribed year, in terms of DEST Categories 1, 2, 3 and 4 is a measure of the University's research activity .

It should be acknowledged that levels of Research Funding could be strongly influenced by the receipt of large competitive ARC grants. The relatively small numbers of such grants may limit the capacity of the institution to maintain performance at a high level. This is a lagging indicator, reflecting the delay in its production by DEST. Results for 2002 are the latest available for the current reporting period.

The University's performance is trending higher over the period and all categories other than Category 1, are higher in 2002 than in the previous year. While the trend is broadly upward, the specific variation by category is reflective of the vagaries of the competitive grant process, and the pipelining effects of relatively small numbers of comparatively large grants working through the system.

Source: DEST Annual Financial and Research and Development Collections Return, Edith Cowan University
* Note: Category 2 - Other Public Sector Research Funding has been amended from the last year amount of $7.424M to $2.724M, in line with DEST definitional specifications .

Improving Outcomes for Students

Desired Outcomes -

“ECU will provide a challenging and supportive environment suited to the students that study at the University, so they can realise their potential and develop the skills and flexibility needed to succeed in their careers. It will shape policies, programs, services and infrastructure to foster the active engagement of students in support of ECU's strategic positioning as a leader in education for the service professions. This priority will be outcomes focused.” ECU Strategic Plan 2003 – 2007, page 16

The Key Performance Indicators related to these outcomes are –

• Graduate Employment
• Growth in First Preferences

5. Graduate Employment

Graduate Employment is used to indicate the extent to which graduates find employment and is a measure used by the sector to indicate the quality of learning outcomes. The indicator is limited to Bachelor Pass and Honours students to remove any skewing of the data by research and higher degree graduates (many of whom already have jobs in their chosen professions). This indicator is defined in such a way as to factor in the desired employment mode (full-time or part-time) of graduates. As such, a graduate working part-time but not seeking full-time work is included as ‘working in the mode of their choice'. In recognition of the diverse nature of the University's students, graduates are included irrespective of their mode of study. Results for 2002 are the latest available, as again this is a lagging indicator reflecting the availability and production of data from GCCA.

The indicator shows an increase between 1999 and 2000, continuing the growth between 1998 and 1999. The increase of 3.9 percentage points between 1999 and 2000 reflects an improvement in the proportion of graduates who have secured full-time employment, coupled with a continued reduction in the proportion of graduates in part-time employment who are seeking full time employment.

Between 2000 and 2002, there is a decline in graduate employment. This is mostly explained by the downfall in number of jobs offered in IT related fields, Marketing & Tourism, and Performing Arts areas. These declines reflect State and national trends.

The GCCA states that comparison of CEQ or GDS results between disciplines and universities is not recommended. This reflects the different student cohorts, subject matter, and professional requirements that make such comparisons problematic.

Source: Annual Graduate Destination Survey, Edith Cowan University
Definition: The percentage of Bachelor Pass and Honours graduates working in the mode of their choice as a percentage of the total number of graduates seeking work. Mode relates to full-time or part-time.

6. Growth in First Preferences

This Key Performance Indicator is important since it is the most public expression of the level of demand for entry to the University within the broader competitive market and reflects the composite achievements of many initiatives and the general perception of the University by potential students and the community.

Since 2001, the University has increased its share of first preferences with 2003 showing a marked increase and representing an historic high. Part of the increase reflects ECU adopting a practice in line with other W.A. Universities by including its University Preparation Courses within TISC's preference management system. (Previously, students from these courses applied directly to the University for entry, and were thus not included within TISC statistics.) To this extent, the level of growth is partially an artefact of the change in reporting system. Beyond this, however, the figure still represents a significant growth of share for ECU in 2003.

It is to be expected that changes in the national Higher Education Sector as a result of the Commonwealth Government's Our Universities: Backing Australia's Future reforms may cause some volatility in this indicator over coming years.

Source: Tertiary Institution Service Centre
Definition: Each WA University's share of First Preferences processed by TISC

7. Research Higher Degree Completion Rate

Research Higher Degree Completions Rate is defined as the number of Higher Degree by Research (HDR) completions per year. This figure is important because it shows the level of throughput of higher degree research students, which is in turn an indirect indication of the appropriateness of student selection and the quality of higher degree supervision. Completions, however, need to be considered in relation to the size of HDR load if they are to be used as a measure of the efficiency of research training.

There are two significant spikes in 2000 and again in 2002, which reflect the comparatively higher numbers of graduate completions in Arts, Humanities, Social Sciences, Education and Business & Administration.

Note: Previous year annual report figures were using three year rolling average which has now changed to annual figures only.
Source: DEST Higher Education Statistics Collection Return, Edith Cowan University .

8. Research and Development Publication Research per 10 Staff - DEST

This indicator shows the research output in terms of publications per 10 research academic staff of Level B and above (see table below ). The indicator shows that the University has matched or increased in all categories since 2001, with the exception of category E1 & J1. The University's Research and Research Training Management Plan incorporates strategies for further improving the University's research performance, and is intended to lead to growing numbers of research publications in the future. This is a lagging indicator, reflecting the delay in its production by DEST. Results for 2002 are the latest available for the current reporting period.

Source: DEST Annual Research and Development Annual Collection, Edith Cowan University.
Definition: The number of publications in DEST categories A1, B, C1, E1 and J1 per 10 FTE academic ‘teaching and research' and ‘research only' staff at Level B and above, produced during a prescribed year .

9. Teaching Related Expenditure per EFTSU

This indicator is included to show how efficiently the University uses its funds for teaching and learning. The indicator has been refined from that used in previous years by broadening the scope from ‘Teaching Expenditure' to ‘Teaching Related Expenditure'. Teaching Related Expenditure is defined as Total Expenditure less Research Only Expenditure. This enables data to be reported annually rather than biennially, as was previously the case.

The figures for 2000 to 2003 show that Teaching Related Expenditure / Teaching Load (2003 dollars) for 2003 has increased relative to the trend of the previous three years. Significant increases in 2003 relate to computing costs (hardware and software) and student costs (amenities and services). The University will continue to explore opportunities to control Teaching Related Expenditure while improving learning experiences for students.

* CPI is Perth All Groups as at December Quarter
Source: ECU General Ledger and DEST Higher Education Statistics Collections, Edith Cowan University.
Definition: Teaching Related Expenditure expressed per EFTSU (includes TAFE EFTSU) .