Assessing quality, measuring results – Rafael Carpanez

Rafael Carpanez –

LAMSIG – Bruna Caltabiano, Christina Riego, Marcelo Dalpino, Rafael Reis, Sergio Monteiro
Leadership and Management SIG aims at:

• Raising awareness and contributing to the development of good management practices within ELT;
• Providing opportunities for discussion, as well as courses and events concerning management issues;
• Helping improve quality standards in ELT.

Contact: lamsig@braztesol.org.br
Education in general is a very difficult topic to be discussed because everyone has a really strong opinion about it. Teachers, parents, students, school administrators and educators have strong beliefs concerning what education should be, how it should be conducted and who should be responsible for such actions. In this sense, we could all recognize that it is a complex area to work on, but society and governments have been interested in it due to globalization and the effects it has on education.
Fortunately, we live in a world that is always seeking for improvement. Companies look for processes and human resources in order to achieve the best results and we should ask ourselves why not to do the same when it comes to education.
One alternative educators have found to keep up with globalization and market trends is by assessing education and learning.  According to Rowe & Lievesley (2012), “Education systems have been changing to improve the quality of education by means of using Performance Indicators.” Performance indicators (PIs) can be defined as data or information by which the quality of institutions may be measured and evaluated. They also provide evidential parameters for determining the extent of goals and objectives.
 
By using performance indicators, we are able to assure the quality of services we are offering our customers. On the other hand, one might argue that it is virtually impossible to assess education, because it is such a complex area to measure results – there are some characteristics of our service which make it difficult, such as:

• Intangibility: it seems to be difficult to provide tangible evidence of service provision (teaching & learning)
• Inseparability: there is simultaneous production and consumption of the service. Also, the consumption generally takes place in the presence of other consumers which causes the service to be dependent not only on the service itself, but also on other consumers.
• Variability: standardization seems difficult because of heterogeneity.

However, schools and institutions all over the world have proven otherwise. When it comes to school improvement, it is easy to find literature and case studies of institutions that have put performance indicators to the test and the result is no magic, it simply works.
 
Institutions are usually concerned about two kinds of results, business and academic. Every school or institution wants to achieve financial and academic success and the best way they have found to do so is by checking the indicators against what has been found in previous years. By doing that systematically, the institution is sure about the results and changes that took place and even draw some conclusions on how it all happened. If they also write a business plan, they are able to change, adapt and reflect on strategies for future plans.
In the chart below, you will find examples concerning the processes that lead to academic and business results. By identifying each of the topics below inside the institutions, educators should be able to have a broader view of what is happening in each area and make changes accordingly in order to reach better results.
 

 
 
As aforementioned, PI’s  should be aiming at both academic and business results, providing the manager with some kind of guidance in the strategic decision-making process, in a way the administrator is able to assess each indicator and for that reason, shouldn’t be done all at once. Understanding the indicators goes beyond reading the numbers, it implies considering the big scenario, reflecting upon the cause-effect relation of actions, relating results and generating improvement and professional development.
 
An important guiding principle is that PI data should not be collected for their own sake, but rather, because they are needed for specific policy purposes. For instance, if a private language school would like to improve its business results, it might start by assessing their customer satisfaction and from that point on design a plan to improve this element only. By doing that systematically over the years, the school will have a record of data and it will be easy to draw conclusions, plan better strategies and think of all kinds of ideas in order to assess that indicator. From that point on, the school can choose to work on different indicators until the administrators are satisfied with the overall results of each of them. It is really like learning how to ride a bike, we have to start step by step and we only move forward when we have perfected the skill that is necessary for a more complex move.
According to Fullan (2014), “…creating a general inspiring vision and instilling motivation in teachers to join the cause were not specific enough to produce actual results.” We have mentioned some of the characteristics which make our service difficult, but one of our main challenges, in the educational field, is to make teachers aware of the indicators relevance and its applicability in his/her outcome. That is why is so determinant to have a set of PIs that assures relevance, validity, cost effectiveness and result-orientation. It is possible to consider several indicators in the educational field; in the chart below, some possible PIs that can be part of a school business strategy are listed. The decision of which one to consider is based on the institution’s vision, mission and values and orientation to results. The whole team has to be aligned in terms of knowing the indicators, understanding their relevance, measuring progress and establishing strategic actions. PIs are not a matter a control, but a matter of having right maps that will help you see beyond and define a path to follow.
Find below some of the most common PIs used in schools and the impact on results:

 
Once the student has started studying at a school, the director/coordinator needs to monitor his/her satisfaction and align provision with expectation. The use of PI’s allows us to care for customers, both external and internal, bringing the best results.
References

• Fullan, M. The principal – Three Keys to Maximizing Impact. San Francisco:Jossey-Bass, 2014.
• Rowe & Lievesley. Constructing and using educational performance indicators. Acer, 2002.

https://research.acer.edu.au/learning_processes/11

• Scheerens et al. Measuring Educational Quality by Means of Indicators UNECO, 2011.
• White, Ron et al. Management in English Language Teaching. Cambridge: Cambridge University Press, 1991.
• White, Ron et al. From Teacher to Manager . Cambridge: Cambridge University Press, 2008.