problems in evaluating informal education

The forms of evaluation used in the formal education sector and in most other areas pose problems for informal educators.

Getaway GirlsIn recent years informal educators have been put under great pressure to provide ‘output indicators’, ‘qualitative criteria’, ‘objective success measures’ and ‘adequate assessment criteria’. Those working with young people have been encouraged to show how young people have developed ‘personally and socially through participation’. So what are the main problems?

First, the different things that influence the way people behave can’t be easily broken down. For example, an informal educator working with a project to reduce teen crime on two estates might notice that the one with a youth club open every weekday evening has less crime than the estate without such provision. But what will this variation, if it even exists, prove? It could be explained, as research has shown, by differences in the ethos of local schools, policing practices, housing, unemployment rates, and the willingness of people to report offences.

Second, those who may have been affected by the work of informal educators are often not easily identified. It may be possible to list those who have been worked with directly over a period of time. However, much contact is sporadic and may even take the form of a single encounter. The indirect impact is just about impossible to quantify. Our efforts may result in significant changes in the lives of people we do not work with. This can happen as those we work with directly develop. Consider, for example, how we reflect on conversations that others recount to us, or ideas that we acquire second- or third-hand. Good informal education aims to achieve a ripple effect. We hope to encourage learning through conversation and example and can only have a limited idea of what the true impact might be.

Third, change can rarely be monitored even on an individual basis. For example, informal educators who focus on alcohol abuse within a particular group can face an insurmountable problem if challenged to provide evidence of success. They will not be able to measure use levels prior to intervention, during contact or subsequent to the completion of their work. In the end all the educator will be able to offer, at best, is vague evidence relating to contact or anecdotal material.

Last, there is an issue with timescales. Change of the sort with which informal educators are concerned does not happen overnight. Changes in values, and the ways that people come to appreciate themselves and others, are notoriously hard to identify - especially as they are happening. What may seem ordinary at the time can, with hindsight, be recognized as special.

infedcov.jpg (18462 bytes)Taken from Tony Jeffs and Mark K. Smith (2005) Informal Education. Conversation, democracy and learning, Nottingham: Educational Heretics Press.

© Tony Jeffs and Mark K. Smith
First published November 1999. Last update: July 08, 2014