The art of (not) only measuring culture: Evaluation in the theatre and cultural sector
The book by Jenny Svensson - by now only published in German under the title Die Kunst, Kultur (nicht) nur zu messen: Evaluation im Theater- und Kulturbetrieb - presents a functional model of evaluation measures of public funded cultural enterprises. Thus, it makes a fruitful contribution to the path to further professionalization of the cultural industry and cultural management.
Content of the book
In her dissertation, Jenny Svensson discusses the topic of evaluation in theater and the cultural industries. The study, published in 2017, aims to identify industry-specific problems of evaluation measurement and to present methods from practice. Based on this, the author shows the goals and functions of evaluation for different stakeholder groups. Finally, possible outcomes and factors for the sustainable use of evaluations are of interest.
The first third of the book deals with theoretical fundamentals and the following two-thirds with the empirical part. Field access was provided by five theatrical institutions in the southern Swedish province of Skåne. Here, both organisationally implemented evaluations - such as on production and internal communication processes - as well as a selection of evaluation projects of cultural policy goals are examined. A conclusion from both perspectives concludes the work.
Evaluation as a cultural management tool
The boundaries of traditional separation between culture as production of meaning and economy as commodity production are increasingly blurring. At the same time, the legitimacy of cultural institutions is often controversially discussed: they are dependent on funding and widely compete with many other offerings for the time and attention of potential users. However, in practice, especially at publicly funded cultural institutions, a supply-led approach is frequently used, although legitimacy is essentially constructed from feedback from both external and internal stakeholders. For cultural management, evaluation is still a young field that goes beyond the controlling approach of business administration. A management not of but for culture cannot and should not waive this. At this point Svensson starts her study.
An explorative approach to a complex topic
Jenny Svensson introduces the topic extensively, starting with philosophical foundations on historical and social influences on evaluation approaches up to concrete questions of the evaluation practice (What?, Why?, On the basis of which criteria?, By whom ?, How?). Readers are given a holistic overview of the research subject. In the field of art and culture, addressees often show contradictions and skepticism to evaluations. On the basis of her literature study, Svensson demonstrates that artistic action is often characterized by irrationalities: The preservation of power structures on the one hand often plays a more important role than the findings of evaluations or other investigations. On the other hand, evaluation-demanding groups (donors) know too little about the complexity of cultural production processes.
The tensions between artistic autonomy and the social relevance of cultural offerings are skilfully presented by Svensson using the Value and Development Square, a model by communication scientist and psychologist Schulz von Thun. Although the term relevance is sometimes overused in literature and little defined for the respective matter, the author manages filling it with cultural-political and cultural-operational missions of the examined actors in order to build up the empirical part of the study. These missions are referred to in the book as goals, a term that in practice is often used interchangeably, but strictly speaking is somewhat inaccurate.
In order to approach this by now only rarely researched subject, Svensson sensibly chooses an exploratory approach. This is well founded in the book, since evaluation in the arts sector has a wide range of practices and understandings of terms. Thus no valid quantitative measurement of evaluation is possible. The data was collected face-to-face via semi-structured expert interviews (n = 18) and further (n = 28) interviews via e-mail or Skype. A qualitative content analysis according to Mayring was used as an empirical evaluation method. Documentations of completed evaluation measures were used as additional data sources.
Examples and practical relevance
The findings and conclusions are consistently well documented with exemplary interview passages. These anchor examples create proximity and help bridging the gap into the practice of the readers. An example: "[...] it does not have to be primarily about more money, but flexibility to realize the possibility of something," said the artistic director of the opera workshop, Malmö (p.419). Svensson examines concrete evaluation projects on artistic quality, diversity and participation as well as on children, youth and audience orientation. She vividly explains the problem of ambiguous conceptual and analytical dimensions: How should artistic quality be assessed? What does cultural diversity mean? What exactly does "new" audience mean? These questions show that parts of the problems of evaluation - as stated above - are already grounded in the supposed goals that they should evaluate. This connection as well as the potential problematic, because positive connoted all-too-common concepts (who is not for quality, diversity and new audience?!) are short, but sufficiently set out.
Another strength of the study is the practical, extensive presentation of the very heterogeneous data collection approaches. Practitioners thus get a good overview of the strengths and weaknesses of a variety of different methods and may get inspired to own survey designs. In addition to the classic functions of evaluation (knowledge, control, development and legitimization), the author also identifies two other essential functions for the cultural industry: communication and identity creation. Svensson sufficiently points out the frequently limited reliability and validity of the data collected in the examples. Therefore their interpretation and conclusions should be treated with caution. Moreover, she emphasizes on the fact that study designs in evaluation practice often do not meet scientific standards. Nevertheless, Svensson also shows advantages of explorative, partly ad-hoc evaluation methods as processes of awareness and self-reflexivity. This dialectic is an integrated part of evaluation in her own functional model. Svensson concludes her study on the importance of self-initiative for the sustainable use of evaluations. Very clearly, she emphasizes the importance of references to various stakeholder groups, which must also be used more intensively for evaluations.
The book is highly recommendable for practitioners in cultural and cultural policy as well as for teachers and advanced students of cultural management. It shows the importance and necessity of a methodologically founded reflection of own actions for the cultural industry. So the question is rather not if, but how it is evaluated. For this purpose, the study opens up a broad range of impulses, even though they are focused on a narrow field - which is scientifically a good thing. The reading can help to get to know a range of evaluation methods and not just to understand them as art-remote control. Thus, Svensson gives good arguments to counter industry-typical skepticism towards evaluations. Also for cultural enterprises, which are concerned with the idea of an ISO certification, the process-oriented evaluation approaches can provide appropriate suggestions.