Analysing the network analysis of cultural operators and drivers
Who are the relevant players in the European Metropolitan Area of Nuremberg? What are their creative networks?
We have taken a close look at the report of the network analysis study of cultural operators in the region and their regional and international networks, as commissioned by the bid office (the official German title of the document is: “Netzwerkanalyse zu regionalen und internationalen Kulturkontakten in der Europäischen Metropolregion Nürnberg”). The bid office has published the report on its website. Overall, we unfortunately need to say that this given analysis does not provide viable insight and result.
We provide a detailed analysis of the work (as PDF published on our website, together with a summary report in our blog, both German only). Our work already takes into account the further findings we have got in the discussion with the bid office N2025. They have mostly not reacted to the specific issues addressed by us, describing our criticism as “academic”. We offered N2025 to publish their position together with ours, but they preferred not to do so.
To specifically name the major issues with the work: the work fails at the very beginning by not establishing exactly which types of cultural actors and contacts should be analyzed. The findings are grounded on address data pre-existing in the cultural administrations and all results are shaped by this. In particular, the results are strongly influenced by their specific area of responsibility. It is even unclear whether the authors may be subject to circular reasoning, since the presentation gives a single reference to the persons studied, and that reference cannot be found in the description of the study design but rather in the study’s results.
The publishers claim that the analysis should merely be “exploratory”. This gives no reason to relax the standard: When considering biased data, the results are most likely to be impacted by the bias, thus featuring some tendencies. When trying to find new hypotheses, there is consequently a danger that while one thinks to have found something new, one only discovers a new representation of an old, well-known pattern.
To be able to make robust statements with regard to culture and the people who shape it, the core of the research assignment would have needed to be defined first.
Such a study may legitimately leave “white spots”. However, any such limitations must comprehensively be presented, their impact needs to be assessed—this work leaves both to be done. To name one of the evident topics that have not been studied: the authors have chosen to analyze regional and international networks but not national ones; which is a decision that seem somewhat random (and we have not found any decision grounds in that regard).
For this reason, we recommend that this so-called study be filed quickly and not used any further; if decisions have already been made on the basis of the outcome, these would need to be critically examined to see whether the bias in the data and the systematic errors of the study design have negatively influenced the reliability of the findings addressed. If this is the case, the decisions would have to be revised.