EdTechIE18 presentation slide
Conferences Research

The quality of metrics matters: looking ahead to #EdTechIE18

I am really looking forward to taking part in ILTA’s upcoming conference, TEL Quality Matters – People, Policies and Practices, 31 May – 1 June at IT Carlow. You can see the full programme here http://programme.exordo.com/edtech2018/ .

Working with ALT’s new strategic working group for the development of the Open Access journal Research in Learning Technology, I have been working on understanding more about alternatives to the established Impact Factor for independent Open Access journals generally and more specifically for researchers in Learning Technology (revisit my first and second post on this subject) and next week’s conference is a valuable opportunity for me to meet some of the colleagues who are part of the group and find out more about their experiences.

Meanwhile I have come across two new interesting articles (thanks to Neil Morris for making me aware of the blog post).

The first is another interesting post from the LSE Impact BlogThe academic papers researchers regard as significant are not those that are highly cited . The authors describe the current perspective as follows:

Citations, JIF, and h-index have served as the triumvirate of impact evaluation for many years, particularly in STEM fields, where journal articles are frequently published. Many studies have pointed out various flaws with reliance on these metrics, and over time, a plethora of complementary citation-based metrics have been created to try and address various proficiencies. At the same time, we see altmetrics emerging as a potential alternative or complement to citations, where we can collect different data about the ways in which research is viewed, saved, and shared online.

The authors share useful insights from their work surveying chemistry researchers to “gauge their perceptions of significance, importance, and highly cited materials. The results, while not truly startling, were nevertheless a stark illustration of how different these concepts are.”

The post ends with reflecting on how meaningful assessment of research can be developed within individual disciplines and I think this is a useful approach for Learning Technology in particular, given the diversity of research and research active professionals as well as the broad range of media and technologies we utilise.

Another interesting post I have come across is this from the Scholarly Kitchen which examines the role of preprint repositories and their impact on journal citation rates https://scholarlykitchen.sspnet.org/2018/05/21/journals-lose-citations-preprint-servers-repositories/ . The article in itself is interesting, but in particular I found it helpful to look at how some of the practices in Open Access publishing have developed. What I am particularly interested in is the messiness that results in different versions of the same research being linked to and cited on different platforms and how this impacts on citation rates and access to research over time.

One of the questions in the post is why people keep linking to and citing pre-prints even when the published version of an article is available. There is no tidy way of making everything link up in one place and so whichever platform is used most becomes the most useful even though the life cycle of publication is designed with a different aim in mind. The post uses the example of bioRxiv to examine how this, but this can easily be applied to other, similar platforms formal and informal.

The post concludes with this:

A citation is much more than a directional link to the source of a document. It is the basis for a system of rewarding those who make significant contributions to public science. Redirecting citations to preprint servers not only harms journals, which lose public recognition for publishing important work, but to the authors themselves, who may find it difficult to aggregate public acknowledgements to their work.

I am looking forward to exploring these and related questions next week. You’ll be able to access all details of my presentation via the conference website http://programme.exordo.com/edtech2018/delegates/presentation/1/ . I’d also like to add a note of thanks to the wonderful Bryan Mathers for making his Visual Thinkery available under Creative Commons licences and thus enabling me to use (and credit) them in my presentation. Thank you.