Maren Deepwell

Category: Learning Technology

  • Which direction to take… researching alternative ways of measuring impact in Learning Technology

    Which direction to take… researching alternative ways of measuring impact in Learning Technology

    This is the second post about my current work on researching alternative ways of measuring impact in Learning Technology. Go back to the first post in which I have set out the context of my work and what I am particularly focused on.

    Alongside the practical work with the ALT Journal Strategic Working Group, I am pleased that my proposal of a short session ‘The quality of metrics matters: how we measure the impact of research in Learning Technology’ has been accepted for ILTA’s Annual Conference in Carlow, Ireland later this month. 

    In the meantime, I have been doing more reading and research into innovative ways of measuring impact and this time my work has come up against some very practical questions, not least because as a UK-based publisher we are in the process of ensuring the the journal’s operations comply with the incoming GDPR legislation. Open Source journal systems are not at the forefront of compliance and like other independent publishers we work as part of the community to move towards compliance.

    At first glance factors like GDPR may not seem to be closely related to how impact is measured, but my thinking links them closely as a lot of the opportunities around developing the journal are dependent on technical solutions that have data processing implications:

    A convincing alternative
    Discussing how important having an impact factor is quickly runs into the question of what the alternative looks like. As well as the technical challenges in implementing innovative tools or mechanism for measuring impact (to which the new GDPR legislation adds another level of complexity), the sustainability and longevity of both tool and data storage need to be examined. For example, introducing a tool like Altmetrics requires us to educate all stakeholders and ensure that the level of digital literacy required is not a barrier to making the tool useful. The user interface and experience needs to be robust and practical, building confidence in alternative or innovative ways of measuring impact. With new tools and platforms being created all the time there is a certain amount of churn and in order to really build a convincing alternative there needs to be a certain level of consistency.

    Scrutiny of new vs. established ways of measuring impact
    The kind of scrutiny with which we are examining alternative ways of measuring impact isn’t easily applied to the established method. There is a critical discourse, for example this recent blog post on the LSE impact blog, which argues:

    Many research evaluation systems continue to take a narrow view of excellence, judging the value of work based on the journal in which it is published. Recent research by Diego Chavarro, Ismael Ràfols and colleagues shows how such systems underestimate and prove detrimental to the production of research relevant to important social, economic, and environmental issues. These systems also reflect the biases of journal citation databases which focus heavily on English-language research from the USA and north and western Europe. Moreover, topics covered by these databases often relate to the interests of industrial stakeholders rather than those of local communities. More inclusive research assessments are needed to overcome the ongoing marginalisation of some peoples, languages, and disciplines and promote engagement rather than elitism.

    It’s really helpful to read this kind of perspective, but in my experience there is a strong sense that institutions and senior management place much importance on the established value of the impact factor. We have decided to carry out consultation with stakeholders, but in the absence of a convincing alternative (which in our case we simply haven’t had time to implement as yet) I am not sure what we would be asking our stakeholders to compare or comment on. There is such a range of options being implemented by Open Access publishers, that we can a learn a lot from their example and work towards putting in place improvements that will help establish what might be an alternative or a complimentary perspective to the traditional impact factor.

    Measuring beyond impact: peer review
    Through our Editorial Board, the working group has now also begun to look at platforms like Publons, which promises to ‘integrate into the reviewer workflow so academics can track and verify every review and editorial contribution on the fly and in complete compliance with journal review policies’ (read more). It’s clearly a widely-used platform and some colleagues seem to be enthusiastic users, so it’s made me consider what this kind of platform could add to the user experience alongside innovative tools to measure impact. As a journal that does not charge any APCs, the value proposition for authors is clear, but resources to improve the experience of reviewers are limited. More work is needed for us in this area to examine whether we can compliment our efforts to improve the ways in which the impact is measuring could be complimented by enhancing the experience of peer review.

     

    Read more (with thanks to everyone who’s sent me comments or links):

    Information for publishers from DOAJ: 
    DOAJ does not believe in the value of impact factors, does not condone their use on journal web sites, does not recognise partial impact factors, and advocates any official, alternative measure of use, such as article level metrics.

    There is only one official, universally recognised impact factor that is generated by Thomson Reuters; it is a proprietary measure run by a profit-making organisation. This runs against the ethics and principles of open access and DOAJ is impact-factor agnostic. DOAJ does not collect metadata on impact factors. Displaying impact factors on a home page is strongly discouraged and DOAJ perceives this as an attempt to lure authors in a dishonest way.

    Full information here.

  • Re-post #altc: My Chief Executive Officer’s Report, May 2018

    Re-post #altc: My Chief Executive Officer’s Report, May 2018

    This is my report to Members of ALT for May 2018, originally published on the #altc blog here.

    Dear Members

    As I am writing this we are just beginning a particularly busy period for the Association, so my report to you this time will be a whistle-stop tour of what’s happening across our community. I am pleased in particular to welcome new member organisations who have recently joined ALT: Staffordshire University, Ajenta, MyKnowledgeMap, TES, Northern Regional College, University of Dundee and Wrexham Glyndwr University.

    A global perspective on professionalisation in Learning Technology

    On 3 May 2018 I was honoured to join the BOLT project organisers and partners at the Hong Kong Polytechnic University at their blended learning Symposium @PolyU,  a celebration and culmination of the BOLT project. As this 4-year University Grants Committee-funded project draws to a close, the Symposium celebrated at its impact so far – evidenced by its shortlisting for the Reimagine Education 2017 awards – and also looked to the future and its sustainable legacy. A particular highlight for me was being invited to present two members of staff, Seth Neeley and Arinna Nga Ying Lee, with their CMALT Certificates.

    Congratulations to Seth, Arinna and the 20 other individuals who have achieved CMALT accreditation so far this year.  

    Launching a new Award for the Learning Technology Research Project of the Year

    One of our strategic priorities for this year is to enhance recognition for research in Learning Technology and the launch of this year Award helps us achieve this aim. The ALT Learning Technologist of the Year Awards celebrate and reward excellent research and practice and outstanding achievement in Learning Technology. Established in 2007, the Awards have established a benchmark for outstanding achievement in Learning Technology on a national scale and attract competitive entries from the UK and internationally. All entries are reviewed by an independent judging panel chaired by the President of ALT. We gratefully acknowledge the support from our sponsors Catalyst, open course technologists, for supporting the Awards this year. The Awards are now open for entries.

    We seek Member input for UNESCO Recommendation on Open Educational Resources (OER)

    ALT is collating a response for the following UNESCO consultation. Please use this shared doc to provide input https://go.alt.ac.uk/2FsAbBA . If you would like to provide input for the response, please use the heading structure provided. Alternatively you can also email your contribution to maren.deepwell@alt.ac.uk . The deadline for responses is 1 June 2018.

    General Data Protection Regulation (GDPR)

    As I’m sure many of you are aware new data protection regulations come into effect at the end of this month. We welcome these regulations as they will hopefully permit greater transparence in the use of personal data including in learning and teaching. In preparation of the new regulation we have updated ALT’s own Privacy Policy and carried out a number of actions in-line with guidance from the ICO.

    We have also been supporting Members with a series of webinars to raise awareness around GDPR in learning in teaching. As part of this we were delighted to host Martin Dougiamas, Moodle Founder and CEO, along with Gavin Henrick, Moodle Business Development Manager to highlight actions the Moodle community have taken, Stephan Geering, Blackboard Global Privacy Officer and Associate General Counsel, and Mark Glynn, Head of the Teaching Enhancement Unit at DCU. If you missed any of these sessions recording and resources have been added to the event pages accessible from the past events section of our website.

    ALT Annual Survey data & report

    I will conclude my report with a reflecting briefly on the findings from this year’s ALT Annual Survey, the report of which was published in March by my colleague Martin Hawksey. As with previous years the Annual Survey is designed to:

    • understand current and future practice;
    • show how Learning Technology is used across sectors; and
    • help map the ALT strategy to professional practice to better meet the needs of and represent our Members.

    With the survey in its fourth year we are able to record and report and number of changes. This year some of the biggest changes are in the enablers and drivers for use of learning technology. The insights gained go beyond the trends in technology and organisational change, but help us understand the needs of staff enabling students and building a more empowered relationship with Learning Technology. Themes that we can look forward to exploring more at ALT’s Annual Conference this September.

    Maren Deepwell

     Maren Deepwell, Chief Executive of the Association for Learning Technology (ALT), @marendeepwell

    If you enjoyed reading this article we invite you to join the Association for Learning Technology (ALT) as an individual member, and to encourage your own organisation to join ALT as an organisational or sponsoring member

     

  • Critical Perspectives on Blended Learning: Hong Kong Polytechnic University BOLT Symposium

    Critical Perspectives on Blended Learning: Hong Kong Polytechnic University BOLT Symposium

    This week I contributed to the BOLT Symposium at Hong Kong Polytechnic University as keynote speaker. The symposium brought together colleagues from universities in the region to disseminate outputs from the BOLT project, showcasing successful collaboration, shared provision of staff development and initiatives to ensure that the sustainability of what has been achieved. As part of my work for ALT I have supported the project in a small way for a number of years, mostly around the CMALT accreditation framework and it was very rewarding for me to be able to meet the project team in person. 

    In my talk I focused on critical perspectives on blended learning, using some common technology focused reports and frameworks as a starting point, including the recently published preview of the 2018 Horizon Report. Using insights from the findings from the ALT Annual Survey 2017 I went on to examine the enablers and drivers being the adoption of learning technology, and shared an update on the developments of new pathways to CMALT accreditation. In the last part of my talk I looked at the bigger picture, drawing on historical perspectives to assess the validity of the claims made for technological innovation including sharing the work of Audrey Watters’ Teaching Machines timeline. My talk concluded with a reflection on the importance of equity for the future of blended learning and empowering staff and students in their relationship to learning technology. We had time for a brief Q&A session and I was grateful for the engaging questions from participants.

    I also want to mention a particular highlight of the trip for me, which took place the day before the symposium, as I was able to present two members of staff with their newly awarded CMALT accreditation.

    As well as a warm welcome and great hospitality I was inspired by the many conversations with colleagues from the different institutions taking part in the event. It was interesting to hear and see how each institution contributed to the collaborative project as well as finding strategic alignment with organisational priorities.

    Whilst the context may be very different, there seemed to me to be a lot of commonality between what is being achieved here in the provision of staff development to help scale up use of blended learning and the kinds of projects I am familiar with in the UK and elsewhere. I returned with a suitcase full of ideas to take back and share with the community here, having gained a new perspective on how the work of ALT can be of value to Members all across the world and inspired by the shared questions we have about the future of learning technology.

  • Being grounded in itinerant professional practice

    Being grounded in itinerant professional practice

    Recently, I have started writing a series of blog posts with my colleague Martin Hawksey. It’s an interesting undertaking in which we take an open approach to leadership, to sharing our perspective on leading the organisation we work for through a period of change towards adopting a virtual mode of operating. And it’s got me thinking on parallel lines about my own professional practice and how it’s developed over the last 20 years, from being a practising artist making stuff, to being an anthropologist studying cemeteries, to being an academic and learning technologist and now to working in a leadership role.

    My work has been on the move constantly. I have changed countries, cities, institutions, offices, roles and colleagues. I like change, and having new challenges, but I also take things with me. Some things remain constant, part of my routine no matter where or how I work.

    That isn’t too say that my environment, technology or company doesn’t have a big impact. They do. But they don’t define my practice. What really matters, what makes me work well, what helps me achieve, that I take with me. A bit like the suitcases I used to create as a sculptor or draw in my sketchbooks, I think I have a carry-on of essentials that I don’t leave behind. They help ground me and my work when things change or I do. In my experience everyone has an equivalent of those types of things, but here are my top 5:

    Reflective writing
    Whether it’s in a journal, on loose paper, on my private blog or digital doc, at least once a week and often more frequently I sit down, reflect and write. It doesn’t matter where I am or how busy things get, reflective writing forms an essential part of my practice. It helps me gain perspective, empty my head and make time to enjoy what’s gone well or give myself a break for things that have gone awry. I write around 80,000 words a year, so cloud storage is a good thing. 

    Long term list
    No matter what shape my daily to list may take, I always keep a list of ideas, links and actions to consider in the long term. It’s a dump for anything important, but not urgent, as well as more creative ideas or plans. I review it periodically, maybe once a month or so, to mark things as completed, delete and re-order. Some items take years to complete… .

    Pictures
    A visual record of what I am working on, where I go or whom I meet is really important to me. It’s useful to be able to look back, revisit particular conferences or trips or lunch. That also includes images or artwork I (help) create – in particular if I have been working on something for a while.

    Making time to listen
    Harder to make the suitcase metaphor work with this kind of thing, but still an essential. It sounds very obvious that listening is important, but so few people actually do. Most of the time, it’s all about talking, transmitting, being heard. The busier things get the more difficult it becomes to find time to listen and make space for others to find their voice, share their thoughts. So from train journeys to coffee breaks, I make a point of putting my laptop or phone to one side and really focus on the people I am with.

    Complete something fun
    As often as I can, I dedicate some time to doing something fun. For example, I enjoy making a web page and publishing it. Or making an image for something. Or reading/writing something. It doesn’t really matter what it is, it’s about completing it and getting a sense of achievement from it. A quick win. A lot of what I work on is a) long term, b) invisible or c) collaborative and so in order to balance these with feeling that I have accomplished something I often do ‘something fun’ and see it completed. It reminds me of how good I am at making things happen and often helps me see bigger, more complex things through to completion with a greater sense of confidence.

  • #femedtech #OER18 #OER17… because equality matters for all of us

    #femedtech #OER18 #OER17… because equality matters for all of us

    If you have been following the reporting on the gender pay gap in the UK, then this has been a sobering week indeed. You can search for the reports from different employers here. I have had a look through many of the education providers and sector bodies that I work with and the scale of the ‘gaps’ highlighted in some of the reports is staggering. Not a surprise, given my day to day experience of the sector, but still – staggering.

    As a chief executive I have reflected much during this week on how we can change things across the system. There are so many aspects to the problem that there is definitely no shortage of things we should tackle and there is much to do in relation to the professionalisation of Learning Technology.

    But on a more personal note this has also reminded me of how important it is that we continue to work towards achieving greater equality – in all its forms. So with a large international conference on openness in education just around the corner I hope that there’ll be much to learn and discuss from different, global perspectives. I also want to help give a voice to this conversation together with colleagues, and make sure that we consider equality in the context of openness.

    Powerfully, Catherine Cronin spoke of criticality, equality and social justice at OER17 in London last year. In the closing plenary we were asked to respond to a call to action… #Iwill #OER17 and many participants in the room and on social media joined in, making their voices heard and sharing their aspirations, making a commitment to taking action. I think it’s time to renew our vows to take action #OER18.

  • Trying out new online courses: GDPR and online engagement

    I have been trying out new online courses, starting with a Futurelearn course ‘Understanding the General Data Protection Regulation‘ which many colleagues are also taking part in. I am a Data Controller and responsible for my organisation’s compliance with the GDPR framework when it comes into force, and this course forms part of our way to becoming compliant.

    I have been taking part in the course mostly by reading and absorbing the material rather than discussing/exchanging ideas with other participants – I have plenty of debate around these issues in other contexts.

    The course is clearly structured, easy to follow and has plenty of references and articles to follow up on specific issues further. I found the video component of the course, which is usually more engaging for me, less interesting in this case. I found I prefer to read the more detailed articles and lists.

    After one more week of GDPR, I am really looking forward to starting a completely different kind of course, the upcoming ‘Engagement in a Time of Polarization‘ run on edX led by Delia Deckard and Bonnie Stewart. The course is described as follows:

    How can we work together in a society where our communications channels have become so polarized? Can we engage in active, effective collaboration in a media ecosystem designed to make money from driving us apart?

    This two-week course convenes a conversation on participatory engagement models, and on building understanding and relationships even within the very real limits of contemporary social media. The course will enact the same participatory ideas it explores, and will feature input from leading voices in media literacies, disinformation, and polarization. Participants can engage on their own time and in real time, and if they wish, can build towards action in their local communities.

    The focus of this course closely relates to some of the work I do day to day, in particular around enabling engagement in online communities at scale, supporting effective open, online governance and reflecting on the skills required to participate.

    It’s also a topic I am interested in for my own practice, and I am looking forward to learning more about how debate gets polarized and how we interact online.