Virtual team, face to face team day: putting our blended approach to the test

This is a joint post with Martin Hawksey (cross-posted here) continuing the story of how we, as senior staff, lead our organisation to adopt virtual operations. You can read previous posts here.

June

Last time we discussed how we used GDPR as an opportunity to strengthen how we work as a virtual team. Since then we had our first face to face team day, an important milestone in creating a blended approach to running a virtual organisation.

Maren: The team day was followed the next day with the meeting of the Board of Trustees, in which for the first time all of our colleagues took part. I learnt a lot from both days: for example, I had to adjust my mindset from having a ‘team day out’ (which was previously the only time we would all travel somewhere together once a year to do some team building and have a meal together) to having a day working together. Whilst that may seem an obvious point to make, it’s an important distinction to communicate to everyone involved. The agenda we set out helped us prepare and be ready to focus on the task at hand. In order to make the most of team days we plan them in different locations. This time it included a site visit to the venue for our upcoming conference. Whilst combining activities like a site visit with a day working together face to face as a lot of advantages, having a changing location means that we need to sort out all the practicalities, like somewhere quiet to work or access to WiFi afresh each time. That’s a big change from having an office at which we convene and also means ensuring that we think about how to support the team as a group, taking into account individual needs. Some of the startup leaders I work with talk about how this degree of agility is difficult to make work in an equitable way and I found that the two days took more preparation than I had anticipated. As well as logistics, it is important that these days reflect our aims and values as an organisation. Inviting all staff to take part in the Board meeting is a good example of how we are trying to do this, but as well as time to work together, we also planned in time to eat and catch up informally and I felt those pockets of unstructured time were really important. This time, the Study, Museum of Manchester, became our work space for the day.

Martin: The opportunity to work in new places is a nice feature of our approach, particularly when they are as nice as the Study. Thinking about the practicalities some thoughts that came to mind are with a team of 5, soon growing to 6, we are perhaps at the limits of places where it is easy to just turn up and get a seat. If I’ve got time before/after events or meetings in Edinburgh I’ll often go to the Dome on Potterrow, or when in Glasgow the Saltire Centre. These are great spaces but get very busy at times and you’d struggle to get more than 6 people around the same table. There however seems to be an explosion of affordable hot desking and meeting spaces, in particular, as part of start-up hubs. As a membership organisation there is also perhaps opportunities to combine visiting some of our Organisational Members and have some space for a team meeting. Combining the two could be interesting as it would let other members of the team see what we do. I suppose the danger of such an approach is the site visit becomes a distraction from getting team business done. Something else that came out of our meeting which I hadn’t really considered before is while I was happy and able to get up and leave for Manchester at 4:30am this might not be possible for everyone (plus getting home after 11:30pm because of delayed flights ain’t fun). Raises interesting questions about equitability.

Maren: That’s true. As our team is distributed across the U.K. every location means longer travel times for someone. Achieving a balance between the flexibility of home working day to day and occasional travel whilst meeting our organisations’ needs can be a challenge. Organising the logistics is still a learning curve, but the other aspect I’d like to talk about is the actual work we did together. There were three parts to our day: first, the site visit which we all took part in and which helped us plan for the upcoming event. I think it was very useful to have everyone contribute ideas and ask questions on site and it saved us a lot of reporting back and another visit in the long run. Next, we spent some time together reviewing our project plan. We do review the plan regularly during our virtual team meetings, but it was insightful to do so in person as a team, seeing it projected on a big screen. For example, I noticed that we discussed more, asked more questions of each other. That continued when we spent time together working on different things after lunch. There was an opportunity for building rapport more informally that I hope will translate at least to some degree into our virtual collaboration. Two of our team had not met in person before, so that was an important function of the day, too.

Martin: I agree that it felt like there was more collaboration and communication when we were working together in the same room. I think this underlines that it’s important to recognise there are differences between working as a distributed team rather than in a shared physical space. Until we all have a holodeck in our homes I don’t see this changing but it also is a reminder that we need to think about distributed teams differently and not just faithfully recreate the physical space online. This was captured a post by Noello Daley on ‘What Co-located Teams Can Learn From Remote Teams’. In particular, Noello highlighted “the importance of shifting not only process, but mindset”. Noello goes on to say that you should “shift from a local, spoken culture to a global, written culture”. I think ALT had a strong global written culture before become a distributed organisation so perhaps the difference is less apparent to me. The process and mindset is an interesting area. There is inevitably disruption in becoming a distributed team and in some ways you want to minimise this to allow people to adapt to a new working environment, the tension as a manager you don’t always want to continue old processes either because they are less efficient or don’t work in the new model. One of the challenges is something else Noello highlighted was in distributed teams there needs to be “empathy for each others’ needs”. This is something we have touched upon already. I feel part of the solution is also touched upon by Noello and mirrors our own blended approach: “plan to get together about four times per year. Use that time to re-establish team goals and culture”. Given the Manchester trip was an opportunity to share our individual perspectives of goals achieved to our Trustees and the process for identifying those goals was collaborative I think we should be having more staff days around Board meetings.

Maren: I agree. Clearly our first attempt has highlighted how much potential these days have. But what about the day to day? Following on from the post by Noello you mention, I have been reflecting on how important it is to take some personal responsibility for what this recent article (found via @hopkinsdavid, thank you!) called ‘finding connection points’: the author suggests taking time to schedule regular lunches with co-workers, meet clients (Members, in our case) and getting involved in the community to ensure there is some face to face contact or at least time focused on building relationships. Some of this I would have previously thought about more in the context of CPD. As a small team we have long set up days to visit other organisations or Members to learn from and see how they do things, but in the context of operating as a virtual team, that’s taken on a different significance. Now it’s about making connecting with others an integral part of what we do as individuals, part of our professional practice. We can set an example and enable others, but to some extent it depends on how much an individual is willing to or interested in being part and contributing to that kind of working culture. Tech-focused solutions, like for example shared bookmarks, can help build a sense of shared space online. Yet it still depends on everyone contributing, everyone recognising the importance of working in a certain way and what benefits that has for us as a team and the organisation as a whole. In the last few hours of our team day we each prepared to talk to the Board about some key ways in which we have individually contributed to the success of the organisation in the past year and listening to that was probably the most powerful moment of the two days for me. We may not all be in the same place very often, but we are all part of the same vision and when we presented to each other I really felt that come across.

On a tangent to leading a virtual team… going on holiday

This blog is mostly about the work I do, and things I find interesting. But, as a Learning Technologist in a leadership role, I think a lot a lot about work/life balance even if I don’t share much about my life away from work on this platform. The current series of posts written jointly with Martin Hawksey in which we take an open approach to leading a virtual team has prompted me to reflect on how important it is to be aware of a how less formal, home-based work environment can affect one’s well being. As I have holidays coming up, I wanted to share my approach to taking a break. In an environment where many talk about their work ethic, on being indispensable or never having weekends or evenings off, it’s important to also talk about taking time away from work, taking a break.

So, here is what works for me when I want to get away:

Lists: a list of things to do before I go away, a list of things to handover/cover, a list of urgent things for when I get back, a list of long term things for when I get back, a list of things to check… ticking things off my lists and having somewhere to note all the things I won’t get done, is essential.

Ruthless inbox clearing: delete, delegate, deal with… always nice to see how a firm deadline focuses the mind and suddenly everything that has been languishing in my inbox for a while is gone. I also make an effort to send much fewer messages in the week before I go away, to get fewer responses back.

Make sure everything is covered: as a Line Manager and in a role where things have to be covered whilst I am away, I am fortunate to have colleagues who will deputise or cover for me. I put my arrangements in place well in advance and remind everyone that I’ll be away, to help them plan ahead, too. It’s a good opportunity for me to model what a handover looks like and how to plan ahead around urgent deadlines.

Keep for everything that I’ve forgotten: I am a big fan of Google Keep and use it for managing my to do lists. In the first few days whilst I am on holiday I’ll remember lots of things that I have forgotten and so I make a special list to which I can add everything that comes to mind. If I write it down, I won’t worry about it and it’ll be waiting for me when I get back.

Really actually go on holiday: if I intend to have proper time off, I don’t monitor email at all. I set my out of office and that is it.

Interesting reading: one of my main priorities when I don’t have to work is to catch up on lots of reading. That may be work-related research at times, and having that to look forward to is a big motivation for me.

I look forward to having a break from my usual routine, time to go for a run when I’m usually at my desk, time to spend with family and friends, time to stay up late… time enough to feel more curious, keen and inspired again when I head back to work.

Don’t think you are brilliant? Exploring the full colour spectrum of reflection.

paintone

Last year I wrote a post called Don’t think you are brilliant? Think again… and in it I shared my insights from being an assessor for CMALT, ALT’s peer-based accreditation scheme and how I observed that reflecting on practice can help you realise your own strengths and gain a sense of achievement.

Since then, I have assessed a lot more portfolios and also explored those shared by other Certified Members (thanks for sharing!) and I have come across a few more thoughts on this, which I’d like to share with you now – mainly because I think most of us could do with a bit more confidence, recognition and a sense of a job well done before we hurry on to the next deadline or deliverable.

It’s hard to feel that you are doing well when things are tough and often it takes a long time before you can look back and see how much progress you made at the time. When people in my network, mostly open practitioners, write brave posts sharing stories about their ups and downs I am always reminded that for everyone things can be difficult even if as an observer I can only see the what it looks like from the outside: I see the video, or hear the talk or read the paper, I pick up all the indicators of success, achievement, new thinking and so forth, but so often those only tell a small part of the story. But if you are chairing a meeting, on a stage or behind a lectern, people generally assume that you have it figured out. I salute those who let us peek behind the scenes, who share the more human side of their work. I hear people often advocate that we embrace sharing failures more, and I, too, support this notion strongly – but it is harder to do than it looks, I think. If failures are shared, it is mostly on the practical aspect of what we work on, on technology and tools, based on survey results or case studies. Very useful, but not personal like reflection can be.

Some common factors in successful reflection:

Making time: the most effective reflection is often born out of a habit to set aside time for it. Regularly. So that both good and bad weeks are considered, and over time a balanced perspective created. Some people write or have conversations, others think inside their own heads. Most come to record their ideas, thoughts or feelings in some ways. Some of the most insightful accounts I have read as an assessor demonstrate a clear sense of progression, i.e. someone having taking stock at different points in time and building a narrative around that.

Sitting with it: reflection doesn’t always lead to expected or pleasant places. Sometimes a train of thought ends in an uncomfortable place. Or in a realisation that things aren’t going well. Failure looms large. One of the techniques I most admire is the ability to be present with that kind of realisation, to develop a more mature relationship to thinking about topics that are difficult to reflect on. It takes a lot of confidence to do that.

Confidence in yourself: one of the ways in which reflection can go wrong is to rely only on external measures of success or failure. For example, you might reflect on a project as a failure if it didn’t meet its objectives fully even if you learnt a lot from it or it was a useful stepping stone to something new. Developing a sense of confidence and trusting one’s own judgement is a common component of effective reflection.

So, make time to reflect and discover the brilliant or shiny part of your practice alongside all the grey bits, or the darker ones, too. Space to think comes in all colours.

Opportunity knocks: Using GDPR to strengthen virtual teams

This is a joint post with Martin Hawksey (cross-posted here).  If you have missed our earlier posts we encourage you to revisit the beginning of the story of how we, as senior staff, lead our organisation to adopt virtual operations.

May

This month we discuss our approach to GDPR, evolving virtual working practices and the importance of explaining the reasons for new procedures as part of implementing them.

Maren: We at the end of a super busy month and part of what’s been keeping us busy is GDPR… (thanks for writing handy blog posts for us to reference here). We’ve worked hard on the contractual, technical & legal aspects, but it’s also been an opportunity to review our relatively new virtual working practices. One issue I have been thinking about is finding the right balance between providing guidance and support whilst ensuring individuals also take appropriate responsibility. For example, we have policies about how to secure laptops or delete temporary files and we regularly review these as a team and share updates on how we are implementing them. Yet even though you can monitor and review processes regularly there is a large element of trust in our virtual working culture. To some extent we have to rely on everyone taking responsibility and making it part of their day to day working habits to follow new procedures. Explaining the reasons why we mandate certain things should help ensure that everyone understands their importance. In the GDPR training we did as a team, talking about how the new legislation relates to our values as an organisation (e.g. how that is reflected in ALT’s Privacy Policy) and why it affects us as staff on an individual basis was a really important moment for me. What’s your view on this?

Martin: GDPR has been a great opportunity to think about how as a team we store and process data. As a data controller one of the things we have implemented is documenting our data processing activities which includes how and where data is stored. Another critical aspect is how data is transferred. For our team this is greatly simplified by predominately being a Chromebook based organisation with centrally managed devices. This means we can mitigate a number of risks through device security policies and the build-in security features of Chrome OS. Another key aspect is we have a ‘home working’ rather than ‘remote working’ policy. This removes risks associated with regularly using open wifi networks in places like coffee shops, but does however leave open two questions: how do we ensure the security of home networks; and given a number of our team also travel maintaining security on the road. The process of preparing for GDPR has highlighted that there is more we can do to secure data transfer, the solution being investigating VPN options. Besides the technical solutions it’s also been useful to reflect on how the team is responding to personal responsibilities mentioned. In the case of GDPR it’s been great to see our team respond to the training we’ve provided and being proactive in both highlighting areas where our procedures can be improved and also suggesting or making the changes required themselves. Not being co-located removes some of the opportunities to get an idea of how someone is doing, for example, body language is largely filtered out in Google Hangouts. It was only when I reflected on this that I realised I’ve started relying on other indicators.  Has our work around GDPR highlighted anything like this for you?

Maren: You make an interesting point about tangible and less tangible indicators and how they can help inform our approach to supporting and leading the team. As you say, GDPR has created a lot of crossover between policies that apply to our organisation as a whole, publicly, like the privacy policy, as well as the workflows that support membership services, and reaching over to personal working practices at home and whilst travelling. Tangible examples of how all the new procedures and policies are being implemented, like seeing new forms, or workflows or questions being discussed, is important. Together with the reporting and monitoring processes we use, these kinds of indicators enable me to manage the operational side of things. The less tangible things you refer to are harder to pinpoint, but I am also finding them more important since we have become a virtual team. They could be things like a casual comment or an informal conversation or something I spot when screen-sharing or working on a shared document. The more time I spend collaborating, the more I get a sense of how things are going. We have mentioned before how we have a ‘Show & Tell’ element at each of our weekly team meetings and recently we had several weeks of sharing what we use to manage our to do lists and plan our work. For the next month or so we will include a GDPR element in each team meeting, with everyone bringing examples of how they are implementing the new policies. All of these opportunities to collaborate, hearing colleagues think out loud, are valuable for helping me understand how others think or see things, and that enables me to better explain/support new processes.

Martin: Another aspect of our GDPR implementation I’ve been reflecting on is the degree of visibility of our individual activity to each other. In the case of GDPR I put a lot of effort into researching what we were required to do as an organisation and understanding various aspects of the new regulations from a legal and practical perspective. Parts of this process left very few tangible outputs and in some cases some of the outputs were not suitable for circulation in the team. It was a reminder that it’s not always possible to share everything we do and a level of trust is required. It was also a reminder of why our weekly team meetings are so important and arguably more important than if we were working in a face-to-face setting. You mentioned that our ‘Show & Tell’ has recently focused on sharing how we each plan our work. It was interesting to see the diversity of approaches and the varying levels of detail that we each use. As my role is very diverse rather than having a single method I adapt my approach. For example, in the case of GDPR I’m using a mixture of our GDPR action plan in Google Documents and Sheets, Google Keep lists and managing my inbox with labels, stars/flags and snooze. For other projects like the Annual Conference we have a shared project plan we can all report our progress against. In the case of the Annual Conference this has changed little from when most of the team was office based. I think this still works well but wonder if we were creating this from scratch as a virtual team would you do something different, in particular, to increase the visibility of what we are all doing at a particular time?  

Maren: I’d like to do that – spend time thinking about what starting from scratch would look like. I imagine that (1) our values, (2) the importance of working together with volunteers, our Members, and (3) our overall policies for working would remain constant. But… there are other factors: the size of the team means necessarily that many tasks are more independent and only some a consistent team effort. With 5-6 staff you can’t easily create sub-teams for example that would work together ‘more visibly’. I’ve also considered tools like Trello or Slack, but I’m not sure how well they’d work for everyone, and I feel allowing everyone a choice in which methods to use for organising work, e.g. what we shared in our to do list show & tell sessions, can really contribute to productivity. We have our overall operational plan which all other plans/lists are related to and in my mind that provides the consistency required – although maybe we could make use of it more frequently. Overall the high level of our output and achievement is a good indicator that our current practice is effective, and that is reassuring. What I mean to say is that we have an opportunity rather than having to re-imagine what this could look like. Hearing you reflect on your perspective and comparing it to my own has opened up the question what this looks like for each of us. With a team away day coming up in a couple of weeks we could take the opportunity to dedicate some time to reflect on this as a group.

Beyond advocacy for change – developing critical & open approaches in Learning Technology #LTHEchat 116

I’m excited that we have been invited to contribute a topic to the #LTHEchat, the weekly Learning and Teaching in HE chat created by the community for the community – Wednesday 8-9pm. It’s the only tweetchat I regularly participate in, although mostly in “listening mode” and I find it a very useful forum.

Below I have re-posted the description of the topic coming up this week:

#LTHEchat 116: Beyond advocacy for change – developing critical & open approaches in Learning Technology with Maren Deepwell @marendeepwell and Martin Hawksey @mhawksey

The next #LTHEchat on Wednesday 6th June 8-9pm (GMT) will be hosted by Maren Deepwell @marendeepwell and Martin Hawksey @mhawksey on developing critical approaches in Learning Technology.

With the 25th Annual Conference of ALT, the Association for Learning Technology, just around the corner, we have been looking forward as well as back over how things have changed (revisit some of the developments with Prof Martin Weller, President of ALT, in his ongoing blog series “25 years of EdTech”).

We define Learning Technology as the broad range of communication, information and related technologies that can be used to support learning, teaching and assessment. ALT’s membership is made up of people who are actively involved in understanding, managing, researching, supporting or enabling learning with the use of Learning Technology. Using technology for learning, teaching and assessment hasn’t been a ‘new’ thing for a long time. But one thing that remains constant is the pace with which innovation moves forward, learner expectations develop and our constant need to evolve our pedagogical approaches. This creates demands/pressures and staff development needs for academics to develop competencies with digital pedagogies/approaches.

Regardless of where we are, or indeed where our institution is, in spreading or scaling up use of technology, we now have research, case studies and practice to move beyond advocacy, beyond enthusiasm for shiny gadgets or dashboards to developing a more critical, nuanced relationship to Learning Technology and to share our work in order to build a stronger, more diverse and robust discourse.

hTIIXeJq_400x400Maren Deepwell @marendeepwell is chief executive of the Association for Learning Technology (ALT) and leads its work on professional recognition and development. Martin Hawksey @mhawksey leads on innovation, community engagement and technology for ALT.

ALT represents individual and organisational Members from all sectors and parts of the UK. Our Membership includes practitioners, researchers and policy makers with an interest in Learning Technology. Our community grows more diverse as Learning Technology has become recognised as a fundamental part of learning, teaching and assessment. ALT aims to increase the impact of Learning Technology for the wider community, strengthen recognition and representation for the Membership at a national level and lead professionalisation for individual Learning Technology professionals in a broad range of roles.

xu6aptqy6a8rb2h2w5by_400x400As the senior staff team of ALT Maren & Martin work with Trustee and Members on a diverse range of projects including ALT’s conferencesannual surveynational policy development and professional development. Sharing their approach to open leadership is a monthly blog series on running a virtual organisation and both actively disseminate their independent professional via their personal sites https://mashe.hawksey.info/ and http://marendeepwell.com/

Messy metrics and beyond: strategy & practice in measuring impact of research in Learning Technology

EdTechIE18 image by M Hawksey for ALT

The titles of the posts in this series are getting longer and longer so it will soon be time to wrap it up. Here is where I am up to with this project:

I started with reviewing recently developments since the major report, the Metric Tide, came out in 2014. Then I reviewed progress on the various alternatives or complimentary services to traditional journal impact factors based on citations alone and this week at the ILTA Conference in Ireland I was able to compare notes with colleagues from a number of professional bodies, who also independently publish their journals, and get very useful feedback from authors and readers, too. I am really grateful to everyone who has been sending me links to further reading, including this recent post on how to the altmetrics for reward and pay negotiations.

Reflecting on this work over the past 2 months, I’ve come up with some practical ways forward:

  1. As an independent Open Access publisher, we’ll continue to enhance our publishing process through working with organisations like COPE to ensure that we have robust and up to date policies and practices for authors, reviewers and readers;
  2. We actually have a lot of data about the impact of going fully Open Access since 2012, but we could do a lot more to make this accessible and useful to our audience and demonstrate the impact of the journal;
  3. There are a number of technical developments, like getting altmetrics back up and running after our change of platforms last year, that are a priority in order to build up a more consistent picture of impact and one advantage of taking ownership of the journal ourselves is that we will no longer be dependent on changing publishers to provide or keep this data. GDPR is a factor here, too;
  4. One question to get some more advice on is how we can leverage our Open Access repository to best advantage for the journal and other similarly how we can optimise other online platforms we have;
  5. Last, I have come across many reasons for why despite its perceived dominance, the traditional impact factor, is not the best fit for research in Learning Technology. Implemented effectively and consistently, I think alternatives are going to be more influential in our messy, changing landscape of a discipline for most professionals.

In 2012 when our journal was one of the first of its kind to adopt Gold Open Access in the UK, there were few peers who really thought that was a good idea. Many watched and learnt from the reports and guides we produced, but few felt able or willing to make the jump, to adopt a different (business) model. This is a similar moment, but this time there are many pioneers we can learn from in other disciplines and also from our own expertise in Open Education and online content provision. I can see strong strategic reasons for making the best use of technology, but also many practical ones. And alongside the developing plans for the journal, there are many ways in which research can be recognised beyond citations, beyond metrics. At the ILTA I mentioned one of the initiatives that I am involved in for ALT to do just that: the inaugural Learning Technology Research Project of the Year Awards (which, incidentally, are now open for entries until 18 June 2018).

Featured image: EdTechIE18 image by M Hawksey for ALT

The quality of metrics matters: looking ahead to #EdTechIE18

EdTechIE18 presentation slide

I am really looking forward to taking part in ILTA’s upcoming conference, TEL Quality Matters – People, Policies and Practices, 31 May – 1 June at IT Carlow. You can see the full programme here http://programme.exordo.com/edtech2018/ .

Working with ALT’s new strategic working group for the development of the Open Access journal Research in Learning Technology, I have been working on understanding more about alternatives to the established Impact Factor for independent Open Access journals generally and more specifically for researchers in Learning Technology (revisit my first and second post on this subject) and next week’s conference is a valuable opportunity for me to meet some of the colleagues who are part of the group and find out more about their experiences.

Meanwhile I have come across two new interesting articles (thanks to Neil Morris for making me aware of the blog post).

The first is another interesting post from the LSE Impact BlogThe academic papers researchers regard as significant are not those that are highly cited . The authors describe the current perspective as follows:

Citations, JIF, and h-index have served as the triumvirate of impact evaluation for many years, particularly in STEM fields, where journal articles are frequently published. Many studies have pointed out various flaws with reliance on these metrics, and over time, a plethora of complementary citation-based metrics have been created to try and address various proficiencies. At the same time, we see altmetrics emerging as a potential alternative or complement to citations, where we can collect different data about the ways in which research is viewed, saved, and shared online.

The authors share useful insights from their work surveying chemistry researchers to “gauge their perceptions of significance, importance, and highly cited materials. The results, while not truly startling, were nevertheless a stark illustration of how different these concepts are.”

The post ends with reflecting on how meaningful assessment of research can be developed within individual disciplines and I think this is a useful approach for Learning Technology in particular, given the diversity of research and research active professionals as well as the broad range of media and technologies we utilise.

Another interesting post I have come across is this from the Scholarly Kitchen which examines the role of preprint repositories and their impact on journal citation rates https://scholarlykitchen.sspnet.org/2018/05/21/journals-lose-citations-preprint-servers-repositories/ . The article in itself is interesting, but in particular I found it helpful to look at how some of the practices in Open Access publishing have developed. What I am particularly interested in is the messiness that results in different versions of the same research being linked to and cited on different platforms and how this impacts on citation rates and access to research over time.

One of the questions in the post is why people keep linking to and citing pre-prints even when the published version of an article is available. There is no tidy way of making everything link up in one place and so whichever platform is used most becomes the most useful even though the life cycle of publication is designed with a different aim in mind. The post uses the example of bioRxiv to examine how this, but this can easily be applied to other, similar platforms formal and informal.

The post concludes with this:

A citation is much more than a directional link to the source of a document. It is the basis for a system of rewarding those who make significant contributions to public science. Redirecting citations to preprint servers not only harms journals, which lose public recognition for publishing important work, but to the authors themselves, who may find it difficult to aggregate public acknowledgements to their work.

I am looking forward to exploring these and related questions next week. You’ll be able to access all details of my presentation via the conference website http://programme.exordo.com/edtech2018/delegates/presentation/1/ . I’d also like to add a note of thanks to the wonderful Bryan Mathers for making his Visual Thinkery available under Creative Commons licences and thus enabling me to use (and credit) them in my presentation. Thank you.

Weekend post: leadership lesson from my favourite starship captain

As you may have gathered, I like Star Trek and Captain Picard is my favourite captain in that particular universe. I don’t agree with every aspect of his approach to leadership, but I like to think that we share a lot of values and every now and then I come across something he says or does that really resonates with me.

I was watching an old episode of The Next Generation recently (fellow nerds: this is the one in which android Data loses his confidence after someone beats him in a futuristic strategy game), in which Picard says to Data that it is possible not to commit any mistakes and still loose. I like the episode in general, but that particular bit of dialogue stuck in my head, because it’s related to things I have been thinking about recently.  Here’s why it resonates with me:

Much of the time following all the right steps and doing all the right things does result in the desired outcome. At least when things are reasonably within your sphere of influence. So, when something goes wrong or something bad happens in response to what we do, we seek to establish what we did wrong, where we made a mistake – same as Data does in the Star Trek episode. He thinks he must have made a mistake because he lost the game. It’s logical, and it’s comforting to have a sense of control, to believe that doing the correct thing will result in the correct outcome.

But, in many instances, that’s not the case. That’s not the case in the well-ordered universe of the Starship Enterprise and certainly not the case in the chaotic reality we call home.

The advice from Captain Picard cheered me. I thought it was a useful lesson to revisit because it reminded me that not everything that goes wrong is my responsibility or within my power to change. It made me reflect on the fact that even when things go wrong, often you can still get to your overall aim in other ways. And it made me think about how valuable it is to make mistakes, to learn how to cope with making them, learn from them and move on with your confidence still intact. 

In a high-performing environment, like Starfleet in this instance, it’s doubly important to be considerate about that part of your working culture. To be able to share when things go wrong, to make the best of them and to move on. And that a little kindness, a moment of listening or recognition can go a long way to helping oneself and others weather difficult moments like that. 

Back in the Star Trek story, a cheered Data returns to duty with the Captain’s advice fresh on his mind, but he also returns to face his opponent at the game table. This time, Data plays not to win, but for a stalemate, and he succeeds in not losing. He changes his winning conditions, what he wants to achieve to succeed. It’s a great ending to the episode and a good example to follow.

My week as guest curator of @femedtech

#femedtech

This week I’ve been volunteering as a guest curator of @femedtech. I actively work to promote equality in everything I do, so when the opportunity came up to support this growing network as a Twitter curator, I took the opportunity to help out gladly – but I wasn’t really sure what it would be like, taking over the voice of this kind of account. So one week in and with one more week to go, here is my personal reflection on how it’s going.

Fortunately I had guidance, encouragement and advice from Frances Bell and Helen Beetham from the outset and they shared their experience and advice generously. I liked their suggestion to add my own identity to the account, which I did, and then I started tweeting and re-tweeting and following and.. a few days in everything started to feel a bit easier.

I also set up a TAGS Twitter archive and TAGSExplorer for the #femedtech hashtag, to help chart and visualise the conversation as it evolves:

As you can see, it’s still a fairly small conversation, but it is starting to grow and I found it really interesting to be able to see what is going on.

Another approach I tried was to find and follow accounts with similar values or related ideas, such as @uncommon_women and the @feministintrnet for example and to welcome new followers or volunteers as they arrived. Hello to you all!

With another week ahead, I hope I’ll get a chance to generate a bit more of a conversation and prompt more contributions (tagged as #femedtech) and have the chance to contribute to the development of information for future guest curators with Helen and Frances. My long term hope is that we can help draw together some of the existing initiatives, amplify the voices of those actively working to promote feminism and equality in Learning Technology and connect more people with each other. There are certainly enough challenges out there to be tackled, so let’s hope that we can help connect the dots to make a stronger movement.

Note: this post was updated with a better TAGSExplorer view and links (thanks to @mhawksey)

Which direction to take… researching alternative ways of measuring impact in Learning Technology

This is the second post about my current work on researching alternative ways of measuring impact in Learning Technology. Go back to the first post in which I have set out the context of my work and what I am particularly focused on.

Alongside the practical work with the ALT Journal Strategic Working Group, I am pleased that my proposal of a short session The quality of metrics matters: how we measure the impact of research in Learning Technology’ has been accepted for ILTA’s Annual Conference in Carlow, Ireland later this month. 

In the meantime, I have been doing more reading and research into innovative ways of measuring impact and this time my work has come up against some very practical questions, not least because as a UK-based publisher we are in the process of ensuring the the journal’s operations comply with the incoming GDPR legislation. Open Source journal systems are not at the forefront of compliance and like other independent publishers we work as part of the community to move towards compliance.

At first glance factors like GDPR may not seem to be closely related to how impact is measured, but my thinking links them closely as a lot of the opportunities around developing the journal are dependent on technical solutions that have data processing implications:

A convincing alternative
Discussing how important having an impact factor is quickly runs into the question of what the alternative looks like. As well as the technical challenges in implementing innovative tools or mechanism for measuring impact (to which the new GDPR legislation adds another level of complexity), the sustainability and longevity of both tool and data storage need to be examined. For example, introducing a tool like Altmetrics requires us to educate all stakeholders and ensure that the level of digital literacy required is not a barrier to making the tool useful. The user interface and experience needs to be robust and practical, building confidence in alternative or innovative ways of measuring impact. With new tools and platforms being created all the time there is a certain amount of churn and in order to really build a convincing alternative there needs to be a certain level of consistency.

Scrutiny of new vs. established ways of measuring impact
The kind of scrutiny with which we are examining alternative ways of measuring impact isn’t easily applied to the established method. There is a critical discourse, for example this recent blog post on the LSE impact blog, which argues:

Many research evaluation systems continue to take a narrow view of excellence, judging the value of work based on the journal in which it is published. Recent research by Diego ChavarroIsmael Ràfols and colleagues shows how such systems underestimate and prove detrimental to the production of research relevant to important social, economic, and environmental issues. These systems also reflect the biases of journal citation databases which focus heavily on English-language research from the USA and north and western Europe. Moreover, topics covered by these databases often relate to the interests of industrial stakeholders rather than those of local communities. More inclusive research assessments are needed to overcome the ongoing marginalisation of some peoples, languages, and disciplines and promote engagement rather than elitism.

It’s really helpful to read this kind of perspective, but in my experience there is a strong sense that institutions and senior management place much importance on the established value of the impact factor. We have decided to carry out consultation with stakeholders, but in the absence of a convincing alternative (which in our case we simply haven’t had time to implement as yet) I am not sure what we would be asking our stakeholders to compare or comment on. There is such a range of options being implemented by Open Access publishers, that we can a learn a lot from their example and work towards putting in place improvements that will help establish what might be an alternative or a complimentary perspective to the traditional impact factor.

Measuring beyond impact: peer review
Through our Editorial Board, the working group has now also begun to look at platforms like Publons, which promises to ‘integrate into the reviewer workflow so academics can track and verify every review and editorial contribution on the fly and in complete compliance with journal review policies’ (read more). It’s clearly a widely-used platform and some colleagues seem to be enthusiastic users, so it’s made me consider what this kind of platform could add to the user experience alongside innovative tools to measure impact. As a journal that does not charge any APCs, the value proposition for authors is clear, but resources to improve the experience of reviewers are limited. More work is needed for us in this area to examine whether we can compliment our efforts to improve the ways in which the impact is measuring could be complimented by enhancing the experience of peer review.

 

Read more (with thanks to everyone who’s sent me comments or links):

Information for publishers from DOAJ: 
DOAJ does not believe in the value of impact factors, does not condone their use on journal web sites, does not recognise partial impact factors, and advocates any official, alternative measure of use, such as article level metrics.

There is only one official, universally recognised impact factor that is generated by Thomson Reuters; it is a proprietary measure run by a profit-making organisation. This runs against the ethics and principles of open access and DOAJ is impact-factor agnostic. DOAJ does not collect metadata on impact factors. Displaying impact factors on a home page is strongly discouraged and DOAJ perceives this as an attempt to lure authors in a dishonest way.

Full information here.