Video annotation Workshop at The Centre for Digital Humanities, Groningen

Report Video Annotation Workshop, Groningen, April 12, 2018.

By: Tom Slootweg

On April 12, 2018 the Centre for Digital Humanities of the University of Groningen and CLARIAH organized a workshop to explore the possibilities of video annotation for research in the Humanities. The workshop functioned as a follow-up to the session on video annotation held at the Digital Humanities Day 2017 (October 26-27), which also took place in Groningen. As co-organizer Susan Aasman, director of the Digital Humanities Centre, underlined: ‘this day aims to provide a platform for comparing our notes on and experiences with video annotation. First we need to establish where we are and then we should discuss where we want to go to’. During the first hour it became clear that the use of video annotation tools is not necessarily confined to one field of research alone. Nor does their use always invite one “gold standard”.

Video annotation workshop

Video annotation workshop

The start of the workshop featured four fifteen-minute presentations that showcased the highly interdisciplinary interest in video annotation as a viable research tool. Two of the four presentations were related to CLARIAH research pilot projects aligned with novel ways of doing media historical research. Rob Wegter, research assistant to the M&M pilot project , outlined the project’s use of the integrated video annotation functionality of the CLARIAH Media Suite; the digital research infrastructure provided by CLARIAH through which media scholars have access to various collections of audiovisual archives and other cultural heritage institutions. A particularly challenging aspect of the project’s deployment of video annotation, as Wegter explained, relates to its use for more traditional hermeneutic research.

Video annotation workshop participants

Video annotation workshop participants

In contrast, Christian Olesen, principal investigator of the MIMEHIST pilot project, devoted his presentation to explaining how video annotation can be fruitful in linking to each other items from materially heterogeneous collections. As Olesen argued, when working with a fragmented archive, such as the Desmet Collection held by the EYE Film Museum , video annotation can help in “defragmentation”, but also allows for a ‘more advanced form of mixed media analysis’. Moreover, as Olesen underlined, video annotation could potentially enable experiments with “distant reading” methodologies often not taken in consideration in traditional media historical research.

After the two CLARIAH research pilots, communication scholars Tom Koole and Gisela Redeker brought to a close the first hour of the workshop with their respective presentations. Prof. Tom Koole brought back into mind the call by the KNAW in which the question was posed: ‘what should be the tools of the future of your field in 2025’. Video annotation, in Koole’s view, is surely one of those tools, in particular when it concerns his own research into the way in which language is used in social interaction. His main argument was that we need to aim for international collaboration as well as an interdisciplinary outlook on the development of video annotation tools.

Based on the discussions that took place during the “ Collecting, Annotating & Analyzing Video Data ” workshop at the Lorentz Centre in the fall of 2017, and the “ Big Video Sprint 2017 ” conference in Aalborg, Denmark, he listed several important requirements necessary for video annotation tools to become useful for a ‘highly interdisciplinary group of researchers’:

  • video-transcript synchronisation;
  • frame-by-frame mode;
  • drama format of transcripts;
  • flexible, self-manageable annotation options;
  • search options; collection building options;
  • secure mode for sensitive data; various data presentation modes;
  • automated speech/speaker recognition;
  • sustainability and modularity of the tools.

Prof. Gisela Redeker, concluding the first part of the workshop, devoted her presentation to the practical use of video annotation tools for the description and evaluation of multimodal instructions deployed in educational first-aid videos. This project, which is part of the PAT Work Bench project (PI Ielka van der Sluis), uses the ELAN software package, which enables highly precise, multi-tier annotations of the video data under scrutiny.

“Code book” for annotating in ELAN

The second part of the workshop provided an opportunity for some hands-on exercises with video annotation tools. Whereas one half of the group worked with the CLARIAH Media Suite and its annotation tool, the other half explored more in-depth the range of possibilities afforded by ELAN. Rob Wegter and Liliana Melgar Estrada, organizing the CLARIAH-session, invited the participants to first create “user projects” in order to subsequently store videos by “bookmarking” them to one’s project. Afterwards the possibilities of annotating video were explored together with possibilities for exporting metadata for further analysis. In the meantime, Redeker, supported by Ielka van der Sluis and Charlotte Vijfvinkel, guided the other group of workshop participants through the various possibilities of ELAN for multimodal analysis.

Wrapping up, Aasman concluded that the workshop was ‘stimulating and offered quite some food for thought’. Concerning any future projects, it is important to keep in mind that any “gold standard” for our use of video annotation seems to depend on the research project in which it is deployed. Moreover, as the use of video annotation tools grows in importance across various, often interdisciplinary research fields, it becomes necessary to devote more attention to such issues as corpus management and the continuity and sustainability of the digital research environments and the storage of datasets. A follow up video annotation workshop is already planned for the next Groninger Digital Humanities Days in October 2018, when the export and analysis of video annotations will be the central issue. We keep you posted!