Introducing MIMEHIST: Annotating EYE’s Jean Desmet Collection

This is a cross-posting of a post that I just wrote for my research blog Film History in the Making . I repost it here as the first in a series of blog posts I will write on the research project MIMEHIST: Annotating EYE’s Jean Desmet Collection, of which some will appear here first and others on my research blog. In this post I discuss some of the theoretical underpinnings and assumptions of the MIMEHIST project. In the coming blog posts I plan to also discuss practical and technical issues of the development – in particular the OCR’ing of Jean Desmet’s business documents – as well as the historical case studies we will carry out, in greater detail.

There is a very brief presentation of MIMEHIST on the website of CLARIAH (Common Lab Research Infrastructure for the Arts and Humanities) . However, the description is very general and not very detailed which was also a reason for me to write a blog post. In this post I will describe how our project builds on earlier multimedia annotation projects in media studies, discuss some of its theoretical underpinnings and how these may nurture new ways of using annotation in film historical research. The focus in this post is mostly on theory, methodology and functionalities. The historical case studies of the project will be described in greater detail in another post.

CLARIAH and MIMEHIST: Annotating EYE’s Jean Desmet Collection

Towards the end of February I received a grant for a video annotation project which will last a bit more than a year – from April 2017 to June 2018. The project is called MIMEHIST: Annotating EYE’s Jean Desmet Collection and is a pilot project within the larger research program CLARIAH (Common Lab Research Infrastructure for the Arts and Humanities) which aims at developing a national research infrastructure for humanistic disciplines and the arts in the Netherlands. Within this research program there was a call for pilot projects last year, which invited applications focusing on digitized collections and different historical research questions and case studies. The cases and collections will be embedded in the Media Suite developed by CLARIAH in order to advance digital research methods in media studies. It will be available for scholars in the Netherlands with a Dutch university login.

With help from Liliana Melgar, postdoctoral researcher at the University of Amsterdam, I wrote an application in the fall of 2016 focusing on developing an annotation environment for the Jean Desmet Collection of EYE Filmmuseum in Amsterdam. The project officially started in April and I am working on it together with Liliana Melgar who is in charge of defining the user requirements with me, as well as Willem Melder from the Netherlands Institute for Sound and Vision who works on data interoperability and Roeland Ordelman and Jaap Blom who are taking care of the software development and engineering within the project. After the project’s beginning we also had the great luck of being joined by Ivan Kisjes, who works as an archaeologist and programmer at the University of Amsterdam’s CREATE project , and who will be doing OCR work and text mining in our project.

Jean Conrad Ferdinand Theódore Desmet (1875-1956)

The Jean Desmet Collection, preserved at Amsterdam’s EYE Filmmuseum, contains the archives of film distributor and cinema owner Jean Desmet (1875-1956) who was active mostly in the early period of silent cinema and its transitional years. The collection consists of approximately 950 films produced between 1907 and 1916, a business archive containing more than 120.000 documents (these will be OCR’ed in our project), some 1050 posters and around 1500 photos. Parts of the collection were acquired by the Filmmuseum shortly after Desmet’s death in 1957 and then gradually expanded throughout the years with additional acquisitions. The Desmet Collection is unique because of its large amount of rare films from the transitional years of silent cinema, and because of the richness of its business archive holds extensive documentation of early film exhibition and distribution practices in the 1910s. These features contribute to its immense historical value which was one of the main reasons why it was inscribed on UNESCO’s Memory of the World Register in 2011.

Most of the material in the Collection has been digitized in different projects throughout the years. In the past few months I have worked on a report to outline exactly how the digitization work for each of the Collection’s subcollections was carried out which will provide a fundament for embedding it in the CLARIAH Media Suite. For more information on the Desmet Collection, EYE Filmmuseum has made a dossier on its website . And, for a more elaborate history of the Collection, film historian Ivo Blom has written a beautiful monograph on Desmet’s business activities and life which is available as an open access book from the Amsterdam University Press in the OAPEN Library .

Scholarly background: Audiovisualcy, multimedia editions and film segmentation as interpretation

Where do the methods and assumptions of MIMEHIST come from and why are they important for media historians? This is what I will try to answer in this section.

Readers of this blog will know that I have a great deal of enthusiasm for multimedia editions of archival film which facilitate analytical interaction with film – or video versions of films. These formats work with audiovisual media in media res by insist on a certain kind of audiovisualcy – as film scholar Catherine Grant has aptly called it – as a basis for scholarly interpretative processes – be they critical or historiographical. A fundamental key tenet in this kind of scholarship is that scholars, instead of only writing about film, cite, segment and re-edit films in different ways and to different ends as a foundation for their reflection and interpretation. There are several projects and theoretical writings which MIMEHIST finds inspiration in, in this regard. I would like to highlight some of them here.

In particular, I have been inspired by the Digital Formalism research project carried out in Vienna. This was a research project on Dziga Vertov’s work and theory which ran from 2007-2010 involving media scholars at the University of Vienna, archivists from the Austrian Filmmuseum and computer scientists from the Vienna University of Technology. Through extensive manual segmentation and data visualization the project explored Vertov’s montage style in relation to his theories on film montage and rhythm. Moreover, I have previously written enthusiastically about the Hyperkino DVD series which aimed at developing a historical-critical presentation format for annotated versions of Soviet classics or rarities.

Before these more recent projects, there have been several advanced efforts to conceive scholarly multimedia projects for film historiography in the past decades. In the 1990s, for instance, there was a rich experimentation with what was then referred to as hypermedia in CD-Rom formats in film studies, in the US in particular. Several groundbreaking projects were developed such as Lauren Rabinovitz’ The Rebecca Project (1995), the multimedia textbook The Virtual Screening Room developed at MIT by Henry Jenkins, Ben Singer, Ellen Draper and Janet Murray between 1992-1999, as well as Yuri Tsivian’s CD-Rom on pre-Soviet silent cinema Immaterial Bodies: Cultural Anatomy of Early Russian Cinema (2000). The latter was released by the University of Southern California’s Labyrinth project in its groundbreaking series of Cine-Discs edited by Marsha Kinder.

An original order form for Yuri Tsivian’s Cine-Disc Immaterial Bodies: A Cultural Anatomy of Early Russian Film (2000) which Marsha Kinder has uploaded to her great website .

Conceptually however, one may trace the urge to work directly on films as a historicizing working method even further back, for instance to the 1920s when Parisian cinephiles would be making compilation films of moments of particular cinematic beauty – or photogénie – to mark the end of a year spent at the cinema with a critical hit list. Or for instance – as film scholar Michael Witt has discussed with great insight – to Jean Mitry’s Film sur le montage (France, 1965), as well as to a great deal of compilation and found footage films.

There are also several interesting texts written in the 1970s, which tend to be considered hermeneutical antecedents in today’s digital humanities and discussions on audiovisualcy. In particular, Raymond Bellour’s classic article Le texte introuvable (1975) – in English The Unattainable Text – is often cited as almost prophetic in its heartfelt plea for a format or situation which allows film scholars to cite audiovisual fragments and to manipulate playback speed and direction as a way to interpret films through segmentation and scrutiny. Elsewhere in his writings on film analysis – to be precise in the article “To Segment/To Analyze (on Gigi )” – Bellour has also beautifully highlighted film segmentation as a process of interpretation in which one constantly finds new meanings and layers:

…segmentation is a mise-en-abîme, a ‘plumbing of depths’, a process that has no end theoretically – which does not mean that it has no meaning, in fact, that is its whole meaning.

In recent years, Bellour’s ideas to a large extent provided the fundament for the brilliant scholarly video annotation software Lignes de Temps, developed by the Pompidou Centre’s Institut de recherche et d’innovation. Unfortunately, this software is no longer being updated.

Screen cap from Lignes de Temps (here I was busy analyzing the fantastic concert scene with the Yardbirds in Antonioni’s Blow-Up (UK/I/US, 1966))

Another text which is less frequently cited today, but which I find immensely interesting, is an article written by film scholar and co-founder of the Harvard Film Archive Vlada Petric in 1974-1975, “From a Written Film History to a Visual Film History”, published in Cinema Journal in 1975. Different from Bellour’s hermeneutical approach to film analysis, Petric suggested a more scientistic, quantitative approach to especially the study of montage and style, as a way to create a more accurate empirical basis for film historiography. While the canon of films and historiographical assumptions underpinning Petric’s article may seem dated today, I think the article is hugely interesting in the way it articulates a special role for film archives and insists on an analytical situation in which scholars scrutinize film prints in archives to develop a “visual/analytical” approach to film analysis. As Petric wrote in his article:

…the appropriate methodology of film history cannot be attained in our time without the full cooperation of the film archives, which possess the prints and have access to technical facilities, without which it is impossible to grasp the cinematic structure of a film.

In many respects, these projects and theoretical texts underpin the MIMEHIST project’s development of annotation functionalities for media scholars, working with the example of EYE’s Jean Desmet Collection. It would undoubtedly sound a bit grand to claim that MIMEHIST will fulfill all the ambitions articulated by them. However, it is certainly a theoretical lineage which we feel indebted to and find inspiration in, in developing video annotation for historical analysis in new directions. Working from these theoretical coordinates we try to nurture a historical methodology relying on video annotation which we may qualify as more open-ended and dynamic, something we will also try to achieve by drawing on annotation practices in other disciplines.

Open-ended and dynamic historical representations in annotated film editions

How are we planning to achieve this and how will this differ from previous multimedia editions of (archival) films or annotation software/projects?

There are several aspects in which we could say that our project wishes to create a research situation which allows for doing audiovisual analysis reminiscent of the projects discussed above but which also builds on them by drawing on annotation software and practices in other disciplines.

First of all, we will facilitate and encourage segmentation and labelling of films as a basis for interpretation of, in particular, stylistic and formal aspects of films. The annotation environment resulting from MIMEHIST should ideally allow scholars to analyze digitized films from the Jean Desmet Collection – and audiovisual material more broadly – through a process of segmentation and labelling. It will allow scholars to engage in a process of coding through labelling, in a manner which we consider congruent with the “plumbing of depths” of film segmentation as suggested by Raymond Bellour; a subjective interpretation process which is not finite, but which may have different end points for different scholars because they observe and annotate different aspects of a film. By creating different layers and timelines in the CLARIAH Media Suite’s Segmenting Player scholars may categorize shot types, scenes and sequences following a personal coding scheme. Furthermore, scholars may also use such a coding procedure to get a better grasp on the structure of films, as suggested by Petric, in order to establish firm empirical evidence for their interpretation. MIMEHIST will not facilitate statistical functionalities however – for instance shot boundary detection – but may allow scholars to annotate and count film features as they like.

A first glimpse of the CLARIAH Media Suite Segmenting Player in its early development phase.

In order to develop this, MIMEHIST also looks beyond film theory, and finds inspiration in qualitative methodology and Grounded Theory approaches used for coding video or audio interviews in the social sciences, anthropology and ethnography. What scholars in these fields often do is to use video/audio editing software to tag and label interview bits and organize them into different categories and color-coded layers on a time-line, while linking them to related materials. This work lasts until the researcher feels the material is exhausted in a subjective – or collaborative – coding process. The coding procedures of qualitative methodology may be considered somewhat similar to the mise-en-abîme -feeling of film segmentation as described by Bellour because it is an analytical process without a logical end point. For MIMEHIST this is interesting because it allows us to draw productively on a lot of expertise and practical experiences on interpretation of audiovisual material from qualitative methodology in order to develop film and media annotation further.

Beyond labelling and tagging, it is also the idea that scholars should be able to present an annotated version of a film with MIMEHIST and synthesize the main findings of their work in an edited and staged version for peers as a last step in their historiographical operation. This may be reminiscent of the kind of presentation format which Hyperkino developed. In particular, it will rely on links made between films and related material in the Desmet Collection which researchers can organize in a personal user space. Furthermore, the presentation format of MIMEHIST should ideally offer a kind of video playback mode in which annotations – or footnotes as they were called in Hyperkino – may be accessed by clicking on an icon. It will also differ in several fundamental aspects from a project such as Hyperkino. First of all, MIMEHIST will allow scholars to make different versions of the same film. Scholars may thus produce different – perhaps competing – interpretations of the same title. This should allow for a greater multiplicity of viewpoints to be expressed about the films in the Desmet Collection from various theoretical angles and will also, hopefully, nurture a more open-ended and dynamic historical interpretation of the films as well as representations of them. Whereas Hyperkino relied on the closed DVD-format to present one interpretation, MIMEHIST will not have the limitation of such a format. Furthermore, we also aim to make the scholarly annotation work more transparent. For instance, in addition to making an edited, annotated version available to other scholars, a researcher may also offer access to the work folder and coding underlying a version so as to allow other scholars to critically scrutinize exactly what they did or to engage in collaborative annotation.

Hopefully this will all work out. These functionalities are now in the development phase and what I have discussed above is the goal we are working towards. I will write more about the case studies which the project will focus on later. But as it looks now, I have lined up two types of cases: one case, which will focus on non-fiction films in the Desmet Collection, with particular attention to the representation of war during WWI and the films’ distribution in the Netherlands in this period. A second case study will focus on an incomplete film, to analyse how its form and distribution life may possibly be better understood by linking it to film-related material in the Desmet Collection. More about that later.

References

Bellour, Raymond, ed. Constance Penley. The analysis of film . Bloomington: Indiana University Press, 2000.

Gauthier, Christophe. La passion du Cinéma. Cinéphiles, ciné-clubs et salles spécialisées à Paris de 1920 à 1929 . Paris: AFRHC and École des Chartes, 1999.

Grant, Catherine, “Film and Moving Image Studies: Re-Born Digital? Some Participant Observations”, in Frames Cinema Journal, vol. 1, no. 1 (2012)

Olesen, Christian Gosvig, Eef Masson, Jasmijn van Gorp, Giovanna Fossati & Julia Noordegraaf “Data-Driven Research for Film History: Exploring The Jean Desmet Collection” in The Moving Image , Vol. 16, No. 1 (2016)

Petric, Vladimir, “From a Written Film History to a Visual Film History”, in Cinema Journal, Vol. 14, No. 2, Symposium on the Methodology of Film History (Winter, 1974-1975) 21.

Singer, Ben, “Hypermedia as a Scholarly Tool”, in Cinema Journal, vol. 34, no. 3 (1995)

Witt, Michael. Jean-Luc Godard. Cinema Historian . Bloomington: Indiana University Press, 2013.