Research project BR/165/A3/INSIGHT (Research action BR)
PROJECT DESCRIPTION
Heritage institutions across the globe are currently going through a far-reaching phase of digitization. Because of the rapid pace of this process, these institutions lack the time, staff and funding to annotate these raw, digitized materials with descriptive metadata. Such metadata are typically provided by trained experts for the original objects, using thesauri or controlled vocabularies. Metadata remain crucial for providing high-quality access to heritage collections, especially when it comes to a broader audience in society. Larger, international players in the GLAM sector increasingly publish their collections in a digitally enriched form, under liberal licenses that encourage re-use. For smaller players in the field this nevertheless remains very difficult, because of financial and other practical issues.
In this project, we target the digital assets of two museum clusters in Brussels: Royal Museums of Fine Arts of Belgium and Royal Museums of Art and History. This project aims to deploy the recent advances in Artificial Intelligence (language technology and computer vision in particular) to support the enrichment of these collections with descriptive metadata. An important focus of this project is the issue of transferring knowledge from one (e.g. larger) institution, such as The Rijksmuseum, to other players in the field. To this end, we investigate issues relating to multimodality or the way in which we can simultaneously model different information streams about digital heritage objects (e.g. in different languages, or across different media). Apart from multimodality, multilinguality will be another crucial aspect of our research, which is of course important in the context of federal heritage collections in Belgium. The end goal of this project is to develop and release a series of practical Machine Learning tools for managing digital collections. The main outcome of this project will be an export of the digital collections involved in a “Europeana-ready” format, which will contribute to the broader accessibility of these collections.
Both subprojects (language technology on the one hand, and computer vision on the other hand) are situated in the “Deep Learning” paradigm in Machine Learning. Within this line of research, several important breakthroughs have been realized in the past years through making use of deep neural networks. Deep learning models are flexible enough to model multimodal information streams about heritage object simultaneously (e.g. in multiple languages at once or even across different media), which gives these methods a significant advantage over other techniques. First, the project has planned an intra-collection phase, in which techniques are tested on objects within a single modality and within a single collection. In the next, inter-collection phase, we will investigate how these models can be used to transfer knowledge from one collection to another (e.g. automatically date object from the Brussels collections using the dates provided for objects in the Rijksmuseum dataset). Of methodological significance is our aim to compare deep learning-methods to more established learning methods in the field of AI.
This project can be situated within the “Digital Humanities” and brings together a unique combination of researchers from the Humanities (e.g. art history) with experts in Artificial Intelligence. Across the world, digital humanities have been gaining much momentum in recent years and this project aims to contribute to this development from a Belgian context. In this way, the project can be expected to contribute to ongoing initiatives which aim to investigate how computational methods can enhance and support conventional research practices in the Humanities.
INSIGHT will make the important contribution of opening up and providing access to the digital collections of two major federal heritage institutions in Brussels and Belgium. Moreover, this project will stimulate the debate surrounding A.I. and society and raise awareness as to the increasing role which computers can (and will) play in the future Arts.
Throughout the project’s lifecycle, its partners will present and publish their research at scientific conferences and peer-reviewed journals (several times a year). On an informal webblog we also frequently report on our progress. INSIGHT will also host a number of outreach events, including our launch meeting (7 November 2017 in Brussels). The main deliverable of this project will be an export of the “Europeana-ready” museum data by the end of the project, in a format that is widely accessible to a large audience.
INSIGHT on the website Brain-be
Intelligent Neural Systems as InteGrated Heritage Tools (INSIGHT) : final report
Geurts, Pierre - Lasaracina, Karine - Verbruggen, Christophe ... et al. Brussels : Belgian Science policy, 2023 (SP3234)
[To download]
[To order]
