Dutch Ships and Sailors project started

a very unofficial DSS logo i madeLast week saw the kickoff of the new Clarin NL-funded project “Dutch Ships and Sailors”(*). This project will run for one year and gives me the opportunity to work with historians from both VU and Huygens ING on applying Linked Data principles to Dutch maritime-historical data. From the official description:

As a sea-faring nation, a large portion of Dutch history is found on the water. However, much of the digitized historical source material is still scattered across many databases and archives. This curation and demonstrator project aims to bring together the rich maritime historical data preserved in the many different databases. We propose a (semantic) web-based infrastructure

that will house various maritime-historical datasets. We will provide a tool chain and methodology for converting legacy datasets. The infrastructure includes common vocabularies to normalize and enrich existing data. Links are established between the datasets and to other relevant datasets on the Web. Although the infrastructure will be set up to facilitate 25+ identified datasets, we initially populate the infrastructure with four selected datasets. These will allow us to investigate two case studies in order to answer the historical research question “To what extent did patterns of shipping and recruitment in the Dutch maritime sector change over the course of the 18th and 19th centuries?”

(*) the project’s official title is Dutch Ships and Seamen, but we think this is potentially less problematic 🙂

The Verrijkt Koninkrijk Hackathon Report

On Friday, March 8th, we organized a Verrijkt Koninkrijk Linked Data Hackathon at the Intertain Lab of VU Amsterdam. The event was co-sponsored by the Network Institute. The goal of the hackathon was to allow third party developers to produce (ideas for) innovative applications beyond the Verrijkt Koninkrijk core research questions. We especially encouraged the use of the Linked Data produced in the project.

image015

As organizers, we are very happy with the produced prototypes. The benefits are following:

  • The produced applications show the (unexpected) reusability of the VK (Linked) Open Data. The applications produced or suggested give new browsing opportunities, links to other datasets or show how the data can be used in a completely novel context.The hackathon revealed that indeed the data is usable for external developers using the documentation provided. Some bugs were found, some of which could be fixed during the hackathon.
  • Important concepts around data quality were articulated by the users. Although it falls outside of the scope of this project, subsequent curation of the dat should involve considering ways of allowing experts or amateurs to correct errors in the data.
  •  The VK project data is made known to researchers and developers from related projects, for example that of Agora or BiographyNed. We expect that this ensures future use of the data by related projects.

We here present short descriptions of what the six hacker teams cooked up. Two prize winners were announced by the jury, for “best use of data” and “coolest app” respectively. The jury consisted of Kees Ribbens and Edwin Klijn from NIOD, Serge ter Braake and Victor de Boer from VU. More photos of the event can be seen at www.few.vu.nl/~vbr240/verrijktkoninkrijk/hackathon/.

TOUR APPLICATION AND TOUCH TABLE DEMO [Niels Ockeloen]  WINNER “COOLEST APP”

image024

Niels used the data from the Named Entity index to create a history browser which allows the user to browse information about WWII on basis of persons, locations, organisations, etc. (the NER classes). For this he reused the Agora Touch demonstrator. When a class is chosen a list of entities is shown with images which are resolved through the alignment with DBpedia. Niels used the LDtogo framework to map the selected data on the API interface of the Agora demo.

VERRIJKT KONINKRIJK ON FACEBOOK [Albert Merono & Wouter Beek] WINNER “BEST USE OF DATA”image016

This group set out to to recreate the network of important people of the Netherlands during WWII and their quotes in fake Facebook profiles, trying to imitate the reality of their time. We feed automatically these streams with the contents of the VK datasets: little Cliopatria and Python snippets retrieve data from SPARQL endpoints, resolve the structured XML texts, extract the quotes and expose them using the Facebook Graph API. View the project on GitHub and see the live demo at  http://www.facebook.com/verrijkt.koninkrijk

INTEGRATION WITH AGORA RIJKSMUSEUM DATA [Lourens van der Meij]
image031Lourens aligned the VK data with that of Agora Rijksmuseumusing the Amalgame alignment tool. This is used to link VK data to RM images using the Rijksmuseum API via http://eculture2.cs.vu.nl:43020/ (results shown here (pdf)) He furthermore started to use the Verrijkt Koninkrijk data to add links to VK from within our AGORA demo that is an event centered browser for the Rijksmuseum content. Very rough results show a AGORA demo entry for Duitsland.

CUBE-BASED BROWSING [Chris van Aart]
image028The application of Chris van Aart shows how the monument data from Vier en Vijf Mei can be browsed using the Cube browser on IOS. THis allows for multi-faceted browsing between Dutch war monuments. By flipping the screen, one can actually look at the RDF data!

MAP LAYERS SHOWING THE LIBERATION OF NIJMEGEN [Michiel van Dijk]
image029Michiel built a web map application showing the liberation of Nijmegen in 1944. 1940s data and current maps scan be superimposed over eachother therefore showing for example what part of the city was damaged during the liberation. Further additions include 17,19 and 20th Century maps. A demo can be seen at www.numagapp.nl An attempt was made to include Vier en Vijf Mei monument data in this dataset

INCONTEXT DATA VISUALISATION [Willem Melder]
image018Willem presented the idea to visualise the VK data using the InContext RDF visualizer for enriched publications. Unfortunately, due to time constraints, Willem did not succeed in getting everything up and running.  [screencast]

 

 

image010

Verrijkt Koninkrijk Hackathon Instructions

This post will provide all the participants of the Verrijkt Koninkrijk Hackathon with the information and data they need to start building great applications.

Piratepad
On this Piratepad, I suggest we all note down the progress and results: [http://piratepad.net/tzvFB5AGuk]

The text

In this deliverable document [ dx1- deliverable pdf], you can find more detailed information about (the origin of) the data.

The Verrijkt Koninkrijk Data concerns Dr Loe de Jong’s Het Koninkrijk der Nederlanden in de Tweede Wereldoorlog and was based on the PDFs as provided by NIOD at http://www.niod.knaw.nl/koninkrijk/ . The books have been OCRed and transformed to structured XML by researchers from the Universiteit van Amsterdam. This data is available through www.loedejongdigitaal.nl. A search interface is available at http://search.loedejongdigitaal.nl.  A resolver server was installed which responds by presenting the structure (in XML) when presented with a URL. For example, http://resolver.loedejongdigitaal.nl/nl.vk.d.1.6.1.43 is resolved to the XML fragment of that paragraph. Removing the last number of the identifier (43) results in its broader section, etcetera. Paragraphs are the smallest logical units (also, a page is not a logical unit).

Linked Data

We provide two RDF ‘stepping stones’ into the book text. The ‘Back of the Book index’ and the ‘Named Entities index’. Both are SKOS vocabularies and consist of terms pointing to resolver.loedejongdigitaal.nl URIs. These vocabularies are linked to external sources as well. All RDF is available as Linked Data at the VK Semantic Layer at http://semanticweb.cs.vu.nl/verrijktkoninkrijk/ The base namespace for the VK/NIOD triples is http://purl.org/collections/nl/niod/ (abbreviated as niod:). Datasets, mapping sets and schemata are all loaded as separate named graphs (http://semanticweb.cs.vu.nl/verrijktkoninkrijk/browse/list_graphs).

A SPARQL endpoint is also available at http://semanticweb.cs.vu.nl/verrijktkoninkrijk/sparql/ with an interactive SPARQL editor available at http://semanticweb. cs.vu.nl/verrijktkoninkrijk/flint/ You can login with “hacker”/”hacker” (if needed).

Back of the book index (BotB index)

The BotB index consists of 15,234 SKOS Concepts, consolidated from the manual index. They link to RDF blank nodes using the niod:pageRef predicate. The blank node links to individual paragraphs using niod:parRef predicates.  An example is shown below.

hack2

The BotB index is partially aligned with the NIOD thesaurus (see below), GeoNames, Cornetto and AATNed. The BotB index, the schema, the alignments are found in separate RDF turtle files

Named Entity index (NE index)

This SKOS vocabulary consists of 88,243 concepts, resulting of  Named Entity recognition. The NEs are of type person, location, organisation, misc, product and event. They link into the text through direct niod:pRef links. An example is shown below.

hack1

The NE concepts are partially aligned with DBPedia (through wikilinks established during the NER process), with GeoNames (locations only), with GTAA and the NIOD thesaurus. There is also mapping to the BotB index (can be used to use a ‘higher quality’ subset).

Example SPARQL Queries

This page lists a number of sparql queries that exploit some of the links presented above. It accompanies a paper and deliverable.

Other data sources in the semantic layer

Pillarization

Within the project, we are very much interested in the concept of pillarization and how Loe de Jong describes it. For this reason, we have added a turtle file what links Pillar concepts (Protestants, Jews, Communists, etc) to persons, organisations etc. found in the BotB index. This is a manual list of 60 links, which was semi-automatically expanded to 254 links. You can find the original list  here and the expanded one here.

We did a number of  analyses using this data, a (Dutch) PDF document describing the results can be found here [zuilen (pdf)].

License

Het Koninkrijk is licensed under the Creative Commons Naamsvermelding 3.0 Nederland licentie. The VK Linked Data are NIOD thesaurus are also available under that licenseV. The VM monument data is available under the CC0 Publieke Domein Dedicatie verklaring license