Map Examples

[paulduchesne]

I know we were trying to avoid parallel conversations, but I thought I would let you know that I have started applying this model to a data sample kindly provided by Heidi.

The source file can be found here, which is an XML record for a single work taken from the Bundesarchiv OAI-PMH endpoint. I have been using this as an opportunity to work with RML, which allows for the specification of an explicit data transformation which is processed by RMLMapper to produce a turtle file ready to be loaded into a triplestore.

As I work through applying the model I will highlight issues here and look forward to any thoughts on how to resolve them.

[paulduchesne]

@ladislav-nfa @torbjornbp @natashafairbairn @stephenmcconnachie @heftberger @annahoegner

[stephenmcconnachie]

Paul if you need another record for working example model, Natasha found a BFI National Archive example that is complex and a mix of messy and clean data – let me know if you need it and I can send JSON or XML.

[paulduchesne]

Please do - would be good to have a plurality of examples to look at. Maybe XML serialization? Thanks!

[stephenmcconnachie]

I’ll email you with that XML

[paulduchesne]

I've tried various iterations and approaches to the transformation from XML to TTL, but the strategy I will be focusing on now is using RML to define a formal transformation to match the structure of the ontology, self declaring all entities.

A secondary process, likely via RDFLIB, would do find and replace: eg selfdeclared https://bundesarchiv.org/activity/Darsteller is replaced by https://fiafcore.org/ontology/cast, as well as replace replacing the non-vocabulary elements with fiaf identifiers (WorkVariant, Agent, etc). If no relevant term is available in the source data (for example manifestation type) declare the superclass.

[paulduchesne]

Notes around the transformation from Bundesarchiv XML to ontology-mapped TTL.
Work identifier is synthesised from "Filmwerk" uuid:

<Filmwerk uuid="cd6685c5-d104-4cef-8173-2aeafdfcc78c"> 

translates to

<https://www.bundesarchiv.de/work/cd6685c5-d104-4cef-8173-2aeafdfcc78c> a fiaf:WorkVariant;

Note that a workvariant type is defined (as "Filmart: Spielfilm"), but it does not seem to obvious conform with the FIAF manual terms (monographic, serial) so is not specified in the transformation.

[paulduchesne]

Working with blank nodes XML -> RML is quite difficult, but this model using parentTriplesMap seems to work okay:

@prefix : <http://example.org/rules/> .
@prefix fiaf: <https://fiafcore.org/ontology/> .
@prefix ql: <http://semweb.mmlab.be/ns/ql#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix rr: <http://www.w3.org/ns/r2rml#> .

fiaf:work_transform a rr:TriplesMap;
  rml:logicalSource [
    rml:source "cd6685c5.xml";
    rml:referenceFormulation ql:XPath;
    rml:iterator "/Filmwerk"
  ].

fiaf:work_transform rr:subjectMap [
    rr:template "https://www.bundesarchiv.de/work/{@uuid}";
    rr:class fiaf:WorkVariant
  ].

fiaf:work_transform rr:predicateObjectMap [
  rr:predicate fiaf:workProperty;
  rr:objectMap [
    rr:constant "work work work";
  ]
].

fiaf:work_transform rr:predicateObjectMap [
  rr:predicate fiaf:hasActivity;
  rr:objectMap [
    rr:parentTriplesMap fiaf:act ;
  ]
].

fiaf:act a rr:TriplesMap;
  rml:logicalSource [
    rml:source "cd6685c5.xml";
    rml:referenceFormulation ql:XPath;
    rml:iterator "/Filmwerk/Credit/Person"
  ].

fiaf:act rr:subjectMap [
   rr:termType rr:BlankNode;
    rr:class fiaf:Activity;
].

fiaf:act rr:predicateObjectMap [
  rr:predicate fiaf:ActivityProperty;
  rr:objectMap [
    rr:constant "activity activity activity"
  ]
].

Produces resulting turtle:

@prefix : <http://example.org/rules/> .
@prefix fiaf: <https://fiafcore.org/ontology/> .
@prefix ql: <http://semweb.mmlab.be/ns/ql#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix rr: <http://www.w3.org/ns/r2rml#> .

_:0 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:1 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:10 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:11 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:2 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:3 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:4 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:5 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:6 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:7 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:8 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

_:9 a fiaf:Activity;
  fiaf:ActivityProperty "activity activity activity" .

<https://www.bundesarchiv.de/work/cd6685c5-d104-4cef-8173-2aeafdfcc78c> a fiaf:WorkVariant;
  fiaf:hasAgent _:0, _:1, _:10, _:11, _:2, _:3, _:4, _:5, _:6, _:7, _:8, _:9;
  fiaf:workProperty "work work work" .

[paulduchesne]

More involved example, compact formatting:

@prefix : <https://fiafcore.org/mapping/> .
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

:work1
  a rr:TriplesMap ;
  rml:logicalSource [
    rml:source "cd6685c5.xml" ;
    rml:referenceFormulation <http://semweb.mmlab.be/ns/ql#XPath> ;
    rml:iterator "/Filmwerk"
  ] ;
  rr:subjectMap [
    rr:template "https://www.bundesarchiv.de/work/{@uuid}" ;
    rr:class <https://fiafcore.org/ontology/WorkVariant>
  ] ;
  rr:predicateObjectMap [
    rr:predicate <https://fiafcore.org/ontology/workProperty> ;
    rr:objectMap [ rr:constant "work work work" ]
  ], [
    rr:predicate rdfs:label ;
    rr:objectMap [ rml:reference "IDTitel" ]
  ], [
    rr:predicate <https://fiafcore.org/ontology/hasActivity> ;
    rr:objectMap [ rr:parentTriplesMap :activity1 ]
  ] .

:activity1
  a rr:TriplesMap ;
  rml:logicalSource [
    rml:source "cd6685c5.xml" ;
    rml:referenceFormulation <http://semweb.mmlab.be/ns/ql#XPath> ;
    rml:iterator "/Filmwerk/Credit/Person"
  ] ;
  rr:subjectMap [ rr:termType rr:BlankNode ] ;
  rr:predicateObjectMap [
    rr:predicate <https://fiafcore.org/ontology/ActivityProperty> ;
    rr:objectMap [ rr:constant "activity activity activity" ]
  ], [
    rr:predicate rdf:type ;
    rr:objectMap [ rr:template "https://www.bundesarchiv.de/role/{Funktion/@Funktion}" ]
  ], [
    rr:predicate <https://fiafcore.org/ontology/hasAgent> ;
    rr:objectMap [ rr:template "https://www.bundesarchiv.de/agent/{@uuid}" ]
  ] .

:agent1
  a rr:TriplesMap ;
  rml:logicalSource [
    rml:source "cd6685c5.xml" ;
    rml:referenceFormulation <http://semweb.mmlab.be/ns/ql#XPath> ;
    rml:iterator "/Filmwerk/Credit/Person"
  ] ;
  rr:subjectMap [
    rr:template "https://www.bundesarchiv.de/agent/{@uuid}" ;
    rr:class <https://fiafcore.org/ontology/Agent>
  ] ;
  rr:predicateObjectMap [
    rr:predicate <https://fiafcore.org/ontology/FirstName> ;
    rr:objectMap [ rml:reference "@Vorname" ]
  ], [
    rr:predicate <https://fiafcore.org/ontology/LastName> ;
    rr:objectMap [ rml:reference "@Nachname" ]
  ] .

produces

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix rr: <http://www.w3.org/ns/r2rml#> .

_:0 a <https://www.bundesarchiv.de/role/Schnitt%2FMontage>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/b3898d04-f988-4253-89ce-60f0b2834347> .

_:1 a <https://www.bundesarchiv.de/role/Darsteller>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/8509fafe-d14f-4452-b88c-0805d7b780db> .

_:10 a <https://www.bundesarchiv.de/role/Darsteller>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/40e0f039-98c2-4973-851b-5ee4832f91a8> .

_:11 a <https://www.bundesarchiv.de/role/Darsteller>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/5249b851-42f2-4f03-b49d-e74a9626496c> .

_:2 a <https://www.bundesarchiv.de/role/Drehbuch>, <https://www.bundesarchiv.de/role/Regie%20%2F%20Spielleitung%20%2F%20Realisation>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/ad399ad0-dcae-4a22-861e-e0bbbe233840> .

_:3 a <https://www.bundesarchiv.de/role/Produzent>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/e2c8a82f-c71f-49ba-a7c0-39a85ab9599d> .

_:4 a <https://www.bundesarchiv.de/role/Kamera%2FBild%2FBildgestaltung%2FFotografie>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/a751b465-9988-461b-a851-5bea16430669> .

_:5 a <https://www.bundesarchiv.de/role/Drehbuch>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/a2496080-1cd7-4a2b-b414-25ea6e5e839b> .

_:6 a <https://www.bundesarchiv.de/role/Kamera%2FBild%2FBildgestaltung%2FFotografie>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/d728f72b-ef54-4d4a-a68f-9592df278ba9> .

_:7 a <https://www.bundesarchiv.de/role/Musik%20%28Filmkomponist%29>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/0260476f-7f35-44ce-9b96-2cbaf2ca2300> .

_:8 a <https://www.bundesarchiv.de/role/Darsteller>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/40a3bbfe-1122-4289-9f97-6609090d53c4> .

_:9 a <https://www.bundesarchiv.de/role/Darsteller>;
  <https://fiafcore.org/ontology/ActivityProperty> "activity activity activity";
  <https://fiafcore.org/ontology/hasAgent> <https://www.bundesarchiv.de/agent/d9f7f51f-de7a-44e0-97d6-81c07a4a1d1e> .

<https://www.bundesarchiv.de/agent/0260476f-7f35-44ce-9b96-2cbaf2ca2300> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/LastName> "The Can" .

<https://www.bundesarchiv.de/agent/40a3bbfe-1122-4289-9f97-6609090d53c4> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/FirstName> "Didi";
  <https://fiafcore.org/ontology/LastName> "04. Petrikat" .

<https://www.bundesarchiv.de/agent/40e0f039-98c2-4973-851b-5ee4832f91a8> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/FirstName> "Rüdiger";
  <https://fiafcore.org/ontology/LastName> "01. Vogler" .

<https://www.bundesarchiv.de/agent/5249b851-42f2-4f03-b49d-e74a9626496c> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/FirstName> "Yella";
  <https://fiafcore.org/ontology/LastName> "02. Rottländer" .

<https://www.bundesarchiv.de/agent/8509fafe-d14f-4452-b88c-0805d7b780db> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/FirstName> "Edda";
  <https://fiafcore.org/ontology/LastName> "05. Köchl" .

<https://www.bundesarchiv.de/agent/a2496080-1cd7-4a2b-b414-25ea6e5e839b> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/LastName> "Fürstenberg, Veith von" .

<https://www.bundesarchiv.de/agent/a751b465-9988-461b-a851-5bea16430669> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/FirstName> "Martin";
  <https://fiafcore.org/ontology/LastName> "19. Schäfer" .

<https://www.bundesarchiv.de/agent/ad399ad0-dcae-4a22-861e-e0bbbe233840> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/FirstName> "Wim";
  <https://fiafcore.org/ontology/LastName> "Wenders" .

<https://www.bundesarchiv.de/agent/b3898d04-f988-4253-89ce-60f0b2834347> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/LastName> "Weitershausen, Barbara von" .

<https://www.bundesarchiv.de/agent/d728f72b-ef54-4d4a-a68f-9592df278ba9> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/LastName> "Müller, Robby" .

<https://www.bundesarchiv.de/agent/d9f7f51f-de7a-44e0-97d6-81c07a4a1d1e> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/FirstName> "Elisabeth";
  <https://fiafcore.org/ontology/LastName> "03. Kreuzer" .

<https://www.bundesarchiv.de/agent/e2c8a82f-c71f-49ba-a7c0-39a85ab9599d> a <https://fiafcore.org/ontology/Agent>;
  <https://fiafcore.org/ontology/LastName> "PIFDA" .

<https://www.bundesarchiv.de/work/cd6685c5-d104-4cef-8173-2aeafdfcc78c> a <https://fiafcore.org/ontology/WorkVariant>;
  rdfs:label """
		ALICE IN DEN STÄDTEN (1973-1974) (Originaltitel)
	""";
  <https://fiafcore.org/ontology/hasActivity> _:0, _:1, _:10, _:11, _:2, _:3, _:4, _:5,
    _:6, _:7, _:8, _:9;
  <https://fiafcore.org/ontology/workProperty> "work work work" .

[paulduchesne]

Transformation of Bundesarchiv example almost complete, findings and notes:

• Issue with formatting XML elements correctly into URIs, "Bundesrepublik Deutschland" is being rendered as
  https://www.bundesarchiv.de/country/%0A%09%09Bundesrepublik%20Deutschland%0A%09

• We are consciously generating a single production event blank node against the work.

• BA have provided production start and end dates, we only have hasEventDate currently in the ontology.

• Dates are being rendered to the graph as strings (wrong datatype).

• Interestingly BA have used the same term for form and genre.

• Mavis and thomson ids not retained.

• "Funktionsanmerkung" appears to be used mostly for character names. First problem, no appropriate property in the ontology, also how to filter where a placeholder "-" has been used in source data?

• Interesting that in the BA schema, the type of the agent is expressed in the activity statements (eg Person/Koerperschaft). These are mapped Person -> person and Koerperschaft -> corporatebody.

• BA have provided first and last name separately, but this is not possible in the ontology. Should these two fields be joined together into an agent label - not obvious how to do this in RML. Also leading numbers used for credit order, could these be extracted? How to split strings in RML?

• GND for Wim Wenders in source data, can this be retained?

• Not clear what the unit of extent is. Assumed as minutes, but would this be true of all records?

• colourcharacteristic are currently wrong, these need to be modelled as blank nodes to allow for further properties.

• Technical information appears to be mostly duplicated at item and carrier level. Possibility of "pulling these up" to item, as the properties do not exist at carrier level. Good idea would be further analysis on whether these are consistently applied at both levels.

• Appropriate translation to format and element is not clear, and a good example where these might not be delineated in the same way as expressed by ontology vocabularies.