POPIMPACT - Data harmonisation

*Describes the harmonization issues encountered in the POPIMPACT use case.*

# Implementation state
 In progress

# Use Case Scope
POPIMPACT- The object of this work is to obtain a layer of pan-European buildings to which the population has been assigned.

# Spatial Scope / Priority Scope
3 scenarios, with 5 proofs of concepts, have been worked on to validate the use case
<html>
<body>


POC | Definition | Scope
-- | -- | --
POC001 | cadastral buildings and 1km grid population | Seville (Picture1)
POC002 | cadastral buildings and 250m grid population | Seville (Picture1)
POC003 | cadastral buildings and census sections population | Seville (Picture1)
POC004 | Cadastral buildings of France and 1km grid population | France (Picture2)
POC005 | Cadastral buildings of La Palma and 250m grid population | Area affected by the volcano of La Palma (Picture3)


</body>
</html>

<html>
<body>


Picture1.-Seville
--
 ![image](https://user-images.githubusercontent.com/94920015/143062402-63322989-5167-48cc-9b97-62371ef813ab.png)

</body>
</html>
<html>
<body>


Picture2.-France
--
![image](https://user-images.githubusercontent.com/94920015/143063294-da8633c4-e5cf-45a4-b7b6-a13e14de5865.png)
</body>
</html>

<html>
<body>


Picture3.-La Palma--
![image](https://user-images.githubusercontent.com/94920015/143063228-3e376c1c-3198-4edf-9cf5-9973f124c280.png)

</body>
</html>

 

#Source Schema and Format

<table>
    <tr>
        <th colspan="4">Building layer</th>        
    </tr>
    <tr>
        <th>POCs</th>
        <th>Layer</th>
        <th>Format</th>
        <th>EPSG</th>
    </tr>
    <tr>
        <td>POC001,POC002,POC003</td>
        <td>A.ES.SDGC.BU.41900.building</td>
        <td>GML</td>
        <td>25830</td>
    </tr>
    <tr>
        <td>POC004</td>
        <td>Bâtiments</td>
        <td>Shaperfile</td>
        <td>2154,4326</td>
    </tr>
    <tr>
        <td>POC005</td>
        <td>A.ES.SDGC.BU.38045.building</td>
        <td>GML</td>
        <td>32628</td>
    </tr>
</table>



<table>
    <tr>
          <th colspan="4">Population Layer</th > 
    </tr>
    <tr>
        <th>POCs</th>
        <th>Layer</th>
        <th>Format</th>
        <th>EPSG</th>
    </tr>
    <tr>
        <td>POC001,POC004</td>
        <td>JRC_POPULATION_2018</td>
        <td>Shaperfile</td>
        <td>3035</td>
    </tr>
    <tr>
        <td>POC002</td>
        <td>mep19_250</td>
        <td>Shaperfile</td>
        <td>25830</td>
    </tr>
    <tr>
        <td>POC003</td>
        <td>población-2019-secciones-censales-por-lustros-2</td>
        <td>json</td>
        <td>4326</td>
    </tr>
    <tr>
        <td>POC005</td>
        <td>Poblacion</td>
        <td>GeoJSON</td>
        <td>4326</td>
    </tr>
</table>





# Source Dataset(s)

 <table>
    <tr>
       <th colspan="3">Building layer</th>        
    </tr>
    <tr>
        <th>POCs</th>
        <th>Layer</th>
        <th>URL</th>
    </tr>
    <tr>
        <td>POC001,POC002,POC003</td>
        <td>A.ES.SDGC.BU.41900.building</td>
        <td>http://ovc.catastro.meh.es/INSPIRE/wfsCP.aspx?</td>
    </tr>
    <tr>
        <td>POC004</td>
        <td>Bâtiments</td>
        <td>https://geoservices.ign.fr/parcellaire-express-pci</td>
    </tr>
    <tr>
        <td>POC005</td>
        <td>A.ES.SDGC.BU.38045.building</td>
        <td>http://ovc.catastro.meh.es/INSPIRE/wfsCP.aspx?</td>
    </tr>
</table>
 

<table>
    <tr>
        <th colspan="3">Population Layer</th> 
    </tr>
    <tr>
        <th>POCs</th>
        <th>Layer</th>
        <th>URL</th>
    </tr>
    <tr>
        <td>POC001,POC004</td>
        <td>JRC_POPULATION_2018</td>
        <td>https://ec.europa.eu/eurostat/web/gisco/geodata/reference-data/population-distribution-demography/geostat</td>
    </tr>
    <tr>
        <td>POC002</td>
        <td>mep19_250</td>
        <td>https://www.juntadeandalucia.es/institutodeestadisticaycartografia/datosespacialesestadisticos/index.htm</td>
    </tr>
    <tr>
        <td>POC003</td>
        <td>población-2019-secciones-censales-por-lustros-2</td>
        <td>https://opendata.esri.es/datasets/ideSEVILLA::poblaci%C3%B3n-2019-secciones-censales-por-lustros-2/explore?location=37.392686%2C-5.992520%2C15.00</td>
    </tr>
    <tr>
        <td>POC005</td>
        <td>Poblacion</td>
        <td>https://www.opendatalapalma.es/datasets/poblaci%C3%B3n/explore?location=28.578079%2C-17.922026%2C13.71</td>
    </tr>
</table>


# Source Data Size

|                                                 | **Layer**                   | **Size** | **Records** |
| ----------------------------------------------- | --------------------------- | -------- | ----------- |
| **Building**                                    | A.ES.SDGC.BU.41900.building | 98 MB    | 58718       |
| Bâtiments                                       | 21 GB                       | 53851462 |
| A.ES.SDGC.BU.38045.building                     | 13 MB                       | 12381    |
| **Population**                                  | JRC\_POPULATION\_2018       | 676 MB   | 2416631     |
| mep19\_250                                      | 32 MB                       | 52599    |
| población-2019-secciones-censales-por-lustros-2 | 376 kB                      | 531      |
| Poblacion                                       | 5952 kB                     | 11621    |

# Target Schemas

The target schema  used is based on:Buildings - Annex 3	 
link: https://inspire.ec.europa.eu/data-model/approved/r4618-ir/html/index.htm?goto=2:3:2:2:7911
![image](https://inspire.ec.europa.eu/data-model/approved/r4618-ir/html/EARoot/EA2/EA3/EA2/EA2/EA7912.png)
 

# Transformed Data size
| **POC** | **Data result** | **Size** | **Records** |
| --- | --- | --- | --- |
| POC001 | results01\_bulding\_124a9cad\_6d21\_4037\_a362\_f41821d35054 | 35 MB | 58718 |
| POC002 | results02\_bulding\_bb578db5\_79c1\_4a5c\_a12c\_9464437438ea | 33 MB | 58718 |
| POC003 | results03\_bulding\_b653d3c7\_f678\_4d74\_a7ef\_14796a0e16e6 | 35 MB | 58718 |
| POC004 | results04\_bulding\_af1cb4a3\_4694\_4a82\_853d\_4f731b4e8647 | 31 GB | 51896280 |
| POC005 | results05\_bulding\_b48186b6\_28fd\_49d6\_89f4\_fc67d3ea11b4 | 1096 kB | 2599 |

# Target Data set resources
 None

# Effort / Time Spent
| **POC** | **Definition** | **Time Spent** |
| --- | --- | --- |
| POC001 | cadastral buildings and 1km grid population | 00:19:57.556388 |
| POC002 | cadastral buildings and 250m grid population | 00:10:39.677794 |
| POC003 | cadastral buildings and census sections population | 01:24:12.19327 |
| POC004 | Cadastral buildings of France and 1km grid population | 144 hr. |
| POC005 | Cadastral buildings of La Palma and 250m grid population | 00:00:24.621887 |

# Transformation Project
None

# **Harmonisation problem**
| **Harmonisation problem** | **Definition** | **Example** |
| --- | --- | --- |
| **Difficulty of access** | It is very difficult to find the datasets for all member states. | Buildings in Portugal. |
| **Segmented data** | When data sets are highly segmented, a process has to be done to bring them together in order to process them as a whole. | Building in Spain, separated into more than 8000 municipalities |
| **Diversity of formats** | Each dataset can be in a different format. | Building in France is in GeoJSON and in Spain in GML. |
| **Data without semantic interoperability** | Each data has its own data model. Sometimes even the same organism does not write the attributes in the same way. | Building of Spain different model on La Palma, with data model Inspire. |
| **Different EPSGs** | Each data has a different EPSG. and even the same datum in different areas. | Building of France. |

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

POPIMPACT - Data harmonisation #16

Implementation state

Use Case Scope

Spatial Scope / Priority Scope

Picture1.-Seville

Picture2.-France

Source Dataset(s)

Source Data Size

Target Schemas

Transformed Data size

Target Data set resources

Effort / Time Spent

Transformation Project

Harmonisation problem

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

POC	Definition	Scope
POC001	cadastral buildings and 1km grid population	Seville (Picture1)
POC002	cadastral buildings and 250m grid population	Seville (Picture1)
POC003	cadastral buildings and census sections population	Seville (Picture1)
POC004	Cadastral buildings of France and 1km grid population	France (Picture2)
POC005	Cadastral buildings of La Palma and 250m grid population	Area affected by the volcano of La Palma (Picture3)

Building layer
POCs	Layer	Format	EPSG
POC001,POC002,POC003	A.ES.SDGC.BU.41900.building	GML	25830
POC004	Bâtiments	Shaperfile	2154,4326
POC005	A.ES.SDGC.BU.38045.building	GML	32628

Population Layer
POCs	Layer	URL
POC001,POC004	JRC_POPULATION_2018	https://ec.europa.eu/eurostat/web/gisco/geodata/reference-data/population-distribution-demography/geostat
POC002	mep19_250	https://www.juntadeandalucia.es/institutodeestadisticaycartografia/datosespacialesestadisticos/index.htm
POC003	población-2019-secciones-censales-por-lustros-2	https://opendata.esri.es/datasets/ideSEVILLA::poblaci%C3%B3n-2019-secciones-censales-por-lustros-2/explore?location=37.392686%2C-5.992520%2C15.00
POC005	Poblacion	https://www.opendatalapalma.es/datasets/poblaci%C3%B3n/explore?location=28.578079%2C-17.922026%2C13.71

	Layer	Size	Records
Building	A.ES.SDGC.BU.41900.building	98 MB	58718
Bâtiments	21 GB	53851462
A.ES.SDGC.BU.38045.building	13 MB	12381
Population	JRC_POPULATION_2018	676 MB	2416631
mep19_250	32 MB	52599
población-2019-secciones-censales-por-lustros-2	376 kB	531
Poblacion	5952 kB	11621

POC	Data result	Size	Records
POC001	results01_bulding_124a9cad_6d21_4037_a362_f41821d35054	35 MB	58718
POC002	results02_bulding_bb578db5_79c1_4a5c_a12c_9464437438ea	33 MB	58718
POC003	results03_bulding_b653d3c7_f678_4d74_a7ef_14796a0e16e6	35 MB	58718
POC004	results04_bulding_af1cb4a3_4694_4a82_853d_4f731b4e8647	31 GB	51896280
POC005	results05_bulding_b48186b6_28fd_49d6_89f4_fc67d3ea11b4	1096 kB	2599

POC	Definition	Time Spent
POC001	cadastral buildings and 1km grid population	00:19:57.556388
POC002	cadastral buildings and 250m grid population	00:10:39.677794
POC003	cadastral buildings and census sections population	01:24:12.19327
POC004	Cadastral buildings of France and 1km grid population	144 hr.
POC005	Cadastral buildings of La Palma and 250m grid population	00:00:24.621887

Harmonisation problem	Definition	Example
Difficulty of access	It is very difficult to find the datasets for all member states.	Buildings in Portugal.
Segmented data	When data sets are highly segmented, a process has to be done to bring them together in order to process them as a whole.	Building in Spain, separated into more than 8000 municipalities
Diversity of formats	Each dataset can be in a different format.	Building in France is in GeoJSON and in Spain in GML.
Data without semantic interoperability	Each data has its own data model. Sometimes even the same organism does not write the attributes in the same way.	Building of Spain different model on La Palma, with data model Inspire.
Different EPSGs	Each data has a different EPSG. and even the same datum in different areas.	Building of France.

POPIMPACT - Data harmonisation #16

Description

Implementation state

Use Case Scope

Spatial Scope / Priority Scope

Picture1.-Seville

Picture2.-France

Source Dataset(s)

Source Data Size

Target Schemas

Transformed Data size

Target Data set resources

Effort / Time Spent

Transformation Project

Harmonisation problem

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions