Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
126 commits
Select commit Hold shift + click to select a range
efb4bf5
speedup: use fast, async cache
Pfeil Aug 28, 2024
160bfe0
speedup: use default work stealing executor for "async" cache
Pfeil Aug 29, 2024
f976bdd
speedup: use extra executors for validation and deserialization
Pfeil Aug 29, 2024
0d8ce40
Merge branch 'master' into validation-speedup-experiments
Pfeil Nov 8, 2024
c7e6cbb
chore: rename TypeRegistry to DtrTest, as it depends on dtr-test sche…
Pfeil Nov 16, 2024
439e8b0
feat: add main code base for type-api support
Pfeil Nov 20, 2024
5ee3e4a
feat: more flexible validation
Pfeil Nov 20, 2024
dabaa4d
feat: use virtual threads for async execution
Pfeil Nov 20, 2024
3ff56e4
cleanup: remove unused methods which depended on the old TypeDefiniti…
Pfeil Nov 20, 2024
91538cb
fix: assumed wrong type of baseUrl
Pfeil Nov 20, 2024
df87d27
chore: cleanup, formatting, typos
Pfeil Nov 20, 2024
a04dc32
feat: introduce SchemaSetGenerator to support different schema genera…
Pfeil Dec 12, 2024
9b89c01
Merge remote-tracking branch 'origin/master' into validation-speedup-…
Pfeil Dec 12, 2024
2221af1
added bulk create feature
maximiliani Dec 13, 2024
b9d0da4
added more information
maximiliani Dec 17, 2024
0695cc6
feat: read additionalAttributes allowed from profiles and consider th…
Pfeil Jan 3, 2025
8544909
fix: throw error on PID not found
Pfeil Jan 7, 2025
7938072
fix: missing slash after type api endpoint
Pfeil Jan 7, 2025
85db2a0
fix(tests): check assumptions in etag test setup
Pfeil Jan 7, 2025
19b030a
Merge remote-tracking branch 'origin/master' into validation-speedup-…
Pfeil Jan 7, 2025
e7361f4
fix(tests): type api test for complex type now properly reflects the …
Pfeil Jan 7, 2025
18a06ba
test: some general profile attributes with type API
Pfeil Jan 7, 2025
69b0d57
cleanup: remove unused domain classes
Pfeil Jan 8, 2025
aaba94d
fix: apply json schema validation correctly
Pfeil Jan 8, 2025
c4d1aa2
fix: a attribute which is not registered, shall being presented as a …
Pfeil Jan 8, 2025
24c5beb
cleanup: apply linter suggestions
Pfeil Jan 8, 2025
186e4a8
fix(tests): profiles which do not specify their handling of additiona…
Pfeil Jan 8, 2025
e61ba54
fix(tests): sanity check about the number of tests to execute with lo…
Pfeil Jan 8, 2025
4fd4543
cleanup: apply linter suggestions
Pfeil Jan 8, 2025
04bff17
docs: add note about why this test currently fails
Pfeil Jan 8, 2025
ef0242e
fix: use different executor per cache
Pfeil Jan 13, 2025
7fdf554
refactor: use modern http client in dtr test schema generator
Pfeil Jan 13, 2025
dc7fa02
fix(CI): use temurin openJDK instead of zulu openJDK.
Pfeil Jan 13, 2025
bc2bb02
fix: throw correct errors on pid creation
Pfeil Jan 13, 2025
d1144f3
cleanup: spelling and outdated comments
Pfeil Jan 13, 2025
70ee315
cleanup: avoid log spamming in the large record tests.
Pfeil Jan 13, 2025
ae2625a
refactoring
maximiliani Jan 14, 2025
3ea8931
fix(test): improve output and adjust test to new profile behavior
Pfeil Jan 14, 2025
b12a4dd
fix(test): fix invalid values before validation of real record
Pfeil Jan 14, 2025
8a9609b
cleanup: remove useless newline in exception message
Pfeil Jan 14, 2025
90539c2
cleanup: make executor types easily changeable in one line
Pfeil Jan 14, 2025
643e2be
cleanup: rewrite unpacking of exceptions without recursion
Pfeil Jan 15, 2025
d2b5582
cleanup: better names for some config properties
Pfeil Jan 17, 2025
a19e2ea
cleanup: use clearer name for additional attributes
Pfeil Jan 17, 2025
d9d507c
docs: document config properties in application-default.properties
Pfeil Jan 17, 2025
698392e
Merge remote-tracking branch 'origin/master' into validation-speedup-…
Pfeil Jan 17, 2025
8a9691d
added time debugging logs
maximiliani Jan 20, 2025
c1fdc5d
feat: warn via logging if the cache size is unusual low.
Pfeil Jan 21, 2025
0aada09
feat: implement dockerized benchmarks
Pfeil Jan 21, 2025
c6fb821
fix: make sure test script does not influence caches or else in prepa…
Pfeil Jan 22, 2025
4c9350b
cleanup: remove TODO comment
Pfeil Jan 22, 2025
a3c496b
benchmarks: wait longer between health check and starting the requests
Pfeil Jan 22, 2025
8fe47fc
benchmarks: avoid virtual threads for now until the ecosystem is more…
Pfeil Jan 22, 2025
ad429ba
benchmarks: add first results
Pfeil Jan 22, 2025
e4aa487
CI: test zulu now that we do not use virtual threads anymore
Pfeil Jan 22, 2025
81f9d23
CI: remove zulu
Pfeil Jan 22, 2025
85e1ddf
docs: fix information about validation times
Pfeil Jan 22, 2025
a92bd4d
feat: log warnings if a request took longer than 400ms
Pfeil Jan 23, 2025
0927d2b
test: test for records with invalid values
Pfeil Jan 24, 2025
a29e029
cleanup: fix linter warnings
Pfeil Jan 24, 2025
471a5a3
Merge pull request #218 from kit-data-manager/validation-speedup-expe…
Pfeil Jan 24, 2025
76e4641
version: set version to v3.0.0-rc1
Pfeil Jan 24, 2025
8c2377b
Merge branch 'master' into dev-v3
Pfeil Jan 27, 2025
4c17b00
fix: use a EOSC DTR type to avoid regular issues with dtr-test within…
Pfeil Jan 27, 2025
61c449f
Implement meta-resolver.
Pfeil Jan 29, 2025
2e0b6db
chore: more ergonomic implementation of index calculation in HandleIndex
Pfeil Jan 29, 2025
58f9984
chore: optimize record-to-handleValues conversion and explicitly unco…
Pfeil Jan 29, 2025
919c4de
chore: move HandleProtocolAdapter to its own module
Pfeil Jan 29, 2025
76236f5
chore: split HandleProtocolAdapter into different responsibilities.
Pfeil Jan 29, 2025
3608964
Merge pull request #271 from kit-data-manager/separate-resolver
Pfeil Jan 29, 2025
41ccb11
Merge branch 'dev-v3' into 109-multipleFAIR_DOs
maximiliani Jan 31, 2025
77266b6
minor fixes
maximiliani Feb 4, 2025
8f2a29b
chore: switch to other json schema validator
Pfeil Feb 12, 2025
dcb1605
cleanup: apply linter suggestions
Pfeil Feb 12, 2025
8014464
added some error handling mechanisms
maximiliani Feb 18, 2025
3b035e6
fixed typo
maximiliani Feb 18, 2025
aeb8a8d
fixed typo that stopped the tests from working
maximiliani Feb 18, 2025
b3662a1
added builder for PIDRecords and PIDs for testing
maximiliani Feb 19, 2025
c88eebc
added cloning capability for domain classes PIDRecord and PIDRecordEn…
maximiliani Feb 19, 2025
a7e980d
added capability to connect builders
maximiliani Feb 19, 2025
2a56592
ci: create multi-arch docker images instead of amd64 only
Pfeil Mar 4, 2025
8294bca
ci: remove custom docker setup and use only documented platform types…
Pfeil Mar 4, 2025
23c1134
ci: add custom docker setup again (revert removal)
Pfeil Mar 4, 2025
ed8da03
ci: fix wrong step reference name for docker annotations
Pfeil Mar 4, 2025
3eb410b
ci: for now, restrict the container targets to what we actually need
Pfeil Mar 4, 2025
71b6d8f
ci: pre-build java, then reuse in images
Pfeil Mar 4, 2025
10a34aa
ci: use windows runner for docker ci
Pfeil Mar 4, 2025
2aba06f
ci: fix windows runner for docker ci
Pfeil Mar 4, 2025
31077e3
Revert "ci: fix windows runner for docker ci"
Pfeil Mar 4, 2025
89a3681
Revert "ci: use windows runner for docker ci"
Pfeil Mar 4, 2025
c2551d6
ci: do not create image for windows for now
Pfeil Mar 4, 2025
ffdd7e9
added tests
maximiliani Jun 3, 2025
edb1dec
added tests
maximiliani Jun 4, 2025
44c98f2
fix(EmbeddedStrictValidatorStrategy): await profile validation future…
Pfeil Jun 10, 2025
e02b058
test(RestWithInMemoryTest): add test for record with only a profile
Pfeil Jun 10, 2025
acbb333
Merge pull request #309 from kit-data-manager/305-dev-v3-branch-pid-r…
Pfeil Jun 10, 2025
315379d
Merge branch 'dev-v3' into 109-multipleFAIR_DOs
Pfeil Jun 11, 2025
22011d4
fix: correct message formatting in ExternalServiceException
Pfeil Jun 11, 2025
45646df
fix: filter out schemas with errors, as they may accept anything
Pfeil Jun 11, 2025
1930938
feat: add logging for validation errors and exceptions in AttributeInfo
Pfeil Jun 11, 2025
262c25a
feat: add logging for schema retrieval errors in SchemaSetGenerator
Pfeil Jun 11, 2025
16423b2
refactor: remove obsolete schema query test in TypeApiTest
Pfeil Jun 11, 2025
efdb386
Merge branch 'dev-v3' into 262-new-schema-validator
Pfeil Jun 11, 2025
bc39406
fix(test): update checksum validation examples to new validator
Pfeil Jun 11, 2025
4484db7
fix: attribute value parsing and corresponding tests
Pfeil Jun 11, 2025
2eade02
fix: schema presence check used isEmpty, meant isNull
Pfeil Jun 11, 2025
97cb372
fix: improve logging message for schema retrieval errors
Pfeil Jun 11, 2025
7d55112
Revert "fix(test): update checksum validation examples to new validator"
Pfeil Jun 11, 2025
4f4bf89
fix: enhance schema validation logging and null checks
Pfeil Jun 11, 2025
7aa7a0f
deps: update json-schema-validator dependency to version 1.5.7
Pfeil Jun 12, 2025
f3d25c3
fix: update schema validation logic and improve error handling
Pfeil Jun 12, 2025
e4dde7e
Merge pull request #310 from kit-data-manager/development
Pfeil Jun 12, 2025
857d912
Merge pull request #272 from kit-data-manager/262-new-schema-validator
Pfeil Jun 12, 2025
cceda8c
Merge pull request #311 from kit-data-manager/development
Pfeil Jun 12, 2025
7d574c3
[Gradle Release Plugin] - pre tag commit: 'v2.2.1'.
Pfeil Jun 12, 2025
201395e
[Gradle Release Plugin] - new version commit: 'v2.2.2'.
Pfeil Jun 12, 2025
74b9428
added mapping from user-provided PIDs to real Handle PIDs to result o…
maximiliani Jul 15, 2025
4e168d7
Improved JavaDoc
maximiliani Jul 15, 2025
b66c146
minor fixes
maximiliani Jul 15, 2025
d1e0782
minor fixes to the Swagger API docs
maximiliani Jul 15, 2025
1d481c4
minor style fixes
maximiliani Jul 16, 2025
8b2ae49
Merge remote-tracking branch 'origin/dev-v3' into 109-multipleFAIR_DOs
Pfeil Jul 17, 2025
fd4478c
Merge pull request #247 from maximiliani/109-multipleFAIR_DOs
Pfeil Jul 17, 2025
260c66d
Merge branch 'development' into dev-v3
Pfeil Jul 29, 2025
0f68003
Merge branch 'master' into dev-v3
Pfeil Jul 29, 2025
269776c
fix: API for creating multiple PIDs returns mappings now with full PI…
Pfeil Sep 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,16 @@ jobs:
strategy:
matrix:
operating-system: [ubuntu-latest, macOS-latest, windows-latest]
jdk: [ 21 ] # (open)JDK releases
jdk: [ 21 ]
distro: ['temurin']

steps:
- uses: actions/checkout@v4
- name: Set up openJDK version
uses: actions/setup-java@v4
with:
java-version: ${{ matrix.jdk }}
distribution: 'zulu' # =openJDK
distribution: ${{ matrix.distro }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@v4
- name: Build and Test with Gradle
Expand All @@ -45,7 +46,8 @@ jobs:
- name: Docker build and test
if: matrix.operating-system == 'ubuntu-latest' && matrix.jdk == 21
run: |
curl --location --remote-name https://github.com/Orange-OpenSource/hurl/releases/download/4.0.0/hurl_4.0.0_amd64.deb
sudo dpkg -i hurl_4.0.0_amd64.deb
VERSION=6.0.0
curl --location --remote-name https://github.com/Orange-OpenSource/hurl/releases/download/$VERSION/hurl_${VERSION}_amd64.deb
sudo apt update && sudo apt install ./hurl_${VERSION}_amd64.deb
time bash ./docker/test_docker.sh
shell: bash
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -135,4 +135,8 @@ $RECYCLE.BIN/
**/*.properties
!/config/application-default.properties
!/config/application-docker.properties
##############
##############

test_prefix_data/
config/*.bin
config/*.pem
3 changes: 2 additions & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ authors:
title: "Typed PID Maker"
type: software
abstract: The Typed PID Maker is the entry point to integrate digital resources into the FAIR DO ecosystem. It allows to create PIDs for resources and to provide them with the necessary metadata to ensure that the resources can be found.
date-released: 2020-10-01
date-released: 2025-06-12
url: "https://github.com/kit-data-manager/pit-service"
repository-code: "https://github.com/kit-data-manager/pit-service"
license: Apache-2.0
version: 2.2.1
26 changes: 23 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,27 @@ PID = prefix + (branding + uniquely-generated-string)

All other configuration properties affect only the `uniquely-generated-string`. For example, you may choose a different generation method (UUID (default) or Hex Chunks) enforce casing (lower-case, upper-case).

## What PID record validation means

Our validation steps include the following:

1. For each attribute, check if the values are valid according to their type specification.
- Values are considered to be in JSON format. If a type is complex, this implies a string-serialized JSON object.
2. For each attribute which indicates a profile, resolve the profile definitions from the values of the attributes.
- Attributes which indicates profiles are internally known, but may be added using the [configuration](https://github.com/kit-data-manager/pit-service/blob/master/config/application-default.properties).
3. For each profile definition, check if mandatory attributes are present.
4. For each profile definition, check if only repeatable attributes have multiple occurrences.

This implies the following properties:

- A profile is not required,
- but all profiles which are present are being used for validation,
- and all of them have to pass.
- Additional attributes are allowed if specified in the configuration of the Typed PID Maker instance.
- Otherwise, they are not allowed. Which makes it almost impossible to use multiple profiles.
- Only dtr-test supports an "additionalAttributesAllowed" boolean property per profile,
- But as it will not last and other DTRs do currently not support it, we don't support it either.

## How to build

> Note: Alternatively, you can use the docker image.
Expand All @@ -111,9 +132,8 @@ All other configuration properties affect only the `uniquely-generated-string`.
- Building (with tests): `./gradlew clean build`
- Building (with verbose test output) `./gradlew -Dprofile=verbose clean build`
- Building (without tests): `./gradlew clean build -x test`
- Run docker integration tests:
- `./gradlew clean build` (by default, this will reuse the local build)
- `time bash ./docker/test_docker.sh` (runs test script)
- Run docker integration tests: `time bash ./docker/test_docker.sh` (will reuse the local build)
- Run dockerized validation benchmarks: `time bash ./docker/test_docker.sh benchmark` (will reuse the local build)
- Doing a release: `./gradlew clean build release`
- Will prompt you about version number to use and next version number
- Will make a git tag which can later be used in a GitHub release
Expand Down
11 changes: 5 additions & 6 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,16 @@ repositories {
}

ext {
springDocVersion = '2.8.8'
springDocVersion = '2.8.9'
}

dependencies {
// Due to the spring boot gradle plugin, we can omit versions in org.springframework.*
// dependencies. It will automatically choose the fitting ones.
implementation("edu.kit.datamanager:service-base:1.3.4")
implementation("edu.kit.datamanager:repo-core:1.2.5")
// com.google.common, LoadingCache
implementation("com.google.guava:guava:33.4.8-jre")
// AsyncLoadingCache https://github.com/ben-manes/caffeine
implementation("com.github.ben-manes.caffeine:caffeine:3.1.8")

// spring core, e.g. @EnableJpaRepositories
implementation "org.springframework:spring-core"
Expand All @@ -81,8 +81,7 @@ dependencies {
implementation "org.springframework.data:spring-data-elasticsearch"

// More flexibility when (de-)serializing json:
//implementation("com.monitorjbl:spring-json-view:1.1.0")
implementation("com.github.everit-org.json-schema:org.everit.json.schema:1.14.4")
implementation(group: 'com.networknt', name: 'json-schema-validator', version: '1.5.7');

implementation('org.apache.httpcomponents:httpclient:4.5.14')
implementation('org.apache.httpcomponents:httpclient-cache:4.5.14')
Expand Down Expand Up @@ -155,7 +154,7 @@ test {
println "Tests will have verbose output"
testLogging {
// tests are never "up-to-date", always print everything
outputs.upToDateWhen {false}
outputs.upToDateWhen { false }
// show stdio when tests are running
showStandardStreams = true
// for junit5
Expand Down
Loading