Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions build-portable/run.bat
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ rem Validate environment
if not exist "%KARNAK_BIN%" (echo ERROR: Karnak executable not found & exit /b 1)

rem Start de-identification image service (optional)
if /i "%DEIDENTIFY_IMAGE_ENABLED%"=="true" (
set "DEIDENT_NAME=%DEIDENTIFY_IMAGE_SERVICE_NAME%"
if /i "%OCR_ENABLED%"=="true" (
set "DEIDENT_NAME=%OCR_SERVICE_NAME%"
if not defined DEIDENT_NAME set "DEIDENT_NAME=image-ocr-identifier"
if exist "%APP_DIR%!DEIDENT_NAME!\!DEIDENT_NAME!.exe" (
echo [run.bat] Starting de-identification image service...
Expand Down
7 changes: 4 additions & 3 deletions build-portable/run.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ DICOM_LISTENER_PORT=11112
DICOM_LISTENER_TLS=false

### Image Deidentification
DEIDENTIFY_IMAGE_ENABLED=true
DEIDENTIFY_IMAGE_URL=http://localhost:8000
DEIDENTIFY_IMAGE_SERVICE_NAME=image-ocr-identifier
OCR_ENABLED=true
### Base URL Karnak uses to reach the de-identification image API
OCR_URL=http://localhost:8000
OCR_SERVICE_NAME=image-ocr-identifier

### TLS configuration for DICOM-S
#TLS_KEYSTORE_PATH
Expand Down
6 changes: 3 additions & 3 deletions build-portable/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ generate_db_password() {
}

start_deidentify() {
if [[ "${DEIDENTIFY_IMAGE_ENABLED:-true}" != "true" ]]; then
log "De-identification image service disabled (DEIDENTIFY_IMAGE_ENABLED)"
if [[ "${OCR_ENABLED:-true}" != "true" ]]; then
log "De-identification image service disabled (OCR_ENABLED)"
return
fi
local name="${DEIDENTIFY_IMAGE_SERVICE_NAME:-image-ocr-identifier}"
local name="${OCR_SERVICE_NAME:-image-ocr-identifier}"
local bin="$APP_DIR/$name/$name"
if [[ ! -x "$bin" ]]; then
log "De-identification binary not found at '$bin' — skipping"
Expand Down
31 changes: 15 additions & 16 deletions docs/automatic-pixel-deidentification.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,19 +82,18 @@ Reference profiles are provided under
## Configuring the external API endpoint

Karnak calls the de-identification image API at the URL configured by the
`DEIDENTIFY_IMAGE_URL` environment variable (default
`http://localhost:8000`). It maps to the `karnak.deidentify-image.url` property
`OCR_URL` environment variable (default
`http://localhost:8000`). It maps to the `ocr.url` property
in `application.yml`:

```yaml
karnak:
deidentify-image:
url: ${DEIDENTIFY_IMAGE_URL:http://localhost:8000}
ocr:
url: ${OCR_URL:http://localhost:8000}
```

| Variable | Default | Description |
|----------|---------|-------------|
| `DEIDENTIFY_IMAGE_URL` | `http://localhost:8000` | Base URL of the de-identification image API. |
| `OCR_URL` | `http://localhost:8000` | Base URL of the de-identification image API. |

The API contract is documented in
[`src/main/resources/deidentification-api.yaml`](../src/main/resources/deidentification-api.yaml).
Expand All @@ -104,14 +103,14 @@ The API contract is documented in
## Deploying the external de-identification service

The de-identification image API is a separate service. Deploy and run it so that
it is reachable from Karnak at `DEIDENTIFY_IMAGE_URL`.
it is reachable from Karnak at `OCR_URL`.

> Deployment guide of the de-identification image API: <https://github.com/nroduit/image-ocr-identifier>

### Standard deployment

1. Deploy the de-identification image API (see the repository linked above).
2. Point Karnak to it by setting `DEIDENTIFY_IMAGE_URL` to the service base URL.
2. Point Karnak to it by setting `OCR_URL` to the service base URL.
3. Restart Karnak so the new configuration is picked up.

### Portable build
Expand All @@ -121,16 +120,16 @@ alongside Karnak. It is controlled in `run.cfg`:

```sh
### Image Deidentification
DEIDENTIFY_IMAGE_ENABLED=true
DEIDENTIFY_IMAGE_URL=http://localhost:8000
DEIDENTIFY_IMAGE_SERVICE_NAME=image-ocr-identifier
OCR_ENABLED=true
OCR_URL=http://localhost:8000
OCR_SERVICE_NAME=image-ocr-identifier
```

| Variable | Default | Description |
|----------|---------|-------------|
| `DEIDENTIFY_IMAGE_ENABLED` | `true` | Start the bundled de-identification sidecar with the portable package. Set to `false` to manage the service yourself. |
| `DEIDENTIFY_IMAGE_URL` | `http://localhost:8000` | Base URL Karnak uses to reach the service. |
| `DEIDENTIFY_IMAGE_SERVICE_NAME` | `image-ocr-identifier` | Name of the bundled service folder and executable. The sidecar lives in `<app>/<name>/` and its binary is named `<name>` (`<name>.exe` on Windows). |
| `OCR_ENABLED` | `true` | Start the bundled de-identification sidecar with the portable package. Set to `false` to manage the service yourself. |
| `OCR_URL` | `http://localhost:8000` | Base URL Karnak uses to reach the service. |
| `OCR_SERVICE_NAME` | `image-ocr-identifier` | Name of the bundled service folder and executable. The sidecar lives in `<app>/<name>/` and its binary is named `<name>` (`<name>.exe` on Windows). |

When enabled, `run.sh` / `run.bat` starts the bundled binary
(`image-ocr-identifier/image-ocr-identifier`, `.exe` on Windows) on launch and
Expand All @@ -143,8 +142,8 @@ is logged.

| Symptom | Likely cause | Action |
|---------|--------------|--------|
| Images with automatic masking are not forwarded | API unreachable or returning errors | Check the API is running and reachable at `DEIDENTIFY_IMAGE_URL`; inspect Karnak logs. |
| `Cannot reach de-identification image API ...` in logs | Wrong URL or service down | Verify `DEIDENTIFY_IMAGE_URL` and service health. |
| Images with automatic masking are not forwarded | API unreachable or returning errors | Check the API is running and reachable at `OCR_URL`; inspect Karnak logs. |
| `Cannot reach de-identification image API ...` in logs | Wrong URL or service down | Verify `OCR_URL` and service health. |
| `SOP Instance UID ... does not match` in logs | API returned a response for a different instance | Verify the API version and that it echoes back the request `sop_instance_uid`. |
| No mask applied although PHI is visible | API detected no sensitive text, or the tag values are absent from the metadata | Confirm the sensitive tags are populated in the source metadata. |

Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,15 @@
*/
package org.karnak.backend.service.profilepipe;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.net.http.HttpClient;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;
import org.dcm4che3.data.Attributes;
import org.dcm4che3.data.BulkData;
Expand All @@ -26,6 +27,7 @@
import org.jspecify.annotations.Nullable;
import org.karnak.backend.model.profilebody.MaskBody;
import org.karnak.backend.model.profilepipe.DeidentifyImageResponse;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.http.HttpEntity;
Expand Down Expand Up @@ -97,7 +99,7 @@ public class DeidentifyImageService {
/**
* @param apiBaseUrl the base URL of the de-identification image API
*/
public DeidentifyImageService(@Value("${karnak.deidentify-image.url:http://localhost:8000}") String apiBaseUrl) {
public DeidentifyImageService(@Value("${ocr.url:http://localhost:8000}") String apiBaseUrl) {
this.apiBaseUrl = apiBaseUrl;
this.objectMapper = new ObjectMapper();

Expand Down Expand Up @@ -147,7 +149,6 @@ public List<MaskBody> callDeidentifyImageApi(Attributes dcmAttributes, Map<Strin

// Send the POST request and get the JSON response
String jsonResponse;
log.info("multipartBody={}", multipartBody);
try {
jsonResponse = this.restClient.post()
.uri("/deidentify-image")
Expand Down
23 changes: 12 additions & 11 deletions src/main/java/org/karnak/backend/service/profilepipe/Profile.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@
*/
package org.karnak.backend.service.profilepipe;

import static org.karnak.backend.dicom.DefacingUtil.isAxial;
import static org.karnak.backend.dicom.DefacingUtil.isCT;

import java.awt.*;
import java.math.BigInteger;
import java.util.ArrayList;
Expand All @@ -22,6 +19,7 @@
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;

import lombok.extern.slf4j.Slf4j;
import org.dcm4che3.data.Attributes;
import org.dcm4che3.data.BulkData;
Expand Down Expand Up @@ -61,6 +59,9 @@
import org.weasis.core.util.StringUtil;
import org.weasis.dicom.param.AttributeEditorContext;

import static org.karnak.backend.dicom.DefacingUtil.isAxial;
import static org.karnak.backend.dicom.DefacingUtil.isCT;

@Slf4j
public class Profile {

Expand Down Expand Up @@ -271,7 +272,7 @@ public void applyCleanPixelData(Attributes dcmCopy, AttributeEditorContext conte
.findFirst()
.orElse(null);

if (cleanPixelDataItem != null && cleanPixelDataItem.isAutomaticMasksGeneration()) {
if (this.isAutomaticMasksGeneration(cleanPixelDataItem)) {
List<MaskArea> apiMasks = this.fetchMasksFromDeidentifyImageApi(dcmCopy, context.getTsuid());
if (!apiMasks.isEmpty()) {
// Use the first API mask as the "primary" mask (set on the context),
Expand All @@ -282,12 +283,8 @@ public void applyCleanPixelData(Attributes dcmCopy, AttributeEditorContext conte
}
}
}
if (mask == null && (cleanPixelDataItem == null || !cleanPixelDataItem.isAutomaticMasksGeneration())) {
// Manual mask should be applied only if automatic mask generation is not
// enabled
// If the API doesn't return any mask (because of an internal error,
// unreachable, etc...)
// then the data should not be sent
if (mask == null && !this.isAutomaticMasksGeneration(cleanPixelDataItem)) {
// Manual mask should be applied only if automatic mask generation is not enabled
mask = this.getMask(new MaskStationCondition(dcmCopy.getString(Tag.StationName),
dcmCopy.getString(Tag.Columns), dcmCopy.getString(Tag.Rows)));
}
Expand All @@ -297,7 +294,7 @@ public void applyCleanPixelData(Attributes dcmCopy, AttributeEditorContext conte
if (this.isCleanPixelAllowedDependingImageType(dcmCopy, sopClassUID)
&& this.evaluateConditionCleanPixelData(dcmCopy)) {
context.setMaskArea(mask);
if (mask == null) {
if (mask == null && !this.isAutomaticMasksGeneration(cleanPixelDataItem)) {
throw new IllegalStateException("Clean pixel is not applied: mask not defined in station name");
}
}
Expand All @@ -307,6 +304,10 @@ public void applyCleanPixelData(Attributes dcmCopy, AttributeEditorContext conte
}
}

private boolean isAutomaticMasksGeneration(@Nullable CleanPixelData cleanPixelDataItem) {
return (cleanPixelDataItem != null && cleanPixelDataItem.isAutomaticMasksGeneration());
}

private @Nullable MaskArea toMaskArea(MaskBody maskBody) {
Color color = StringUtil.hasText(maskBody.getColor()) ? ActionTags.hexadecimal2Color(maskBody.getColor())
: null;
Expand Down
5 changes: 2 additions & 3 deletions src/main/resources/application.yml
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,5 @@ dicom-tools:

# External de-identification image API
# This API checks if sensitive patient data is burned into DICOM pixel data.
karnak:
deidentify-image:
url: ${DEIDENTIFY_IMAGE_URL:http://localhost:8000}
ocr:
url: ${OCR_URL:http://localhost:8000}
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,20 @@
*/
package org.karnak.backend.service.profilepipe;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;

import java.awt.Rectangle;
import java.awt.*;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

import org.dcm4che3.data.Attributes;
import org.dcm4che3.data.Fragments;
import org.dcm4che3.data.Tag;
import org.dcm4che3.data.VR;
import org.dcm4che3.img.op.MaskArea;
import org.jspecify.annotations.Nullable;
import org.junit.jupiter.api.DisplayNameGeneration;
import org.junit.jupiter.api.DisplayNameGenerator;
import org.junit.jupiter.api.Test;
import org.karnak.backend.data.entity.ArgumentEntity;
import org.karnak.backend.data.entity.DestinationEntity;
Expand All @@ -33,10 +34,20 @@
import org.karnak.backend.data.entity.SecretEntity;
import org.karnak.backend.enums.DestinationType;
import org.karnak.backend.enums.PseudonymType;
import org.springframework.boot.test.context.SpringBootTest;
import org.karnak.backend.model.profilebody.MaskBody;
import org.mockito.Mockito;
import org.weasis.dicom.param.AttributeEditorContext;

import org.springframework.boot.test.context.SpringBootTest;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

@SpringBootTest
@DisplayNameGeneration(DisplayNameGenerator.ReplaceUnderscores.class)
class ProfileTest {

@Test
Expand Down Expand Up @@ -314,4 +325,76 @@ void cleanPixelData_imageTypeIsXA_forceCleanPixelByStationNumber() {

}

@Test
void automaticMasksGeneration_true_and_api_returns_no_masks_transmits_image_without_mask() {
ProfileEntity profileEntity = cleanPixelProfile("true");
Attributes dcm = secondaryCaptureImage();
AttributeEditorContext context = new AttributeEditorContext("1.2.840.10008.1.2.4.50", null, null);

DeidentifyImageService deidentifyImageService = Mockito.mock(DeidentifyImageService.class);
Mockito
.when(deidentifyImageService.callDeidentifyImageApi(Mockito.any(), Mockito.any(), Mockito.any()))
.thenReturn(Collections.<MaskBody>emptyList());
Profile profile = new Profile(profileEntity, deidentifyImageService);

// The API is trusted: no coordinates means no mask, but the image is still
// transmitted (no exception, no abort)
profile.applyCleanPixelData(dcm, context, profileEntity);

Mockito.verify(deidentifyImageService).callDeidentifyImageApi(Mockito.any(), Mockito.any(), Mockito.any());
assertNull(context.getMaskArea());
assertEquals("NONE", context.getAbort().name());
}

@Test
void automaticMasksGeneration_false_and_no_manual_mask_aborts_transmission() {
ProfileEntity profileEntity = cleanPixelProfile("false");
Attributes dcm = secondaryCaptureImage();
AttributeEditorContext context = new AttributeEditorContext("1.2.840.10008.1.2.4.50", null, null);
Profile profile = new Profile(profileEntity);

// No automatic generation and no manual mask defined: the transfer must be cut
assertThrows(IllegalStateException.class, () -> profile.applyCleanPixelData(dcm, context, profileEntity));
}

@Test
void automaticMasksGeneration_not_set_and_no_manual_mask_aborts_transmission() {
ProfileEntity profileEntity = cleanPixelProfile(null);
Attributes dcm = secondaryCaptureImage();
AttributeEditorContext context = new AttributeEditorContext("1.2.840.10008.1.2.4.50", null, null);
Profile profile = new Profile(profileEntity);

// automaticMasksGeneration not set behaves like false: the transfer must be cut
assertThrows(IllegalStateException.class, () -> profile.applyCleanPixelData(dcm, context, profileEntity));
}

private static ProfileEntity cleanPixelProfile(@Nullable String automaticMasksGenerationValue) {
ProfileEntity profileEntity = new ProfileEntity();
ProfileElementEntity cleanPixel = new ProfileElementEntity();
cleanPixel.setCodename("clean.pixel.data");
cleanPixel.setName("nameCleanPixel");
cleanPixel.setAction("ReplaceNull");
cleanPixel.setPosition(1);
if (automaticMasksGenerationValue != null) {
cleanPixel.addArgument(new ArgumentEntity("automaticMasksGeneration", automaticMasksGenerationValue,
cleanPixel));
}
Set<ProfileElementEntity> profileElementEntities = new HashSet<>();
profileElementEntities.add(cleanPixel);
profileEntity.setProfileElementEntities(profileElementEntities);
return profileEntity;
}

private static Attributes secondaryCaptureImage() {
Attributes dcm = new Attributes();
Fragments fragments = dcm.newFragments(Tag.PixelData, VR.OB, 2);
fragments.add(null);
fragments.add(new byte[] { 1, 2, 3, 4 });
// Secondary Capture SOP class: clean pixel data is always allowed
dcm.setString(Tag.SOPClassUID, VR.UI, "1.2.840.10008.5.1.4.1.1.7");
dcm.setString(Tag.SOPInstanceUID, VR.UI, "1.2.3.4.5");
dcm.setString(Tag.StationName, VR.SH, "UNKNOWN_STATION");
return dcm;
}

}