Fresh Darktable creates and shows internal tags (darktable|*) after import but not after restart and further imports afterwards

[lazypingu]

Is there an existing issue for this?

• I checked and did not find my issue in the already reported ones

Describe the bug

If I import images on a fresh Darktable (no config, no database, no library), I see that some internal Dartable tags were created.

In the logs I see:

[xmp_import] creating tag: darktable|exported
[xmp_import] creating tag: darktable|format|nef
[xmp_import] creating tag: darktable|changed

This reflects the imported images as some of them have xmp files and were already exported.

These tags are also visible in the lightroom view. As a tag symbol on every picture and in the tagging section (independently of the hidden tag toggle).

The internal tags will also be added to the sidecar xmp files (I guess this is expected).

If I restart Darktable, the internal tags are no longer visible in the lightroom view anymore.

If I then e.g. remove the imported collection and re-import the images, the tags are not shown or added to the xmp files.

This seems to only happens for me on a clean install and only on the first import.

If "after edit" was selected on storage this will lead to xmp files for every imported image (due to the internal tags). The second import would then remove the xmp files again if "after edit" is still in place.

Steps to reproduce

1. Start a fresh Darktable instance (no config, no database, no library)
2. Import images
3. Check if internal tags are visible in lightroom view
4. Restart Darktable
5. Check if internal tags are visible in the lightroom view

Execute the same steps again but change xmp storage to "after edit" and check the generated xmp files.

Expected behavior

I'm not sure if this is an intended behavior or if I'm doing something wrong here. I wanted to use "after edit" to avoid having xmp files for not edited images. But with this happening there are generated due to the created internal tags.

Logfile | Screenshot | Screencast

No response

Commit

No response

Where did you obtain darktable from?

distro packaging

darktable version

5.6.0 (also tested with darktable 5.7.0+219~g4a8a1cfd90 [linux])

What OS are you using?

Linux

What is the version of your OS?

Gentoo

Describe your system

No response

Are you using OpenCL GPU in darktable?

None

If yes, what is the GPU card and driver?

No response

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

No response

[gi-man]

Do you have any auto preset?

[lazypingu]

Do you have any auto preset?

Not that I'm aware of. Should be a vanilla setup.

[lazypingu]

As I mentioned some of the imported images have xmp files and were already exported.
I now tried to import the same images but without any corresponding xmp files.
Now the internal tags are not shown, except if I explicitly toggle them in the tagging section.
Also the "after edit" option now works as expected and do not create the sidecar files.

It looks like the existing xmp files contain the hidden tags as well.
Not sure but this might affect the way Darktable is processing the images and the internal tags are treated as regular tags?

[wpferguson]

darktable uses the hidden tags in the xmp files to maintain information that may be required when rebuilding the database from the xmp files. The default setting is that they remain hidden in the tagging UI since they are meant more for darktable than the user.

[lazypingu]

Here are two screenshots which shows the visible internal tags after import on a fresh Darktable instance:

Import of images with existing xmp files

Import of images without existing xmp files

[wpferguson]

Started darktable with a clean config and imported.

Made darktable tags visible

Hovered over image and tags were displayed

Restarted darktable

Hovered over images and tags where displayed

Removed images

Changed XMP files to after edit

Reimported the removed set of images and tags were visible

Imported another set of images

Tags were visible

Checked and XMP files were not created.

---

Everything appears to be working the way it should. Tested on darktable 5.7.0+333~g167032086d

[lazypingu]

Thanks @wpferguson for testing.

On my xmp files there is a section with the internal tags, e.g.:

   <dc:subject>
    <rdf:Bag>
     <rdf:li>darktable</rdf:li>
     <rdf:li>exported</rdf:li>
     <rdf:li>format</rdf:li>
     <rdf:li>nef</rdf:li>
    </rdf:Bag>
   </dc:subject>
   <lr:hierarchicalSubject>
    <rdf:Bag>
     <rdf:li>darktable|exported</rdf:li>
     <rdf:li>darktable|format|nef</rdf:li>
    </rdf:Bag>
   </lr:hierarchicalSubject>

I guess this might be expected to store the internal tags e.g. for later restore.

If I import an image with a xmp file which contains these internal tags, I see the internal tags in the lightroom UI without toggling the switch to show hidden tags.

Darktable also logs this on import:

    78.0600 [xmp_import] creating tag: darktable|exported
    78.0651 [xmp_import] creating tag: darktable|format|nef

If I remove the section with the internal tags mentioned above from the xmp file and test again with a fresh Darktable instance, all works fine the intenal tags are not visible and I do not get the log mentioned.

Maybe my xmp files are somehow corrupted?

[zisoft]

@lazypingu @wpferguson : I see the problem.

To reproduce:

• start darktable with a clean config (no database, no library, no config)
• import images with existing xmp files from a previous darktable session , so the xmp files contain darktable|... tags
• see the darktable tags appearing in the tagging module, in both lists

The toggle button to hide internal tags doesn't change anything, the tags cannot be hidden.

• quit and restart darktable
• now the tags display is back to normal as it should be

I guess this happens again when importing images which contain darktable| tags which do not exist in the database.

[lazypingu]

Thanks @zisoft for testing and pointing out to also clean the library and the database. I've edited my descriptions accordingly.