Python: which lib to use for handling xattrs?

[pjotrek-b]

AFAIK there are currently 3 options mentioned to be usable for xattr-handling in python.
Listed here in order of historical appearance:

1. pyxattr lib
   (by Iustin Pop, 2002)
2. xattr lib
   (by Bob Ippolito, 2004)
3. built-in python.os library functions
   (since Python v3.3, Sept. 2012)

I'm writing down here what I found out about these, so if someone else (or me again) wants to know which one to use/prefer when and why.

Which one to use?

python.so methods:

Use for nice and quick linux-only (at the moment) code.

* They're built-in. so no extra dependency.
* Currently available on GNU/Linux only (according to official docs).
* Available since Python v3.3, 2012-09.

xattr:

• Legacy stable and widely used library.
• Available in Debian-based default repositories since 2006.
• Tested and developed primarily on/for MacOS.

pyxattr:

Very possibly superseded by xattr, since they both provides the same functionalities - yet xattr is the more popular choice, and more likely to cause less issues on MacOS.

[pjotrek-b]

More details about all of this:

pyxattr library

• Documentation: https://pyxattr.readthedocs.io/latest/
• PyPI: https://pypi.org/project/pyxattr/
• Download stats: https://pypistats.org/packages/pyxattr

(obsolete?)
Superseded by xattr.
Native to Debian, and available by apt since 2003.

quote from its developer (email, 2024-10):

"My package is in Debian since Sep 2003 (https://tracker.debian.org/news/264976/accepted-python-pyxattr-02-1-powerpc-all-source/), which means that at latest in 2003 pyxattr was published. Whereas if I read correctly on https://tracker.debian.org/pkg/python-xattr/news/?page=2, the first upload of python-xattr to Debian took place in September 2006, i.e. 3 years later."

xattr library

• Github: https://github.com/xattr/xattr
• PyPI: https://pypi.org/project/xattr/
• Maintainer's blog entry from 2005, on MacOS
• Download stats: https://pypistats.org/packages/xattr

Native to MacOS;
works on Linux too.
Available in Debian since 2006.

Here's a quote from an email dialogue with the original developers of xattr and pyxattr (2024-10-06):

xattr has been around for nearly two decades, is much more popular, and has
a less restrictive license. [...]

Built-in Python.OS xattr methods:

• Documentation: https://docs.python.org/3/library/os.html#linux-extended-attributes
• "mercs" - a PoC GUI (Qt5/6): https://github.com/pjotrek-b/mercs/

I'm currently using this library without issues (on Debian/Ubuntu, Python 3.8).

The official Python.OS documentation says about its xattrs-methods:

"These functions are all available on Linux only."

Some remarks on value-encoding differences between "xattr vs os xattr methods? (issue #121, github:xattr).

As long as the key/value size stays below 4k (tested on ext4, zfs, btrfs), all works flawlessly!
Values larger between 4k and 64k can be handled on ZFS natively.

[pjotrek-b]

About handling "larger" (>4kB) xattrs in practice

Here are some things I ran into, using the native Python.OS xattr methods to write larger xattr values.

• You get an input/output error with earlier samba versions when trying to write >4kB xattrs over smbd onto ZFS.
• Related Samba mailing list thread: "Reading >4kB xattrs (from ZFS) = input/output error" (February 2025)
  Mentions a patch around 2021, introducing smbd max xattr size, with 64kB as default value.
  Haven't tested it myself yet to confirm it works. But according to Samba developers, it should 😁 !