Add NTFS USN change journal artifact

[m1435]

From Microsoft Docs:

[T]he NTFS file system maintains an update sequence number (USN) change journal. When any change is made to a file or directory in a volume, the USN change journal for that volume is updated with a description of the change and the name of the file or directory.

The USN change journal is a useful artifact to track the creation, modification, and deletion of files in the file system. It is, for example, supported by Plaso with the usnjrnl parser.

Other than for $MFT and $Logfile, there is no predefined artifact for the corresponding \$Extend\$UsnJrnl file that contains the change journal.

I suggest adding something along the following to https://github.com/ForensicArtifacts/artifacts/blob/main/data/ntfs.yaml:

name: NTFSUsnJrnlFiles
doc: |
  The NTFS $UsnJrnl:$J and $UsnJrnl:$Max file system metadata files.
sources:
- type: FILE
  attributes:
    paths:
    - '%%environ_systemdrive%%\$Extend\$UsnJrnl'
    separator: '\'
labels: [System]
supported_os: [Windows]

I've tested this as a custom artifact definition in Plaso/log2timeline with the usnjrnl parser, and it appears to be working.

Potentially related issue: #431

[joachimmetz]

Blocked on #13 since technically the $J ADS is what contains the data of interest not the file entry $UsnJrnl

[joachimmetz]

I've tested this as a custom artifact definition in Plaso/log2timeline with the usnjrnl parser, and it appears to be working.

FYI artifact definitions and parsers are not connected in Plaso, so I'm not sure what you tested or intended to test

[m1435]

FYI artifact definitions and parsers are not connected in Plaso, so I'm not sure what you tested or intended to test

What I did is basically the following (image.e01 is the NTFS image, result.plaso is the plaso file, and usnjrnl.yaml contains the artifact definition as given above):

$ podman run --rm -it -v "$PWD:/data"  docker.io/log2timeline/plaso log2timeline.py \
    --partitions=3 --parsers usnjrnl --artifact-filters NTFSUsnJrnlFiles \
    --custom_artifact_definitions /data/usnjrnl.yaml --storage_file /data/result.plaso \
    /data/image.e01

$ podman run --rm -it -v "$PWD:/data"  docker.io/log2timeline/plaso pinfo /data/result.plaso 
[...]
******************************** Event sources *********************************
Total : 1
--------------------------------------------------------------------------------

************************* Events generated per parser **************************
Parser (plugin) name : Number of events
--------------------------------------------------------------------------------
             usnjrnl : 215392
               Total : 215392
--------------------------------------------------------------------------------
[...]

At least Plaso appears to implicitly consider the ADS of $UsnJrnl when interpreting the custom NTFSUsnJrnlFiles artifact.

[joachimmetz]

At least Plaso appears to implicitly consider the ADS of $UsnJrnl when interpreting the custom NTFSUsnJrnlFiles artifact.

You misunderstand the inner workings of Plaso.

Artifacts in Plaso are used as collection filters: https://plaso.readthedocs.io/en/latest/sources/user/Collection-Filters.html#using-forensic-artifacts-definitions

Your filter tells Plaso to only process a file with path %%environ_systemdrive%%\$Extend\$UsnJrnl

If you specify %%environ_systemdrive%%\$Extend\* it would have similar results.

Internally Plaso looks for the $J ADS since $UsnJrnl is a known metadata file. If the $UsnJrnl would be stored in a different location, nothing would be extracted.

[m1435]

Your filter tells Plaso to only process a file with path %%environ_systemdrive%%\$Extend\$UsnJrnl

That is exactly what I wanted to achieve. I mean, it is basically the same for other artifacts aswell. Take for example the WindowsEventLogs artfiact. When I use it in conjuction with the winevtx parser, I want Plaso to process all the "usual" Windows Event Log files. I could still place some other file with an *.evtx extension (e.g. a renamed PNG file) in that directory and it would be matched by the artifact and only be rejected by the parser (please correct me if I am mistaken here).

If the $UsnJrnl would be stored in a different location, nothing would be extracted.

But that that would also be true when using for example the WindowsEventLogs artifact in case there is an EVTX file stored in some other directory outside %%environ_systemroot%%\System32\winevt\Logs\*.evtx.

I can see the point that relying on the artifact consumer to handle ADS correctly (or rather in a certain way) is not an ideal solution, though. But for my specific usecase, it would be sufficient.

[joachimmetz]

Yes but that is Plaso specific behavior, not related to reusable artifact definitions (which this project is)

[joachimmetz]

But that that would also be true when using for example the WindowsEventLogs artifact in case there is an EVTX file stored in some other directory outside %%environ_systemroot%%\System32\winevt\Logs*.evtx.

no if you define a filter file (outside the normal evtx file location) and the winevt parser that evtx file will be parsed. This is not the case for $UsnJrnl.

Again these are Plaso related questions.

[m1435]

no if you define a filter file (outside the normal evtx file location) and the winevt parser that evtx file will be parsed. This is not the case for $UsnJrnl.

This is the piece of information I was missing. Thanks for explaining it again!