Feature Request: Round-trip Julia structs via StructUtils.jl

[stephenberry]

MAT.jl writes arbitrary Julia structs to .mat files but always reads them back as Dict{String, Any}. JSON.jl solves this by letting users provide a target type on read, powered by StructUtils.jl. MAT.jl could do the same.

Example

struct Person
    name::String
    age::Int
end

# Write works today
matwrite("data.mat", Dict("p" => Person("Alice", 30)))

# Today: returns Dict("name" => "Alice", "age" => 30)
# Proposed: returns Person("Alice", 30)
person = matread("data.mat", "p", Person)

This would also support nested structs, default values (@defaults), field renaming (@tags), Union{T, Nothing} fields, and abstract type dispatch via @choosetype (all handled by StructUtils.jl).

Abstract type dispatch with @choosetype

abstract type Animal end
struct Dog <: Animal
    name::String
    breed::String
end
struct Cat <: Animal
    name::String
    indoor::Bool
end

MAT.@choosetype Animal x -> haskey(x, "breed") ? Dog : Cat

struct Household
    owner::String
    pet::Animal
end

household = matread("data.mat", "h", Household)
# Household("Alice", Dog("Rex", "Labrador"))

Implementation

Since MAT.jl already reads structs as Dict{String, Any}, the typed read is a thin wrapper using StructUtils.constructfrom:

function matread(filename, varname, ::Type{T}) where T
    dict = matread(filename)
    return StructUtils.constructfrom(T, dict[varname])
end

PR

Would you be interested in a pull request that adds this support?

[matthijscox]

Thanks for this suggestion. I discussed this kind of idea here as well: #174, but didn't really have much time for it yet, and I haven't reviewed the new changes to JSON.jl/StructTypes.jl interface in depth. Though I am familiar with the previous JSON3 interface and StructUtils.jl

I did suggest to make it "opt-in", so people have to explicitly define that their types are StructTypes compatible via the StructType trait. Not sure what your thoughts are on that?

I guess when reading, you already opt-in by providing the desired type, as you propose in your example matread("data.mat", "h", Household). But we'd also have to consider how to write types.

[stephenberry]

Yeah, reading is opt-in, so the user just provides the target type. No trait declaration needed:

person = matread("data.mat", "p", Person)

If you don't provide a type, you'll get the current Dict{String, Any} behavior.

For writing, JSON.jl doesn't require StructUtils.dictlike or any opt-in trait. JSON.jl uses StructUtils.structlike, which returns true automatically for any struct (via isstructtype(T)). So JSON.json(Person("Alice", 30)) just works.

MAT.jl writes arbitrary structs via the CompositeKind path, so requiring an opt-in trait for writing would be a regression. It's probably better to match the JSON.jl approach, where any struct just works, via StructUtils.applyeach. This also gives @tags (field renaming/ignoring) and lower (custom serialization) for free.

This would make reads and writes both go through the same StructUtils machinery, and users get the same customization hooks (@tags, @defaults, lower) in both directions.

Proposed API

# Reading: provide a target type
person = matread("data.mat", "p", Person)

# Writing: same syntax, powered by StructUtils
matwrite("data.mat", Dict("p" => Person("Alice", 30)))

[matthijscox]

Okay, I suppose this could work.

For best performance I suppose you'd actually want to convert immediately when reading/writing from the HDF5 types? So avoid the Dicts entirely. I assume that's what JSON.jl does as well? But that's some serious changes deep inside MAT.jl.

On the other hand, your thin wrapper would add a bunch of performance overhead I guess? Converting (nested) Dicts to (nested) types in the read, and convert types to Dicts in the write. And as you show it's actually very low effort for a user to do it themselves whenever they want to. Do we really want to add yet another dependency for this wrapper layer, so potentially somewhat longer MAT.jl precompilation times?

[stephenberry]

Good questions. I expect StructUtils to become a common library in Julia that provides a lot of API and performance benefits, so I don't think precompilation times for it should be a concern, especially because it will reduce MAT.jl code and use a common dependency. This means that using StructUtils will ultimately reduce precompilation times by not having custom implementations of the same logic across libraries. So, larger codebases with lots of dependencies where precompilation time matters will benefit.

Your notes about going through Dicts and this costing a lot in performance is valid. But, this is more about getting a common API and reflection codebase and performance optimizations can code later under the hood. JSON.jl has decent performance and avoids these issues, so MAT.jl should be able to do the same.

[matthijscox]

I'm kinda curious where this might lead, so feel free to experiment and open a PR for review.

I remain skeptical because I would never advise to use .mat files as a Julia data serialisation format :)

Which reminds me that we introduced some special types (other than dicts) to deal with all the MATLAB types, like struct arrays and classes. They might need dedicated StructUtils conversions as well? So things might be a tiny bit more difficult than a thin wrapper.

[stephenberry]

Cool, I'll work on a PR and we can iterate from there.