HDF5 Metadata Issue

[jacobseiler]

I ran into this issue previously. In essence, we save the Number of Halos per tree in a header attribute. However, for large simulations the number of trees is extraordinarily large, the datasize of the attribute exceeds the 64kb limit.

Possible solutions are:

1. Write NHalosPerTree as a data object instead of an attribute (i.e., make it a group that can be accessed via a key). Will need to investigate the reprecussions of this, the least of which would be to adjust the SAGE HDF5 reader.

2. Write to more files so a single file doesn't exceed 64kb. This isn't scalable so probably not the best solution...

For now, I'm just going to write only to a binary file.

[manodeep]

Are the changes to be made in these lines? Definitely should be in a new dataset, but I would also recommend changing datatype to int64_t (or equivalently check what the Illustris folks use)

[jacobseiler]

Yes I think these are the lines that should be changed. It will write to a dataset (called Header perhaps) rather than an attribute.

Looking at it properly, I'm already creating the group as Header. I should be writing it as data though.