Skip to content

Better tests and documentation#7

Open
maxdudek wants to merge 12 commits into
ubccr:masterfrom
maxdudek:tests
Open

Better tests and documentation#7
maxdudek wants to merge 12 commits into
ubccr:masterfrom
maxdudek:tests

Conversation

@maxdudek
Copy link
Copy Markdown
Contributor

  • Rewrote script which generates test data to be easily configurable
  • Added documentation for replacement rules config file
  • Memory is now converted from MB to GB in the supremm transform step, meaning the ingest step no longer has to worry about it.

Comment thread configuration/etl/etl.d/hardware.json Outdated
"cpu_id": "ct.id",
"core_count": "s.core_count",
"memory_gb": "CEILING(s.physmem / 1024.0)",
"memory_gb": "s.physmem",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please can you confirm that this change is intentional. Where does the memory value normalization occur now?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the memory gets converted in the 'transform' step of the SUPReMM archive reader (get_hardware_info.py), although it (temporarily) is also taken care of by the SUPReMM patching script, so that data extracted before this change also gets converted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants