Motivation
Nott's stated goal is to be easier than raw LibTorch. Today, users with any custom tabular dataset (CSV or JSON) must implement their own parsing, tensor construction, and train/test splitting — the same boilerplate PyTorch users write every time. The infrastructure for this is already partially in place (Data::Type::CSV, CSVParameters, column selection), but no loader actually reads a file and produces tensors.
A universal loader removes the single biggest friction point for new users trying Nott on their own data.
Current State
src/data/load/types.hpp defines:
struct CSVParameters {
bool has_header = true;
char delimiter = ',';
std::vector<std::string> columns; // empty = all columns
};
struct CSV { std::string file; CSVParameters parameters{}; };
src/data/load/load.hpp has CSV parsing helpers (split_csv_line, trim_copy) but no public loader function that returns tensors. Users have no supported path to use these types.
Proposed Change
Add Load::Universal in src/data/load/load.hpp:
namespace Nott::Data::Load {
struct UniversalDescriptor {
std::variant<Data::Type::CSV /*, Data::Type::JSON */> source;
std::vector<std::string> input_columns; // empty = all except target_column
std::string target_column;
Data::Type::GlobalParameters split{}; // train/test fraction + shuffle
};
struct UniversalDataset {
torch::Tensor train_inputs;
torch::Tensor train_targets;
torch::Tensor test_inputs;
torch::Tensor test_targets;
};
UniversalDataset Universal(const UniversalDescriptor& descriptor);
} // namespace Nott::Data::Load
Implementation reuses the existing split_csv_line / trim_copy helpers and GlobalParameters splitting logic already used by MNIST/CIFAR loaders. JSON support can follow in a subsequent PR using the same descriptor pattern.
Usage target:
auto dataset = Nott::Data::Load::Universal({
.source = Nott::Data::Type::CSV{"my_data.csv"},
.input_columns = {"open", "high", "low", "volume"},
.target_column = "label",
.split = {.train_fraction = 0.8f}
});
model.train(dataset.train_inputs, dataset.train_targets, ...);
Acceptance Criteria
Motivation
Nott's stated goal is to be easier than raw LibTorch. Today, users with any custom tabular dataset (CSV or JSON) must implement their own parsing, tensor construction, and train/test splitting — the same boilerplate PyTorch users write every time. The infrastructure for this is already partially in place (
Data::Type::CSV,CSVParameters, column selection), but no loader actually reads a file and produces tensors.A universal loader removes the single biggest friction point for new users trying Nott on their own data.
Current State
src/data/load/types.hppdefines:src/data/load/load.hpphas CSV parsing helpers (split_csv_line,trim_copy) but no public loader function that returns tensors. Users have no supported path to use these types.Proposed Change
Add
Load::Universalinsrc/data/load/load.hpp:Implementation reuses the existing
split_csv_line/trim_copyhelpers andGlobalParameterssplitting logic already used by MNIST/CIFAR loaders. JSON support can follow in a subsequent PR using the same descriptor pattern.Usage target:
Acceptance Criteria
Load::Universalaccepts aData::Type::CSVdescriptor and returnsUniversalDatasetinput_columnsempty → all columns excepttarget_columnused as inputsGlobalParametersstd::invalid_argumentload.hppalready includes