Parallelisation of Data Writing and Feature subsetting#25

Merged

Saiyam26 merged 16 commits intomainfrom

parallel_feature_subset_training

Jan 16, 2025

Commits on Oct 24, 2024

parallel chunkwise writing using num_workers param from config -- not working due to event logging in pre-processing
Saiyam26
committed

Commits on Oct 25, 2024

Commits on Nov 7, 2024

Commits on Nov 8, 2024

solved bug for num_workers not working with None
Saiyam26
committed

Commits on Nov 25, 2024

parallel feature subset training; rudimentary
Saiyam26
committed

Commits on Nov 28, 2024

removed erroneous validation by train set in feature_subset_training + removed print statement
Saiyam26
committed

Commits on Dec 5, 2024

Setting seed value every time before model training + handling of empy dataset being written during parallel write
Saiyam26
committed

Commits on Dec 9, 2024

solved error of missing few samples due to integer division of parallel worker_chunksize
Saiyam26
committed

Commits on Dec 30, 2024

fixed error in parallelization not working for a dataset, andata not correctly loading into memory
Saiyam26
committed

Commits on Jan 6, 2025

Commits on Jan 16, 2025