Hello,
When I try to use DataConfig to load my dataset from S3, like so...
DataConfig(
dataset_name='my-dataset',
dataset_uri='s3://my-bucket/some/file.jsonl'
# rest of config...
)
I get the following error:
ArrowInvalid: Expected a local filesystem path, got a URI: 's3://my-bucket/some/file.jsonl'
The documentation states that
dataset_uri: either a local path or s3 URI representing where the dataset is stored
But it seems like this is not the case.
Hello,
When I try to use DataConfig to load my dataset from S3, like so...
I get the following error:
The documentation states that
But it seems like this is not the case.