From a8b116aa3470cc10dc6ed066b2d374135ec94738 Mon Sep 17 00:00:00 2001 From: Dan Noble Date: Mon, 23 Mar 2026 17:31:03 -0700 Subject: [PATCH 1/2] readme updates --- pipeline/ingestion/README.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/pipeline/ingestion/README.md b/pipeline/ingestion/README.md index 1663fb3f..04d3b8f8 100644 --- a/pipeline/ingestion/README.md +++ b/pipeline/ingestion/README.md @@ -30,13 +30,19 @@ The pipeline is configured using `IngestionPipelineOptions`. Key options include ## Example Usage -To run the pipeline locally using the Direct runner: +First, ensure all dependencies are installed locally. After cloning the datacommons/import repository, run the following command from the root directory: + +```bash +mvn clean install +``` + +To run the pipeline locally using the Direct runner, cd to the `pipeline/ingestion` directory and run: ```bash mvn -Pdirect-runner compile exec:java \ -pl ingestion -am \ -Dexec.mainClass=org.datacommons.ingestion.pipeline.GraphIngestionPipeline \ - -Dexec.args="--project=YOUR_PROJECT_ID \ + -Dexec.args="--projectId=YOUR_PROJECT_ID \ --spannerInstanceId=YOUR_INSTANCE_ID \ --spannerDatabaseId=YOUR_DATABASE_ID \ --importList='[{\"importName\": \"Schema\", \"graphPath\": \"gs://path/to/schema/mcf/\"}, {\"importName\": \"SampleImport\", \"graphPath\": \"gs://path/to/data.tfrecord\"}]' \ @@ -49,7 +55,7 @@ To run the pipeline using the Dataflow runner: mvn -Pdataflow-runner compile exec:java \ -pl ingestion -am \ -Dexec.mainClass=org.datacommons.ingestion.pipeline.GraphIngestionPipeline \ - -Dexec.args="--project=YOUR_PROJECT_ID \ + -Dexec.args="--projectId=YOUR_PROJECT_ID \ --spannerInstanceId=YOUR_INSTANCE_ID \ --spannerDatabaseId=YOUR_DATABASE_ID \ --importList='[{\"importName\": \"Schema\", \"graphPath\": \"gs://path/to/schema/mcf/\"}, {\"importName\": \"SampleImport\", \"graphPath\": \"gs://path/to/data.tfrecord\"}]' \ From 55997881f8df2c353981b30e6f03f1dc6e93cc7e Mon Sep 17 00:00:00 2001 From: Dan Noble Date: Mon, 23 Mar 2026 17:34:16 -0700 Subject: [PATCH 2/2] updated readme --- pipeline/ingestion/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/ingestion/README.md b/pipeline/ingestion/README.md index 04d3b8f8..7b7ec095 100644 --- a/pipeline/ingestion/README.md +++ b/pipeline/ingestion/README.md @@ -30,7 +30,7 @@ The pipeline is configured using `IngestionPipelineOptions`. Key options include ## Example Usage -First, ensure all dependencies are installed locally. After cloning the datacommons/import repository, run the following command from the root directory: +First, ensure all dependencies are installed locally. After cloning the `datacommons/import` repository, run the following command from the project's root directory: ```bash mvn clean install