Update Pardot Data in OpenSearch

This Go application fetches data from the Pardot API and updates an OpenSearch index with the retrieved data. It supports pagination based on either a timestamp field or an ID field, allowing for efficient data retrieval and indexing.

Build

To build the Go application, run the following command in the terminal:

CGO_ENABLED=0 go build -a -installsuffix cgo -o update_pardot_in_os .

This will create an executable named update_pardot_in_os in the current directory.

Configuration

The application requires a configuration file in JSON format. Below is an example configuration:

{
    "fields": "id,email,firstName,username,jobTitle,role,roleName,salesforceId,isDeleted,createdAt,updatedAt,createdById,updatedById,tagReplacementLanguage",
    "base_url": "https://pi.pardot.com/api/v5/objects/users",
    "index_name": "prdt_users",
    "opensearch_host": "https://localhost:9200",
    "opensearch_user": "username",
    "opensearch_password": "password",
    "salesforce_username": "user@name.org",
    "salesforce_password": "password",
    "client_id": "id",
    "client_secret": "secret",
    "business_unit_id": "id",
    "timestamp_field": "createdAt"
}

Usage

To run the application, use the following command:

./update_pardot_in_os -f users.json