Obstor Go Client SDK for Amazon S3 Compatible Cloud Storage

The Obstor Go Client SDK provides straightforward APIs to access any Amazon S3 compatible object storage.

This Quickstart Guide covers how to install the Obstor client SDK, connect to Obstor, and create a sample file uploader. For a complete list of APIs and examples, see the godoc documentation or Go Client API Reference.

These examples presume a working Go development environment. To browse buckets and objects from the command line you can use rclone configured with an obstor: remote, or the Obstor web console.

Download from Github

From your project directory:

go get github.com/obstor/obstor-go/v7

Initialize a Obstor Client Object

The Obstor client requires the following parameters to connect to an Amazon S3 compatible object storage:

Parameter	Description
endpoint	URL to object storage service.
_obstor.Options_	All the options such as credentials, custom transport etc.

package main

import (
	"log"

	"github.com/obstor/obstor-go/v7"
	"github.com/obstor/obstor-go/v7/pkg/credentials"
)

func main() {
	endpoint := "demo.obstor.net"
	accessKeyID := "Q3AM3UQ867SPQQA43P2F"
	secretAccessKey := "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
	useSSL := true

	// Initialize obstor client object.
	obstorClient, err := obstor.New(endpoint, &obstor.Options{
		Creds:  credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
		Secure: useSSL,
	})
	if err != nil {
		log.Fatalln(err)
	}

	log.Printf("%#v\n", obstorClient) // obstorClient is now set up
}

Example - File Uploader

This sample code connects to an object storage server, creates a bucket, and uploads a file to the bucket. It uses the Obstor demo server, a public Obstor cluster located at https://demo.obstor.net.

The demo server runs the latest stable version of Obstor and may be used for testing and development. The access credentials shown in this example are open to the public and all data uploaded to demo should be considered public and non-protected.

FileUploader.go

This example does the following:

• Connects to the Obstor demo server using the provided credentials.

• Creates a bucket named testbucket.

• Uploads a file named testdata from /tmp.

• Verifies the file was created using rclone ls.

  // FileUploader.go Obstor example
  package main
  
  import (
  	"context"
  	"log"
  
  	"github.com/obstor/obstor-go/v7"
  	"github.com/obstor/obstor-go/v7/pkg/credentials"
  )
  
  func main() {
  	ctx := context.Background()
  	endpoint := "demo.obstor.net"
  	accessKeyID := "Q3AM3UQ867SPQQA43P2F"
  	secretAccessKey := "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
  	useSSL := true
  
  	// Initialize obstor client object.
  	obstorClient, err := obstor.New(endpoint, &obstor.Options{
  		Creds:  credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
  		Secure: useSSL,
  	})
  	if err != nil {
  		log.Fatalln(err)
  	}
  
  	// Make a new bucket called testbucket.
  	bucketName := "testbucket"
  	location := "us-east-1"
  
  	err = obstorClient.MakeBucket(ctx, bucketName, obstor.MakeBucketOptions{Region: location})
  	if err != nil {
  		// Check to see if we already own this bucket (which happens if you run this twice)
  		exists, errBucketExists := obstorClient.BucketExists(ctx, bucketName)
  		if errBucketExists == nil && exists {
  			log.Printf("We already own %s\n", bucketName)
  		} else {
  			log.Fatalln(err)
  		}
  	} else {
  		log.Printf("Successfully created %s\n", bucketName)
  	}
  
  	// Upload the test file
  	// Change the value of filePath if the file is in another location
  	objectName := "testdata"
  	filePath := "/tmp/testdata"
  	contentType := "application/octet-stream"
  
  	// Upload the test file with FPutObject
  	info, err := obstorClient.FPutObject(ctx, bucketName, objectName, filePath, obstor.PutObjectOptions{ContentType: contentType})
  	if err != nil {
  		log.Fatalln(err)
  	}
  
  	log.Printf("Successfully uploaded %s of size %d\n", objectName, info.Size)
  }

1. Create a test file containing data:

You can do this with dd on Linux or macOS systems:

dd if=/dev/urandom of=/tmp/testdata bs=2048 count=10

or fsutil on Windows:

fsutil file createnew "C:\Users\<username>\Desktop\sample.txt" 20480

2. Run FileUploader with the following commands:

go mod init example/FileUploader
go get github.com/obstor/obstor-go/v7
go get github.com/obstor/obstor-go/v7/pkg/credentials
go run FileUploader.go

The output resembles the following:

2023/11/01 14:27:55 Successfully created testbucket
2023/11/01 14:27:55 Successfully uploaded testdata of size 20480

3. Verify the Uploaded File With rclone ls:

This assumes an obstor: remote configured against your server (see the rclone S3 docs).

rclone ls obstor:testbucket
    20480 testdata

API Reference

The full API Reference is available here.

• Complete API Reference

API Reference : Bucket Operations

• MakeBucket
• ListBuckets
• BucketExists
• RemoveBucket
• ListObjects
• ListIncompleteUploads

API Reference : Bucket policy Operations

• SetBucketPolicy
• GetBucketPolicy

API Reference : Bucket notification Operations

• SetBucketNotification
• GetBucketNotification
• RemoveAllBucketNotification
• ListenBucketNotification (Obstor Extension)
• ListenNotification (Obstor Extension)

API Reference : File Object Operations

• FPutObject
• FGetObject

API Reference : Object Operations

• GetObject
• PutObject
• PutObjectStreaming
• StatObject
• CopyObject
• RemoveObject
• RemoveObjects
• RemoveIncompleteUpload
• SelectObjectContent

API Reference : Presigned Operations

• PresignedGetObject
• PresignedPutObject
• PresignedHeadObject
• PresignedPostPolicy

API Reference : Client custom settings

• SetAppInfo
• TraceOn
• TraceOff

Full Examples

Full Examples : Bucket Operations

• makebucket.go
• listbuckets.go
• bucketexists.go
• removebucket.go
• listobjects.go
• listobjectsV2.go
• listincompleteuploads.go

Full Examples : Bucket policy Operations

• setbucketpolicy.go
• getbucketpolicy.go
• listbucketpolicies.go

Full Examples : Bucket lifecycle Operations

• setbucketlifecycle.go
• getbucketlifecycle.go

Full Examples : Bucket encryption Operations

• setbucketencryption.go
• getbucketencryption.go
• removebucketencryption.go

Full Examples : Bucket replication Operations

• setbucketreplication.go
• getbucketreplication.go
• removebucketreplication.go

Full Examples : Bucket notification Operations

• setbucketnotification.go
• getbucketnotification.go
• removeallbucketnotification.go
• listenbucketnotification.go (Obstor Extension)
• listennotification.go (Obstor Extension)

Full Examples : File Object Operations

• fputobject.go
• fgetobject.go

Full Examples : Object Operations

• putobject.go
• getobject.go
• statobject.go
• copyobject.go
• removeobject.go
• removeincompleteupload.go
• removeobjects.go

Full Examples : Encrypted Object Operations

• put-encrypted-object.go
• get-encrypted-object.go
• fput-encrypted-object.go

Full Examples : Presigned Operations

• presignedgetobject.go
• presignedputobject.go
• presignedheadobject.go
• presignedpostpolicy.go

Explore Further

• Godoc Documentation
• Complete Documentation
• Obstor Go Client SDK API Reference

Contribute

Contributors Guide

License

This SDK is distributed under the Apache License, Version 2.0, see LICENSE and NOTICE for more information.