fileuploader

fileuploader is a Go SDK for uploading local files to a remote server over SSH/SFTP. It also provides a separate remote container restart helper. File synchronization and container restart are independent operations: Sync() only uploads files, and a container is restarted only when the caller explicitly invokes RestartContainer(...).

Features

• Full directory replacement upload.
• Incremental upload for new or modified files.
• Concurrent worker pool uploads.
• Progress bar and structured logging support.
• Atomic file replacement with temporary files and rename.
• Backup and rollback during full sync.
• Explicit remote container restart helper.
• Parameter-only API. The SDK does not load YAML, JSON, env files, or any other config file by itself.

Requirements

• Go 1.19 or later.
• SSH access to the remote host.
• SFTP enabled on the remote host.
• Docker or Docker Compose on the remote host only if you call the restart helper with Docker commands.

Install

For the latest released version:

go get github.com/lpg-it/fileuploader@v1.0.2

For local development inside this repository:

go test ./...

Import

import "github.com/lpg-it/fileuploader/syncer"

Quick Start: Upload Only

Use New(...) when you only need SFTP synchronization and do not need remote command execution.

package main

import (
    "log"

    "github.com/lpg-it/fileuploader/syncer"
    "github.com/sirupsen/logrus"
)

func main() {
    logger := logrus.New()

    sshClient, sftpClient, err := syncer.ConnectSSH(
        "your-server-host.com",
        22,
        "your-username",
        "your-password",
    )
    if err != nil {
        log.Fatal(err)
    }
    defer sshClient.Close()
    defer sftpClient.Close()

    fileSyncer := syncer.New(sftpClient, syncer.SyncConfig{
        LocalPath:  "/path/to/local/directory",
        RemotePath: "/path/to/remote/directory",
        Mode:       syncer.SyncModeIncremental,
        Workers:    10,
    }, logger)

    if err := fileSyncer.Sync(); err != nil {
        log.Fatal(err)
    }
}

Quick Start: Upload, Then Maybe Restart

Use NewWithSSHClient(...) when your application may later call remote commands. Restart remains fully caller-controlled.

package main

import (
    "log"

    "github.com/lpg-it/fileuploader/syncer"
    "github.com/sirupsen/logrus"
)

func main() {
    logger := logrus.New()

    sshClient, sftpClient, err := syncer.ConnectSSH(
        "your-server-host.com",
        22,
        "your-username",
        "your-password",
    )
    if err != nil {
        log.Fatal(err)
    }
    defer sshClient.Close()
    defer sftpClient.Close()

    fileSyncer := syncer.NewWithSSHClient(sshClient, sftpClient, syncer.SyncConfig{
        LocalPath:  "/path/to/local/directory",
        RemotePath: "/path/to/remote/directory",
        Mode:       syncer.SyncModeFull,
        Workers:    10,
    }, logger)

    if err := fileSyncer.Sync(); err != nil {
        log.Fatal(err)
    }

    shouldRestart := true
    if shouldRestart {
        if err := fileSyncer.RestartContainer(syncer.ContainerRestartConfig{
            ContainerName: "app",
        }); err != nil {
            log.Fatal(err)
        }
    }
}

API Reference

ConnectSSH

func ConnectSSH(host string, port int, user, password string) (*ssh.Client, *sftp.Client, error)

Creates an SSH client and an SFTP client using password authentication.

The helper uses ssh.InsecureIgnoreHostKey() for convenience. If you need SSH key authentication, host key verification, bastion routing, or any custom SSH behavior, create your own *ssh.Client and *sftp.Client, then pass them to New(...), NewWithSSHClient(...), or RestartContainer(...).

SyncConfig

type SyncConfig struct {
    LocalPath  string
    RemotePath string
    Mode       string
    Workers    int
}

• LocalPath: local directory root to upload from. The SDK walks this directory recursively and uploads its contents.
• RemotePath: remote directory target. Remote paths use POSIX-style separators.
• Mode: use syncer.SyncModeFull or syncer.SyncModeIncremental.
• Workers: number of concurrent upload workers. Values less than or equal to zero are treated as 1.

Sync Modes

syncer.SyncModeFull

Full sync replaces the remote directory with the local directory:

1. Collects all local files.
2. Uploads them into a temporary remote directory.
3. Renames the current remote directory to a timestamped backup if it exists.
4. Renames the temporary directory to the target path.
5. Removes the backup after success.
6. Restores from backup if the final rename fails.

syncer.SyncModeIncremental

Incremental sync preserves existing remote files and uploads only necessary local entries:

1. Ensures the remote root exists.
2. Creates missing remote directories.
3. Uploads files that do not exist remotely.
4. Uploads files whose size differs.
5. Uploads files whose local modification time is newer than the remote file.
6. Leaves remote-only files untouched.

Constructors

func New(client *sftp.Client, syncConfig SyncConfig, logger *logrus.Logger) *Syncer

Use this when the caller only needs synchronization.

logger may be nil; in that case the SDK creates a silent logger.

func NewWithSSHClient(sshClient *ssh.Client, client *sftp.Client, syncConfig SyncConfig, logger *logrus.Logger) *Syncer

Use this when the caller wants to keep the option to run explicit remote commands later, such as restarting a container.

func (s *Syncer) SetSSHClient(sshClient *ssh.Client) *Syncer

Attach an SSH client to an existing syncer. This is useful if you created the syncer with New(...) and later decided to enable restart capability.

Synchronization

func (s *Syncer) Sync() error

Runs the configured upload mode. Sync() never restarts containers and never runs arbitrary remote commands.

Container Restart

type ContainerRestartConfig struct {
    ContainerName string
    Command       string
}

Restart by Docker container name:

err := fileSyncer.RestartContainer(syncer.ContainerRestartConfig{
    ContainerName: "app",
})

This runs the following command on the remote host:

docker restart 'app'

Use a custom command when you need Docker Compose, sudo, a script, or any other remote restart behavior:

err := fileSyncer.RestartContainer(syncer.ContainerRestartConfig{
    Command: "docker compose restart api",
})

You can also call the package-level helper directly:

err := syncer.RestartContainer(sshClient, syncer.ContainerRestartConfig{
    ContainerName: "app",
}, logger)

Command takes precedence over ContainerName. If both are empty, the SDK returns an error. Restart errors are returned separately from upload errors because upload and restart are separate caller decisions.

Custom SSH Client Example

Use your own SSH client when password authentication or insecure host key handling is not acceptable for your environment.

sshClient, err := ssh.Dial("tcp", "your-server-host.com:22", &ssh.ClientConfig{
    User: "your-username",
    Auth: []ssh.AuthMethod{
        ssh.PublicKeys(signer),
    },
    HostKeyCallback: knownHostsCallback,
})
if err != nil {
    log.Fatal(err)
}
defer sshClient.Close()

sftpClient, err := sftp.NewClient(sshClient)
if err != nil {
    log.Fatal(err)
}
defer sftpClient.Close()

fileSyncer := syncer.NewWithSSHClient(sshClient, sftpClient, syncer.SyncConfig{
    LocalPath:  "/path/to/local",
    RemotePath: "/path/to/remote",
    Mode:       syncer.SyncModeIncremental,
    Workers:    10,
}, logger)

Error Handling Pattern

Keep upload and restart failures separate in your application logic:

if err := fileSyncer.Sync(); err != nil {
    return err
}

if shouldRestart {
    if err := fileSyncer.RestartContainer(syncer.ContainerRestartConfig{
        Command: "docker compose restart api",
    }); err != nil {
        return err
    }
}

Operational Notes

• Always close both the SSH client and SFTP client when you create them.
• Use New(...) for upload-only integrations.
• Use NewWithSSHClient(...) or SetSSHClient(...) only when you need explicit remote command capability.
• Workers controls upload concurrency. Start with 5 to 10 and tune based on network and remote server capacity.
• Full sync is safer for complete release replacement, but it changes the whole remote directory.
• Incremental sync is faster for small changes, but it does not delete remote files that no longer exist locally.
• Restart commands run on the remote host through SSH, not on the local machine.
• This SDK does not decide whether a restart is needed. The caller owns that decision.

Examples

• examples/basic: upload plus optional caller-controlled restart.
• examples/simple: upload-only minimal integration.

Run examples locally from their own directories:

cd examples/basic
go run main.go

The example modules use replace github.com/lpg-it/fileuploader => ../../ so they compile against the local checkout. In a real consuming project, do not add that replace; install the release with go get github.com/lpg-it/fileuploader@v1.0.2.

Test

Run the root SDK tests:

go test ./...

Run all repository checks, including example modules:

go test ./...
(cd examples/basic && go test ./...)
(cd examples/simple && go test ./...)

Versioning

This repository uses Go module tags. Release v1.0.2 includes:

• Explicit container restart helper.
• Separation between synchronization and restart behavior.
• Synchronization mode constants.
• Worker defaulting for safer SDK usage.
• Updated README SDK integration guide.
• Tidied Go module dependencies.