feat: reduce code duplication

.github/workflows/python.yaml

@@ -11,44 +11,41 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-    - name: checkout
-      uses: actions/checkout@v3
+      - name: checkout
+        uses: actions/checkout@v3
 
-    - name: Set up Python
-      uses: actions/setup-python@v4
-      with:
-        python-version: '3.11'
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
 
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install tox
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install tox
 
-    - name: Run lint
-      run: tox -e lint
-    
-    - name: Run type
-      run: tox -e type
+      - name: Run lint
+        run: tox -e lint
+
+      - name: Run type
+        run: tox -e type
 
   test-python:
     runs-on: ubuntu-latest
 
     steps:
-    - name: checkout
-      uses: actions/checkout@v3
-
-    - name: Set up Python
-      uses: actions/setup-python@v4
-      with:
-        python-version: '3.11'
-
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install tox
-
-    - name: Set up fake AWS credentials
-      run: sh tests/fake_aws_creds.sh
-
-    - name: Run tests
-      run: tox -e 3.11
+      - name: checkout
+        uses: actions/checkout@v3
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install tox
+
+      - name: Run tests
+        run: tox -e 3.11

.gitignore

@@ -33,4 +33,5 @@ cover/
 **.env
 
 # Include sample and test config files
-!tests/data/backup/test_local_conf.yaml
+!tests/data/**
+!docs/**

README.md

@@ -1,105 +1,79 @@
 # gooddata-productivity-tools
+
 This repository contains tools that help with GoodData Cloud/CN workspace management, user and user group management, and backup/restore of workspaces.
 
 This section of the documentation contains information on how to set up the environment and relevant authentication files. At the end of the Tools section, there is more specific documentation for each tool. The steps mentioned here are shared between them.
 
 ## Requirements
+
 Python 3.11
 
 Depending on your environment, the statements can start either as
+
 ```sh
-pip  
+pip
 pip3
 ```
+
 ```sh
 python
 python3
 ```
-please use the one that works for you and refers to python 3.10+.
+
+please use the one that works for you and refers to Python 3.11.
 
 The version can be checked by running
+
 ```sh
 python -V
 ```
 
 ## Install
-In order to install tooling requirements to the target environment, run the following: 
+
+In order to install tooling requirements to the target environment, run the following:
 
 ```sh
 pip install -r requirements.txt
 ```
 
 ## Authentication
 
-Overall, the scripts developed within the repository follow the credential/authentication provisioning conventions of the GoodData and any used storage provider (e.g. AWS).
-
-The following section describes what credentials need to be set up, where to find them, and what format they should follow. If you need help with how to edit files in your user home folder (~), you can also refer to [step by step authentication setup guide](docs/SETUPATUHENTICATION.md).
-
-
-### GoodData
-When authenticating against GoodData, you can either export the required credentials using environment variables, or provide a GoodData profiles file.
-
-For example, you can export the environment variables like so:
-
-```sh
-export GDC_AUTH_TOKEN="some_auth_token"
-export GDC_HOSTNAME="https://host.name.cloud.gooddata.com/"
-```
-
-or you can choose to provide a GoodData `profiles.yaml` file of the following format:
-
-```yaml
-default:
-  host: https://host.name.cloud.gooddata.com/
-  token: some_auth_token
-
-customer:
-  host: https://customer.hostname.cloud.gooddata.com/
-  token: other_auth_token
-```
-
-By default, a tool attempts to locate a GoodData profile file at `~/.gooddata/profiles.yaml`, but you can also choose to provide a custom path like so:
+The scripts follow standard credential/authentication conventions for GoodData and storage providers (e.g., AWS).
 
-```sh
-python scripts/restore.py <input.csv> <conf.yaml> -p path/to/profiles.yaml
-```
-
-You can define multiple GoodData profiles in a single profiles file. By default, the `default` profile is used, but you can choose different one to use. For example, if you want to tell a tool to use the `customer` profile defined in the example `profiles.yaml` above, you can do so like this:
-
-```sh
-python scripts/restore.py <input.csv> <conf.yaml> -p path/to/profiles.yaml --profile customer
-```
-
-In case of providing both ways of authentication to a tool, the environment variables takes precedence and the profiles config is ignored.
+### Quick Overview
 
-### AWS
+**GoodData Authentication:**
 
-When authenticating against AWS, the [conventions made by the boto3 library](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) are followed.
+- Environment variables: `GDC_AUTH_TOKEN` and `GDC_HOSTNAME`
+- Profile file: `~/.gooddata/profiles.yaml` (supports multiple profiles)
+- Tools use the `default` profile by default
+- Environment variables take precedence over profile files
 
-From the tool user perspective that means following the points 3. to 8. from the [Configuring Credentials section](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials).
+**AWS Authentication:**
 
-One example of how you can supply AWS credentials for tools, is by defining one or more AWS profiles in `~/.aws/credentials` file.
+- Follows [boto3 credential resolution](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html)
+- Common method: AWS credentials file at `~/.aws/credentials`
+- See tool-specific docs for profile selection
 
-```
-[default]
-aws_access_key_id = some_key_id
-aws_secret_access_key = some_access_key
-
-[services]
-aws_access_key_id = other_key_id
-aws_secret_access_key = other_access_key
-```
+### Detailed Setup
 
-If you want to specify the specific AWS credentials profile to be used, see the tool-specific documentation.
+For step-by-step instructions on creating and configuring authentication files, including file formats and examples, see the [Authentication Setup Guide](docs/guides/SETUPAUTHENTICATION.md).
 
 ## Tools
 
-- [Backup workspace](docs/BACKUP.md)
-- [Restore workspace](docs/RESTORE.md)
-- [Workspace permission management](docs/PERMISSION_MGMT.md)
-- [User management](docs/USER_MGMT.md)
-- [User group management](docs/USER_GROUP_MGMT.md)
+- [Backup workspace](docs/tools/BACKUP.md)
+- [Restore workspace](docs/tools/RESTORE.md)
+- [Workspace management](docs/tools/WORKSPACE_MGMT.md)
+- [Workspace permission management](docs/tools/PERMISSION_MGMT.md)
+- [User management](docs/tools/USER_MGMT.md)
+- [User group management](docs/tools/USER_GROUP_MGMT.md)
+- [User data filter management](docs/tools/USER_DATA_FILTER_MGMT.md)
+- [Custom fields management](docs/tools/CUSTOM_FIELDS.md)
 
+### Reference Documentation
+
+- [Common Arguments Reference](docs/reference/COMMON_ARGUMENTS.md) - Detailed explanations of shared CLI arguments
+- [Storage Configuration Reference](docs/reference/STORAGE_CONFIG.md) - Guide to configuring S3 and local storage backends
 
 ## Known MacOS issue SSL: CERTIFICATE_VERIFY_FAILED
 
@@ -118,6 +92,7 @@ To mitigate, please install your SSL certificates in HD -> Applications -> Pytho
 This section is aimed towards developers wanting to adjust / test the code. If you are regular user you can ignore following parts.
 
 ### Setup
+
 To set up local development environment do the following:
 
 1. (optional) Set up a local python virtual environment:
@@ -133,8 +108,8 @@ To set up local development environment do the following:
 pip install -r requirements.txt -r requirements-test.txt -r requirements-dev.txt
 ```
 
-
 ### Style checking, linting, and typing
+
 The codebase (both, scripts and tests) is style, lint, and type checked when the CI/CD pipeline runs.
 
 Linting and style-checking is done with help of `black` and `ruff`.
@@ -147,19 +122,20 @@ To run either of the mentioned tools locally, just call the tool with a target d
 <black|ruff|mypy> <target_path>
 ```
 
- For example, in order to check the typing in the scripts, call the following from the repository's root directory:
+For example, in order to check the typing in the scripts, call the following from the repository's root directory:
 
 ```sh
 mypy scripts
 ```
 
 To check that the code styling will pass pre-merge checks when creating a pull request, run:
+
 ```sh
 tox -e lint
 ```
 
-
 ### Testing
+
 The tooling test suite makes use of some third party tools, such as `pytest`, `tox`, and `moto`.
 
 To run the test suite locally, ensure you have test and script requirements installed (see Setup step above) change working directory to repository's root and then call:
@@ -168,7 +144,6 @@ To run the test suite locally, ensure you have test and script requirements inst
 pytest .
 ```
 
-
 ### Tox
 
 The pre-merge checks run via GitHub actions use tox to verify the code style and test cases.
@@ -182,4 +157,5 @@ tox
 ```
 
 ## Contributing
+
 If you want to contribute to the project, please read the [contributing guide](CONTRIBUTING.md).