Skip to content

connect --upgrade-environment-file appends malformed, key-less values to offload.env #266

@nj1973

Description

@nj1973

When running connect --upgrade-environment-file to update an environment configuration file with missing variables from a template, the utility appends only the right-hand-side (RHS) value of the configuration variable (e.g. 0 or false ) instead of writing the valid KEY=value assignment.

This happens because the environment file parser in connect.py extracts and stores only the value of the key-value pair, discarding the key prefix when mapping properties:

https://github.com/gluent/goe/blob/main/src/goe/connect/connect.py#L306-L314

    def dict_from_environment_file(environment_file):
        d = {}
        with open(environment_file) as f:
            for line in f:
                kv = config_file.env_key_value_pair(line)
                if kv:
                    k, v = kv
                    d[k] = v  # <--- Only the 'v' (value) is stored
        return d

When appending missing entries to offload.env , the upgrade logic writes the dictionary value directly to the file without reinstating the key name:

https://github.com/gluent/goe/blob/main/src/goe/connect/connect.py#L378-L380

                for key in sorted(set(template.keys()) - set(configuration.keys())):
                    detail("Adding %s" % key)
                    f.write(template[key])  # <--- Writes only 'v' (e.g. "false")

Steps to Reproduce / Test Case

You can reproduce this behavior with the following Python reproduction script:

    import re
    from typing import Optional

    # Copy of regex and parser from config_file.py & connect.py
    KEY_VALUE_PATTERN = re.compile(r"#?[ ]*([A-Z_0-9]+)=(.*)")

    def env_key_value_pair(line_from_file: str) -> Optional[tuple]:
        m = KEY_VALUE_PATTERN.match(line_from_file)
        return m.groups() if m else None

    def dict_from_environment_file(content: str):
        d = {}
        for line in content.splitlines():
            kv = env_key_value_pair(line)
            if kv:
                k, v = kv
                d[k] = v
        return d

    # Simulated template file content
    template_content = """
    FRONTEND_DISTRIBUTION=ORACLE
    DATA_SAMPLE_PARALLELISM=0
    """
    # Simulated active user configuration (missing DATA_SAMPLE_PARALLELISM)
    config_content = """
    FRONTEND_DISTRIBUTION=ORACLE
    """

    template = dict_from_environment_file(template_content)
    configuration = dict_from_environment_file(config_content)

    print("Template variables found:", template)
    print("Active config variables found:", configuration)

    # Replicating write block from connect.py
    for key in sorted(set(template.keys()) - set(configuration.keys())):
        print(f"\nAdding missing key: {key}")
        print(f"DEBUG: Executing f.write(template['{key}'])")
        print(f"ACTUAL OUTPUT APPENDED: {template[key]}")

Actual Output

Template variables found: {'FRONTEND_DISTRIBUTION': 'ORACLE', 'DATA_SAMPLE_PARALLELISM': '0'}
Active config variables found: {'FRONTEND_DISTRIBUTION': 'ORACLE'}
Adding missing key: DATA_SAMPLE_PARALLELISM
DEBUG: Executing f.write(template['DATA_SAMPLE_PARALLELISM'])
ACTUAL OUTPUT APPENDED: 0

Expected Behavior

The missing configuration key should be appended to the environment file in a valid KEY=value format:

DATA_SAMPLE_PARALLELISM=0

Suggested Fix

Modify the write block in upgrade_environment_file() in src/goe/connect/connect.py to write both the key and the value:

    -            for key in sorted(set(template.keys()) - set(configuration.keys())):
    -                detail("Adding %s" % key)
    -                f.write(template[key])
    +            for key in sorted(set(template.keys()) - set(configuration.keys())):
    +                detail("Adding %s" % key)
    +                f.write(f"{key}={template[key]}\n")

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions