Provide a step-by-step guide, how to develop a python extension

[tkilias]

Provide a step-by-step guide, how to develop a python extension

See link in IntRef for a first example adopting the Context Managers and Fixtures mentioned below.

Draft:

Step 1: Define and Build the Language Container Using your Current Poetry Project

See TE/deployment/language_container

1. Add dependency to python-extension-common:

   poetry add exasol-python-extension-common --group dev 

2. Add poetry plugin export in file pyproject.toml:

   [tool.poetry.requires-plugins]
   poetry-plugin-export = ">=1.8"

3. Define a context manager language_container_factory

   from contextlib import contextmanager
   
   from exasol.python_extension_common.deployment.language_container_builder import (
       LanguageContainerBuilder,
       find_path_backwards,
   )
   
   CONTAINER_NAME = "exasol_mlflow_plugin"
   
   @contextmanager
   def language_container_factory():
       with LanguageContainerBuilder(CONTAINER_NAME) as container_builder:
           project_directory = find_path_backwards("pyproject.toml", __file__).parent
           container_builder.prepare_flavor(project_directory)
           yield container_builder

4. Define a context manager export_slc which you can use to export the SLC as tar.gz.
   See TE/deployment/language_container

Where is context manager export_slc used?

Step 2: For Integration Tests: Use the context manager language_container_factory with pytest-slc

Add dependency to pytest-exasol-slc:

poetry add pytest-exasol-slc --group dev 

The README of pytest-slc contains an example, but for short, you have to implement two fixtures:

@pytest.fixture(scope='session')
def language_alias():
    return "MY_LANGUAGE_ALIAS"

@pytest.fixture(scope='session')
def slc_builder(use_onprem, use_saas):
    if use_onprem or use_saas:
        with language_container_factory() as container_builder:
            yield container_builder
    else:
        yield None

and then you can use the SLC in your test with

def test_something_with_slc(deployed_slc):
    ...

deployed_slc returns the language alias which you need to use to create your script with.

Step 3: Add a Nox session to export the SLC

This step only applies, if you are using Nox as task runner in your project.

You can use the context manager, see TE noxfile.py

Step 4: Optional Python Function for Deploying the Container to the Database

• Add a subclass of LanguageContainerDeployer, see TE deployment/te_language_container_deployer.py.
• TeLanguageContainerDeployer, downloads the SLC from the GitHub Releases with download_from_github_and_run(), but is specific to how we release extensions.
• With the run method, you can also deploy a local file.

Step 5: Optional CLI for Deploying

References

• TE deploy.py for inspiration
• PEC user guide describing the usage of the CLI for the SLC deployment

Some additional explanation:

ver_formatter = ParameterFormatters()
ver_formatter.set_formatter(CONTAINER_URL_ARG, TeLanguageContainerDeployer.SLC_URL_FORMATTER)
ver_formatter.set_formatter(CONTAINER_NAME_ARG, TeLanguageContainerDeployer.SLC_NAME)
formatters = {StdParams.version: ver_formatter}

• The SLC deployment CLI can download the Container for a specific version from a URL.
• If the option --version is provided, otherwise the --container-file needs to be provided.
• As I wrote, we are using this for downloading the SLC from GitHub releases.