Skip to content

Anticipate deprecation of *args and *kwargs to BaseOperator  #10

@lauralorenz

Description

@lauralorenz

Recently running some stuff on fileflow running airflow 1.7.1.3 I was getting deprecation warnings for the following:

/Users/llorenz/Envs/fileflow/lib/python2.7/site-packages/airflow/models.py:1719: PendingDeprecationWarning: Invalid arguments were passed to DivePythonOperator. Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:

*args: ()

**kwargs: {'data_dependencies': {'something': 'write_a_file'}}

  category=PendingDeprecationWarning

According to the dev email list this is anticipation of only supporting explicit args:

Hi Laura,

The error is raised if an unused argument is passed to BaseOperator --
basically if there is anything in either args or kwargs. The original issue
was that in a number of cases arguments were misspelled or misused by
Operator subclasses and instead of raising an error, they were just passed
up the inheritance chain and finally (silently) absorbed by BaseOperator,
so there was no warning.

I think a workaround should be straightforward -- when you call
super().init for the BaseOperator, just pass arguments explicitly
rather than with args/kwargs, or (alternatively), pop arguments out of
kwargs when you use them ahead of calling that init.

Our case might be a little harder because we're subclassing PythonOperator not BaseOperator directly, but we can start at this point to see how we want to workaround for Airflow 2.0. That being said, Airflow 2.0 probably isn't coming out for another... year?... so we have some time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions