CI Pipeline Reports Successful Linting Despite Notebook Linter Failing to Start

[iamwatchdogs]

The CI pipeline that performs linting checks is generating false positives, as the success code is 0 while the actual linting task for Jupyter Notebooks fails to start. This problem has existed for some time and was recently observed during the last PR merge.

Analysis

• During the step that lints Jupyter Notebook files, the workflow tries to run a command: pynblint <path>, which appears to use the typer library to create a CLI for linting notebook files.
• The code seems to specify a secondary flag (short option) for a non-boolean parameter in the CLI definition, which is not allowed by click/Typer.
• As a result, the pynblint tool fails to run, causing the workflow to error out for the linting step.

Other observation

Also, using linting tools like pynblint to perform linting on Jupyter Notebooks might not be the best choice, since pynblint never really throws a failed status code to fail the whole linting workflow.

Well, it can be configured to throw an error status code to fail the linting jobs when a potential error is found. However, I believe there are many better tools to handle it properly with more flexibility.

[iamwatchdogs]

Approach to Resolving the Issue

Initially, I thought of resolving this issue by saving the script as a file and executing it with valid permissions, or even moving completely from Python and translating the same logic into a bash script.

But while testing out the pynblint locally, I realised that this is more of a static analyser than a proper linter.

With that, I started searching for other options that better suit the current requirement, and I ended up finding about nbqa tool that lets us lint Jupyter Notebook with a linter of our choice.

Using the nbqa tool also helps us overcome the issue which we faced initially with pynblint, i.e., the TypeError raised by the Typer library that is being internally used by the pynblint tool.

And since we're expecting to have more contributions in the form of Jupyter Notebooks compared to Python files, it might be better to choose a more verbose, highly descriptive and strict linter for Jupyter.

My Initial go-to choice was flake8 due to its wide usage in industry-level linting workflows and also 'cause we're already using it to perform linting for Python files. But after a bit of exploration, I felt like pylint might be a better choice and match the description of the requirement that I just mentioned earlier.