Avoid caching packages

[DonJayamanne]

I think this should be avoided, as it is this is causing a few issues in Jupyter.
E.g. i can uninstall/install packages from Jupyter extension by running code like %pip install in a cell
This is valid jupyter code, and very commonly used practice.

However the Python env tools returns stale information, hence the Python tools used in Copilot return stale information.
Restarting Jupyter kernels. or restarting the chat doesn't fix the issue, I need to reload VS Code.

Not sure why we need the cache, but i think its fair to say that we've learnt that caching can cause a lot of issues.

    async getPackages(environment: PythonEnvironment): Promise<Package[] | undefined> {
        if (!this.packages.has(environment.envId.id)) {
            await this.refresh(environment);
        }
        return this.packages.get(environment.envId.id);
    }

[karthiknadig]

We should monitor and refresh the packages whenever we see a Python package folder get added to site-packages. Depending on the scenario. There can be lots of calls to get package list, and we don’t want it to run pip or conda every single time.

[eleanorjboyd]

What is the best solution given this concern about efficiency? Can we do something like 'getPackages' triggers a hard refresh in certain scenarios such as this one (using the copilot tool)?

[karthiknadig]

We can try File System monitoring. But really there is no general solution. Ideally, there would be some hook the env framework would provide tell us there is an update.

An alternative for. Jupyter could be that we expose a Refresh API that Jupyter can use when it is detected that a cell with %pip was run.

[eleanorjboyd]

@DonJayamanne what are your thoughts here? For the MVP of the envs extension what do you think makes sense for notebooks?

[DonJayamanne]

I think we can close this, if and when we do run into a real issue we can deal with that.
E.g. if user sees stale information, we can provide a refresh button (if not already available)
If an extension gets stale information, we can review that and perhaps provide a refresh argument/method.