jedi-vim is not aware of pyenv path

[frank777777777]

Issue

I am using pyenv to switch between difference python version. The autocomplete and goto function only uses modules from system python. For example, while I am in a pyenv virtual environment, I installed beautifulsoup module, jedi-vim does not find my module in my virtual environment, it searches for module in system site-packages instead.
As you can see my verbose JediDebugInfo, my python2 environment is linking to pyenv, while python3 environment is linking to system python.
In vim, !python --version and !pip --version works, its just jedi seems not finding my pyenv.

Steps to reproduce

Output of “:verbose JediDebugInfo”

##### jedi-vim version

 - jedi-vim git version: 0.9.0-57-g132d850-dirty
 - jedi git submodule status:  a79a1fbef57564d031c59f47def20a6d9adf3439 pythonx/jedi (v0.12.1)
 - parso git submodule status:  e6bc924fbabbfa3090044e475c91a7655b9c6e9b pythonx/parso (v0.3.1)

##### Global Python

Using Python version 3 to access Jedi.
 - global sys.version: `3.5.2 (default, Nov 23 2017, 16:37:01), [GCC 5.4.0 20160609]`
 - global site module: `/usr/lib/python3.5/site.py`

##### Jedi

 - path: `/home/yili/.vim/bundle/jedi-vim/pythonx/jedi/jedi/__init__.py`
 - version: 0.12.1

##### Jedi environment: <Environment: 3.6.5 in /usr>

 - executable: /usr/bin/python3.6
 - sys_path:
    - `/usr/lib/python36.zip`
    - `/usr/lib/python3.6`
    - `/usr/lib/python3.6/lib-dynload`
    - `/usr/local/lib/python3.6/dist-packages`
    - `/usr/lib/python3/dist-packages`

##### Known environments

 - <Environment: 3.6.5 in /usr> (/usr/bin/python3.6)
 - <Environment: 3.5.2 in /usr> (/usr/bin/python3.5)
 - <Environment: 2.7.15 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python2.7)

##### Settings

g:jedi#documentation_command = 'J' (default: 'K')
g:jedi#usages_command = 'n' (default: 'n')
g:jedi#completions_enabled = 0 (default: 1)

omnifunc=python3complete#Complete
Last set from /usr/share/vim/vim81/ftplugin/python.vim
completeopt=preview,menuone
Last set from ~/.vim/bundle/YouCompleteMe/autoload/youcompleteme.vim

#### :version

VIM - Vi IMproved 8.1 (2018 May 18, compiled Jul 6 2018 08:05:07)
Included patches: 1-155
Modified by pkg-vim-maintainers@lists.alioth.debian.org
Compiled by pkg-vim-maintainers@lists.alioth.debian.org
Huge version without GUI. Features included (+) or not (-):
+acl +cursorshape +job -mouse_sysmouse +scrollbind +vertsplit
+arabic +dialog_con +jumplist +mouse_urxvt +signs +virtualedit
+autocmd +diff +keymap +mouse_xterm +smartindent +visual
-autoservername +digraphs +lambda +multi_byte +startuptime +visualextra
-balloon_eval -dnd +langmap +multi_lang +statusline +viminfo
+balloon_eval_term -ebcdic +libcall -mzscheme -sun_workshop +vreplace
-browse +emacs_tags +linebreak +netbeans_intg +syntax +wildignore
++builtin_terms +eval +lispindent +num64 +tag_binary +wildmenu
+byte_offset +ex_extra +listcmds +packages +tag_old_static +windows
+channel +extra_search +localmap +path_extra -tag_any_white +writebackup
+cindent +farsi -lua -perl -tcl -X11
-clientserver +file_in_path +menu +persistent_undo +termguicolors -xfontset
-clipboard +find_in_path +mksession +postscript +terminal -xim
+cmdline_compl +float +modify_fname +printer +terminfo -xpm
+cmdline_hist +folding +mouse +profile +termresponse -xsmp
+cmdline_info -footer -mouseshape -python +textobjects -xterm_clipboard
+comments +fork() +mouse_dec +python3 +timers -xterm_save
+conceal +gettext +mouse_gpm +quickfix +title
+cryptv -hangul_input -mouse_jsbterm +reltime -toolbar
+cscope +iconv +mouse_netterm +rightleft +user_commands
+cursorbind +insert_expand +mouse_sgr -ruby +vartabs
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/defaults.vim"
fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -Wdate-time -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=f
ormat-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lm -ltinfo -l
nsl -lselinux -lacl -lattr -lgpm -ldl -L/usr/lib/python3.5/config-3.5m-x86_64-linux-gnu -lpython3.5m -lpthread -l
dl -lutil -lm

[blueyed]

Try let g:jedi#force_py_version = '2.7.15'.

Also see #836.

[frank777777777]

Try let g:jedi#force_py_version = '2.7.15'.

I am trying to use python 3 not python2...
I just realized that, if I install python3.6 in pyenv, jedi will refer the python executable to pyenv. If I use python 3.7 in pyenv, jedi will ignore pyenv and use system python executable instead. It seems to me that jedi has hardcoded that it only look for python 3.6 and 3.5, because python 3.5 executable still link to system python. Here is the verbose JediDebugInfo after I install python3.6 in pyenv:

##### Jedi environment: <Environment: 3.6.0 in /home/yili/.pyenv>

 - executable: /home/yili/.pyenv/shims/python3.6
 - sys_path:
    - `/home/yili/.pyenv/versions/3.6.0/lib/python36.zip`
    - `/home/yili/.pyenv/versions/3.6.0/lib/python3.6`
    - `/home/yili/.pyenv/versions/3.6.0/lib/python3.6/lib-dynload`
    - `/home/yili/.pyenv/versions/3.6.0/lib/python3.6/site-packages`

##### Known environments

 - <Environment: 3.6.0 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python3.6)
 - <Environment: 3.5.2 in /usr> (/usr/bin/python3.5)
 - <Environment: 2.7.15 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python2.7)

The executable refers to pyenv python, works fine. The sys_path refers to pyenv site-package, works fine. Omni-completion and goto works perfect for modules install in pyenv site package.

However, when I create a virtualenv in pyenv like pyenv virtualenv 3.6.0 scrapper, pyenv local scrapper, the sys_path refer back to system site package. Here is the :verbose JediDebugInfo:

##### Jedi environment: <Environment: 3.6.5 in /home/yili/.pyenv>

 - executable: /home/yili/.pyenv/shims/python3.6
 - sys_path:
    - `/usr/lib/python36.zip`
    - `/usr/lib/python3.6`
    - `/usr/lib/python3.6/lib-dynload`
    - `/usr/local/lib/python3.6/dist-packages`
    - `/usr/lib/python3/dist-packages`

##### Known environments

 - <Environment: 3.6.5 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python3.6)
 - <Environment: 3.5.2 in /usr> (/usr/bin/python3.5)
 - <Environment: 2.7.15 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python2.7)

The executable stills uses pyenv, but the sys_path link back to system site-package, omni-completion and goto does not work anymore.

I am running out ideas.

[blueyed]

Thanks for the input.

Please consider trying #836 and see if that makes it better already - otherwise check the source to see what actually happens.
Currently venv support is very basic, and I do not remember the state of 836 really, but it should allow to switch and tab-complete them actually.

[blueyed]

Re "/home/daniel/.pyenv/shims/scrapper": it is using "Environment: 3.6.5 in /home/yili/.pyenv" there then.
IIRC jedi looks not for "python" first, but with versions appended.. at least it used to do.
Digging into the source (based on #836) is the best you can do currently I guess - if you want to help fix this.
Otherwise try to adjust g:jedi#force_py_version to make it pick up the version you want, but that is also only rudimentary on master.

[frank777777777]

Isn't g:jedi#force_py_version just to specify the python executable version only? I mean it is already picking up the correct python version in pyenv if I install 3.6.0.
It is the sys-path that is not incorrect. I saw in #836 you mentioned use let g:jedi#environment_paths = [$HOME . '/.pyenv/versions]', is it implemented yet?

[blueyed]

It does a bit more on master already:

jedi-vim/pythonx/jedi_vim.py

Lines 166 to 185 in 132d850

	vim_force_python_version = vim_eval("g:jedi#force_py_version")
	if use_cache and vim_force_python_version == current_environment[0]:
	return current_environment[1]
	environment = None
	if vim_force_python_version == "auto":
	environment = jedi.api.environment.get_cached_default_environment()
	else:
	force_python_version = vim_force_python_version
	if '0000' in force_python_version or '9999' in force_python_version:
	# It's probably a float that wasn't shortened.
	try:
	force_python_version = "{:.1f}".format(float(force_python_version))
	except ValueError:
	pass
	elif isinstance(force_python_version, float):
	force_python_version = "{:.1f}".format(force_python_version)
	try:
	environment = jedi.get_system_environment(force_python_version)

you mentioned use let g:jedi#environment_paths = [$HOME . '/.pyenv/versions]', is it implemented yet?

I think so:

jedi-vim/pythonx/jedi_vim.py

Lines 218 to 223 in c06c8ef

	def get_known_environments():
	"""Get known Jedi environments."""
	paths = vim_eval("g:jedi#environment_paths")
	envs = list(jedi.api.environment.find_virtualenvs(paths=paths, safe=False))
	envs.extend(jedi.api.environment.find_system_environments())
	return envs

[frank777777777]

wait, is this still a working progress and not committed to master yet? Because what I see in jedi-vim/pythonx/jedi_vim.py, it is still:

196 def get_known_environments():
197     """Get known Jedi environments."""
198     envs = list(jedi.api.environment.find_virtualenvs())
199     envs.extend(jedi.api.environment.find_system_environments())
200     return envs

[blueyed]

Yes, it is WIP - an open PR.
To get it working for you you could also hardcode something there, of course.

[blueyed]

davidhalter/jedi#1199
#872

[statham-stone]

change all "safe=True" to "safe=False" in "~/.vim/bundle/jedi-vim/pythonx/jedi/jedi/api/environment.py" solved the problem for me.

[bo5o]

I am experiencing a similar issue.

I have system python version 3.6.8 installed and use pyenv to manage multiple python installations. When I create a virtual environment with python -m venv .venv jedi-vim is not able to provide completions. This only happens if I set the python version to anything other than the system version (e.g. via pyenv local 3.7.4). I am using ALE to provide completions.

:verbose JediDebugInfo

#### Jedi-vim debug information

##### jedi-vim version

 - jedi-vim git version: 0.10.0-3-g4f2499e
 - jedi git submodule status:  005f69390c4b1a3b864e7373cfcbdaf95e65411d pythonx/jedi (v0.15.1)
 - parso git submodule status:  c0ace63a6905500e18d8ac06af1a0a9057e63498 pythonx/parso (v0.5.1)

##### Global Python

Using Python version 3 to access Jedi.
 - global sys.version: `3.6.8 (default, Aug 20 2019, 17:12:48), [GCC 8.3.0]`
 - global site module: `/usr/lib/python3.6/site.py`

##### Jedi

 - path: `~/.local/share/nvim/plugged/jedi-vim/pythonx/jedi/jedi/__init__.py`
 - version: 0.15.1

##### Jedi environment: <Environment: 3.7.4 in /tmp/myapp/.venv>

 - executable: /tmp/myapp/.venv/bin/python
 - sys_path:
    - `~/.pyenv/versions/3.7.4/lib/python37.zip`
    - `~/.pyenv/versions/3.7.4/lib/python3.7`
    - `~/.pyenv/versions/3.7.4/lib/python3.7/lib-dynload`
    - `/tmp/myapp/.venv/lib/python3.7/site-packages`

##### Known environments

 - <Environment: 3.7.4 in /tmp/myapp/.venv> (/tmp/myapp/.venv/bin/python)
 - <Environment: 3.7.4 in ~/.pyenv/versions/3.7.4> (~/.pyenv/versions/3.7.4/bin/python3.7)
 - <Environment: 3.6.8 in /usr> (/usr/bin/python3.6)
 - <Environment: 2.7.15 in /usr> (/usr/bin/python2.7)

##### Settings

g:jedi#completions_enabled = 0 (default: 1)

  omnifunc=python3complete#Complete
        Zuletzt gesetzt in /tmp/.mount_nvimt3iaAF/usr/share/nvim/runtime/ftplugin/python.vim line 41
  completeopt=noinsert,menuone,noselect
        Zuletzt gesetzt in ~/dotfiles/nvim/.config/nvim/init.vim line 178

#### :version

NVIM v0.4.0-2014-g554566cce
Build type: RelWithDebInfo
LuaJIT 2.0.5
Features: +acl +iconv +tui

Although it recognizes the correct environment (which is in /tmp/myapp/.venv) and the correct executable (which is python3.7 in ~/.pyenv/versions) I can not autocomplete packages I install in the virtual environment. Weirdly I can only autocomplete packages installed with system python.

[blueyed]

@cbows
With Neovim check :checkhealth provider to see which Python it uses.

[bo5o]

health#provider#check
========================================================================
## Clipboard (optional)
  - OK: Clipboard tool found: xclip

## Python 2 provider (optional)
  - INFO: pyenv: Path: ~/.pyenv/libexec/pyenv
  - INFO: pyenv: $PYENV_ROOT is not set. Infer from `pyenv root`.
  - INFO: pyenv: Root: ~/.pyenv
  - INFO: Using: g:python_host_prog = "/usr/bin/python2"
  - WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results.
    - ADVICE:
      - If you are using Zsh, see: http://vi.stackexchange.com/a/7654
  - INFO: Executable: /usr/bin/python2
  - INFO: Python version: 2.7.15
  - INFO: pynvim version: 0.3.2

## Python 3 provider (optional)
  - INFO: pyenv: Path: ~/.pyenv/libexec/pyenv
  - INFO: pyenv: $PYENV_ROOT is not set. Infer from `pyenv root`.
  - INFO: pyenv: Root: ~/.pyenv
  - INFO: Using: g:python3_host_prog = "/usr/bin/python3"
  - WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results.
    - ADVICE:
      - If you are using Zsh, see: http://vi.stackexchange.com/a/7654
  - INFO: Executable: /usr/bin/python3
  - INFO: Python version: 3.6.8
  - INFO: pynvim version: 0.3.2

I don't understand the warning WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results.. I have activated the environment with source .venv/bin/activate as usual. I am using zsh but the advice didn't help.

If I unset g:python3_host_prog and let nvim figure out the executable, it finds the correct version from pyenv root, but it doesn't solve the problem (also I would have to install pynvim for every version I have installed).

[blueyed]

WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results

This means it is not using the virtualenv's python (which might be fine of course).

As for jedi-vim: there's a host Python (the one Neovim uses), and it will use virtualenvs then.
Jedi/jedi-vim uses /tmp/myapp/.venv/bin/python - what completions are you missing from there?

[blueyed]

maybe you are running into an issue where the host (3.6) cannot handle syntax for the venv (3.7)?