Installation issues with astroquery and setuptools

[astromark]

I'm trying to install astroquery in casa using casa-pip. I received an error telling me that I need to upgrade setuptools as I have an old version (3.4.3). So I run casa-pip install setuptools --upgrade and it says it has installed version 25.2.0 successfully. I try again to install astroquery and again it tells me I have an old version of setuptools (now 0.9.5!). It seems that my casa directory tree has version 0.9.5 and the latest version is being installed by casa-pip in my .casa directory tree but then being ignored by casa-pip when it installs astroquery.

[astrofrog]

@astromark - can you try and use the following technique instead to see if that helps:

http://docs.astropy.org/en/stable/install.html#installing-astropy-into-casa

Once you've done that, do:

CASA <2>: import pip

CASA <3>: pip.main(['install', 'astroquery', '--user'])

[astromark]

I tried that. It gives the same errors.

[keflavich]

Strange. Could you print pip.__path__? I'm not really sure what to do about this yet, but I suspect the path ordering must be at least partially at fault.

[astromark]

'/astro/mark/.casa/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg/pip'

[keflavich]

OK, that looks like it is the right version. Though... I don't fully understand why you have pip-1.5.4 and not 3.4.3 or 25.2.0 or 0.9.5....

Could you paste the full traceback from running the pip command within CASA?

[astromark]

Here it is. I'm not sure why it is doing everything four times. I also attach the pip.log in case that helps.
pip.log.txt

CASA <8>: pip.main(['install', 'astroquery', '--user'])
Downloading/unpacking astroquery
Downloading/unpacking astroquery
Downloading/unpacking astroquery
Downloading/unpacking astroquery
  Downloading astroquery-0.3.2.tar.gz (2.5MB): 2.5MB downloaded
  Running setup.py (path:/tmp/pip_build_mark/astroquery/setup.py) egg_info for package astroquery
  Running setup.py (path:/tmp/pip_build_mark/astroquery/setup.py) egg_info for package astroquery
  Running setup.py (path:/tmp/pip_build_mark/astroquery/setup.py) egg_info for package astroquery
  Running setup.py (path:/tmp/pip_build_mark/astroquery/setup.py) egg_info for package astroquery

Requirement already satisfied (use --upgrade to upgrade): astropy>=0.4.1 in /astro/mark/.casa/lib/python2.7/site-packages (from astroquery)
Requirement already satisfied (use --upgrade to upgrade): astropy>=0.4.1 in /astro/mark/.casa/lib/python2.7/site-packages (from astroquery)
Requirement already satisfied (use --upgrade to upgrade): astropy>=0.4.1 in /astro/mark/.casa/lib/python2.7/site-packages (from astroquery)
Requirement already satisfied (use --upgrade to upgrade): astropy>=0.4.1 in /astro/mark/.casa/lib/python2.7/site-packages (from astroquery)
Downloading/unpacking requests>=2.4.3 (from astroquery)
Downloading/unpacking requests>=2.4.3 (from astroquery)
Downloading/unpacking requests>=2.4.3 (from astroquery)
Downloading/unpacking requests>=2.4.3 (from astroquery)
  Downloading requests-2.11.1-py2.py3-none-any.whl (514kB): 514kB downloaded
Downloading/unpacking keyring>=4.0 (from astroquery)
Downloading/unpacking keyring>=4.0 (from astroquery)
Downloading/unpacking keyring>=4.0 (from astroquery)
Downloading/unpacking keyring>=4.0 (from astroquery)
  Downloading keyring-9.3.1-py2.py3-none-any.whl
  Downloading keyring-9.3.1-py2.py3-none-any.whl
  Downloading keyring-9.3.1-py2.py3-none-any.whl
  Downloading keyring-9.3.1-py2.py3-none-any.whl
Downloading/unpacking beautifulsoup4>=4.3.2 (from astroquery)
Downloading/unpacking beautifulsoup4>=4.3.2 (from astroquery)
Downloading/unpacking beautifulsoup4>=4.3.2 (from astroquery)
Downloading/unpacking beautifulsoup4>=4.3.2 (from astroquery)
  Downloading beautifulsoup4-4.5.1-py2-none-any.whl (83kB): 83kB downloaded
Downloading/unpacking html5lib>=0.999 (from astroquery)
Downloading/unpacking html5lib>=0.999 (from astroquery)
Downloading/unpacking html5lib>=0.999 (from astroquery)
Downloading/unpacking html5lib>=0.999 (from astroquery)
  Downloading html5lib-0.999999999.tar.gz (245kB): 245kB downloaded
  Running setup.py (path:/tmp/pip_build_mark/html5lib/setup.py) egg_info for package html5lib
  Running setup.py (path:/tmp/pip_build_mark/html5lib/setup.py) egg_info for package html5lib
  Running setup.py (path:/tmp/pip_build_mark/html5lib/setup.py) egg_info for package html5lib
  Running setup.py (path:/tmp/pip_build_mark/html5lib/setup.py) egg_info for package html5lib
    html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)
    html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)
    html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)
    html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)
    Complete output from command python setup.py egg_info:
    Complete output from command python setup.py egg_info:
    Complete output from command python setup.py egg_info:
    Complete output from command python setup.py egg_info:
    html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)

---

html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)

---

html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)

---

html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 0.9.5)

---

Cleaning up...
Cleaning up...
Cleaning up...
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_mark/html5lib
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_mark/html5lib
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_mark/html5lib
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_mark/html5lib
Storing debug log for failure in /astro/mark/.pip/pip.log
Storing debug log for failure in /astro/mark/.pip/pip.log
Storing debug log for failure in /astro/mark/.pip/pip.log
Storing debug log for failure in /astro/mark/.pip/pip.log

[keflavich]

Well, I can reproduce the weird repeating error - it seems that each time I repeat a pip command, it prints all results of that command N times, where N is the number of times I previously called the pip command. Don't know why...

I can also reproduce the install failure:

CASA <4>: pip.main(['install', 'html5lib', '--user'])
Downloading/unpacking html5lib
  Downloading html5lib-0.999999999.tar.gz (245kB): 245kB downloaded
  Running setup.py (path:/private/var/folders/_r/c4nkvmpn4vx6qc91qygz8h740000gr/T/pip_build_adam/html5lib/setup.py) egg_info for package html5lib
    html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 14.3)
    Complete output from command python setup.py egg_info:
    html5lib requires setuptools version 18.5 or above; please upgrade before installing (you have 14.3)

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /private/var/folders/_r/c4nkvmpn4vx6qc91qygz8h740000gr/T/pip_build_adam/html5lib
Storing debug log for failure in /Users/adam/.pip/pip.log
  Out[4]: 1

though I have a more recent version. I can't parse the setuptools versions, though - it looks like I have 3.4.3, not 4.3 as claimed by html5lib:

CASA <9>: import setuptools

CASA <10>: setuptools.__path__
  Out[10]: ['/Users/adam/.casa/lib/python2.6/site-packages/setuptools-3.4.3-py2.6.egg/setuptools']

[mknapp55]

I have exactly the same issue - do you have any suggested fix or workaround?
Edit: I was able to install astroquery by completely removing setuptools. Setuptools are then reinstalled when astroquery is installed.

[keflavich]

@mknapp55 thanks for the workaround! It's unfortunate that such a hack is needed; I still don't understand what causes this problem.

[astromark]

I was sure I'd tried that already, but, yes, it is working now. Thanks @mknapp55 !!