HTTPS certificate verification failed

[JasonQSY]

When we try to connect MTurk, i.e. type

./manage.py mtbalance

python shows that HTTPS certificate verification failed.

Fetching account balance...
Account balance:
Traceback (most recent call last):
  File "./manage.py", line 14, in <module>
    execute_from_command_line(sys.argv)
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/jason/Desktop/opensurfaces/server/mturk/management/commands/mtbalance.py", line 25, in handle
    print 'Account balance:', get_mturk_connection().get_account_balance()
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/boto-2.6.0.dev0-py2.7.egg/boto/mturk/connection.py", line 69, in get_account_balance
    ('OnHoldBalance', Price)])
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/boto-2.6.0.dev0-py2.7.egg/boto/mturk/connection.py", line 762, in _process_request
    response = self.make_request(request_type, params, verb='POST')
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/boto-2.6.0.dev0-py2.7.egg/boto/connection.py", line 952, in make_request
    return self._mexe(http_request)
  File "/home/jason/Desktop/opensurfaces/venv/local/lib/python2.7/site-packages/boto-2.6.0.dev0-py2.7.egg/boto/connection.py", line 872, in _mexe
    raise e
ssl.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

It looks like python's the root certificate is too old for MTurk. I try to upgrade the certifi package by

pip install git+https://github.com/certifi/python-certifi --upgrade

But it does not solve the problem. One solution is just disable SSL verification for boto, but it is not secure for production. Anyone has some idea?

Related issues:

• https://stackoverflow.com/questions/38835270/i-get-certificate-verify-failed-when-i-try-to-install-the-spacy-english-language
• SSL3_GET_SERVER_CERTIFICATE: certificate verify failed uskudnik/amazon-glacier-cmd-interface#180

[mjpvz]

Did you run the scripts/install_ssh.sh? This should take care of it.

I don't believe it is called by the install_all.sh.

[JasonQSY]

I think that's not the same thing. install_ssh.sh is to acquire a certificate for the server. Here the certificate cannot be verified is the MTurk's.

[paulu]

The reason is that boto 2.6.0's certificate authorities list is too old. The fix is to copy the more recent CA list from certifi into boto. (An alternative fix is to update boto, but then you would have to check that this doesn't introduce bugs.)

cp -a ../venv/local/lib/python2.7/site-packages/certifi/cacert.pem ../venv/local/lib/python2.7/site-packages/boto-2.6.0.dev0-py2.7.egg/boto/cacerts/cacerts.txt

[JasonQSY]

Exactly. I think it works for me. I'll check if it introduces other bugs. Thank you very much!