Failure installing Eucalyptus: ffi.h: No such file or directory

Problem

Trying to install Eucalyptus, it fails with a message about ffi.h:

TASK [edx_ansible : Install edx_ansible venv requirements] *********************
(decoded....)
Collecting ansible==2.2.0.0 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 1))
Collecting PyYAML==3.12 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 2))
Collecting Jinja2==2.8 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 3))
  Using cached Jinja2-2.8-py2.py3-none-any.whl
Collecting MarkupSafe==0.23 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 4))
Collecting boto==2.33.0 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 5))
  Using cached boto-2.33.0-py2.py3-none-any.whl
Collecting ecdsa==0.11 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 6))
Collecting paramiko==2.0.2 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
  Using cached paramiko-2.0.2-py2.py3-none-any.whl
Collecting pycrypto==2.6.1 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 8))
Requirement already satisfied (use --upgrade to upgrade): wsgiref==0.1.2 in /usr/lib/python2.7 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 9))
Collecting docopt==0.6.1 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 10))
Collecting python-simple-hipchat==0.2 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 11))
Collecting prettytable==0.7.2 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 12))
Collecting awscli==1.11.58 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached awscli-1.11.58-py2.py3-none-any.whl
Collecting requests==2.9.1 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 14))
  Using cached requests-2.9.1-py2.py3-none-any.whl
Collecting datadog==0.8.0 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 15))
Collecting networkx==1.11 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 16))
  Using cached networkx-1.11-py2.py3-none-any.whl
Collecting pathlib2==2.1.0 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 17))
  Using cached pathlib2-2.1.0-py2.py3-none-any.whl
Collecting pymongo==3.1 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 18))
Collecting MySQL-python==1.2.5 (from -r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from ansible==2.2.0.0->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Collecting cryptography>=1.1 (from paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
  Using cached cryptography-1.8.1.tar.gz
Collecting botocore==1.5.21 (from awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached botocore-1.5.21-py2.py3-none-any.whl
Collecting s3transfer<0.2.0,>=0.1.9 (from awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached s3transfer-0.1.10-py2.py3-none-any.whl
Collecting colorama<=0.3.7,>=0.2.5 (from awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached colorama-0.3.7-py2.py3-none-any.whl
Collecting docutils>=0.10 (from awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached docutils-0.13.1-py2-none-any.whl
Collecting rsa<=3.5.0,>=3.1.2 (from awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting simplejson>=2.0.9 (from datadog==0.8.0->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 15))
Collecting decorator>=3.3.2 (from datadog==0.8.0->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 15))
  Using cached decorator-4.0.11-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from pathlib2==2.1.0->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 17))
Requirement already satisfied (use --upgrade to upgrade): idna>=2.1 in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): asn1crypto>=0.21.0 in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): packaging in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): enum34 in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): ipaddress in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Collecting cffi>=1.4.1 (from cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
  Using cached cffi-1.10.0.tar.gz
Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.5.21->awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached jmespath-0.9.2-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.5.21->awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached python_dateutil-2.6.0-py2.py3-none-any.whl
Collecting futures<4.0.0,>=2.2.0 (from s3transfer<0.2.0,>=0.1.9->awscli==1.11.58->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 13))
  Using cached futures-3.1.1-py2-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): pyparsing in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from packaging->cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): pycparser in /edx/app/edx_ansible/venvs/edx_ansible/lib/python2.7/site-packages (from cffi>=1.4.1->cryptography>=1.1->paramiko==2.0.2->-r /edx/app/edx_ansible/edx_ansible/requirements.txt (line 7))
Building wheels for collected packages: cryptography, cffi
  Running setup.py bdist_wheel for cryptography
  Complete output from command /edx/app/edx_ansible/venvs/edx_ansible/bin/python -c "import setuptools;__file__='/tmp/pip-build-6pBvMp/cryptography/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpJ1wdR5pip-wheel-:
  c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
  compilation terminated.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-6pBvMp/cryptography/setup.py", line 335, in <module>
      **keywords_with_side_effects(sys.argv)
    File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
      _setup_distribution = dist = klass(attrs)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/dist.py", line 268, in __init__
      self.fetch_build_eggs(attrs['setup_requires'])
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
      replace_conflicting=True,
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 836, in resolve
      dist = best[req.key] = env.best_match(req, ws, installer)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1081, in best_match
      return self.obtain(req, installer)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1093, in obtain
      return installer(requirement)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/dist.py", line 380, in fetch_build_egg
      return cmd.easy_install(req)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 629, in easy_install
      return self.install_item(spec, dist.location, tmpdir, deps)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 659, in install_item
      dists = self.install_eggs(spec, download, tmpdir)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 842, in install_eggs
      return self.build_and_install(setup_script, setup_base)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1070, in build_and_install
      self.run_setup(setup_script, setup_base, args)
    File "/edx/app/edx_ansible/venvs/edx_ansible/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1058, in run_setup
      raise DistutilsError("Setup script exited with %s" % (v.args[0],))
  distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Running setup.py bdist_wheel for cffi
  Complete output from command /edx/app/edx_ansible/venvs/edx_ansible/bin/python -c "import setuptools;__file__='/tmp/pip-build-6pBvMp/cffi/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpHQU4a4pip-wheel-:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/cffi
  copying cffi/error.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/vengine_gen.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/__init__.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/api.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/ffiplatform.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/recompiler.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/commontypes.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/cparser.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/lock.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/verifier.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/model.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/_cffi_include.h -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/parse_c_type.h -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/_embedding.h -> build/lib.linux-x86_64-2.7/cffi
  running build_ext
  building '_cffi_backend' extension
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/c
  gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o
  c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
  compilation terminated.
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
Failed to build cryptography cffi
Installing collected packages: cffi, cryptography, paramiko, PyYAML, pycrypto, MarkupSafe, Jinja2, ansible, boto, ecdsa, docopt, python-simple-hipchat, prettytable, jmespath, python-dateutil, docutils, botocore, futures, s3transfer, colorama, rsa, awscli, requests, simplejson, decorator, datadog, networkx, pathlib2, pymongo, MySQL-python
  Running setup.py install for cffi
    Complete output from command /edx/app/edx_ansible/venvs/edx_ansible/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-6pBvMp/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-83AZna-record/install-record.txt --single-version-externally-managed --compile --install-headers /edx/app/edx_ansible/venvs/edx_ansible/include/site/python2.7/cffi:
    running install
    running build
    running build_py
    running build_ext
    building '_cffi_backend' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o
    c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------

Solution

This has been fixed on eucalyptus.master.  Instead of installing a numbered Eucalyptus release, use this:

export OPENEDX_RELEASE=open-release/eucalyptus.master