Install Python 3.5.2 from source on Mac
Python
Install latest stable version Python 3.5.2 on Mac.
Seeing official site [1]. However as I was not able to download with Mercurial (hg command), that is on the official site, get the source from git mirror. [2]
Install from Git source
$ git clone git@github.com:python/cpython.git $ cd cpython
As git's tags are not set in the repo, check target 3.5.2 commit hash manually.
$ git branch -r $ git checkout 3.5 $ git log
Then I found it.
commit 7aafe3c4fbff73590de183f095f1d32feeadc450
Author: Larry Hastings <larry@hastings.org>
Date: Sat Jun 25 14:11:29 2016 -0700
Added tag v3.5.2 for changeset 4def2a2901a5
$ git checkout 7aafe3c4fbff73590de183f095f1d32feeadc450
$ ./configure --prefix=/usr/local/python-3.5.2 --with-pydebug $ make -j $ sudo make install ... Successfully installed pip-8.1.1 setuptools-20.10.1 $ /usr/local/python-3.5.2/bin/python3 --version Python 3.5.2
Only python3 (python3.5) is installed command.
python is not installed.
$ ls /usr/local/python-3.5.2/bin/python ls: /usr/local/python-3.5.2/bin/python: No such file or directory
Add its bin path to PATH at .bashrc.
$ which python3 /usr/local/python-3.5.2/bin/python3
Install from Mercurial source
It do not work.
$ cd ~/hg $ hg clone https://hg.python.org/cpython destination directory: cpython requesting all changes adding changesets adding manifests adding file changes transaction abort! rollback completed abort: stream ended unexpectedly (got 222273 bytes, expected 1635000436)
pip
Use pip3 as well as python3. And pip is also available?
$ which pip /usr/local/python-3.5.2/bin/pip $ which pip3 /usr/local/python-3.5.2/bin/pip3
$ pip3 --version pip 8.1.1 from /usr/local/python-3.5.2/lib/python3.5/site-packages (python 3.5) $ sudo pip3 install --upgrade pip $ pip3 --version pip 9.0.1 from /usr/local/python-3.5.2/lib/python3.5/site-packages (python 3.5)
Create own configuration file.
$ vi ~/.pip/pip.conf [list] format=columns
$ pip3 list Package Version ---------- ------- pip 9.0.1 setuptools 20.10.1
Commands
There are key commands for pip.
$ pip3 --help
Usage:
pip <command> [options]
Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
search Search PyPI for packages.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
help Show help for commands.
General Options:
-h, --help Show help.
--isolated Run pip in an isolated mode, ignoring environment variables
and user configuration.
-v, --verbose Give more output. Option is additive, and can be used up to
3 times.
-V, --version Show version and exit.
-q, --quiet Give less output. Option is additive, and can be used up to
3 times (corresponding to WARNING, ERROR, and CRITICAL
logging levels).
--log <path> Path to a verbose appending log.
--proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port.
--retries <retries> Maximum number of retries each connection should attempt
(default 5 times).
--timeout <sec> Set the socket timeout (default 15 seconds).
--exists-action <action> Default action when a path already exists: (s)witch,
(i)gnore, (w)ipe, (b)ackup, (a)bort.
--trusted-host <hostname> Mark this host as trusted, even though it does not have
valid or any HTTPS.
--cert <path> Path to alternate CA bundle.
--client-cert <path> Path to SSL client certificate, a single file containing the
private key and the certificate in PEM format.
--cache-dir <dir> Store the cache data in <dir>.
--no-cache-dir Disable the cache.
--disable-pip-version-check
Don't periodically check PyPI to determine whether a new
version of pip is available for download. Implied with --no-
index.
Install
$ sudo pip3 install numpy
You may need to use absolute path for pip3. I am faced it on Linux environment.
$ sudo /usr/local/python-3.5.2/bin/pip3 install numpy
Unstall
$ sudo pip3 uninstall numpy
Download
This is similar with Rubygem's gem fetch gem_name
$ sudo pip3 download numpy
Freeze
$ pip3 freeze -h
Freeze Options:
-r, --requirement <file> Use the order in the given requirements file and its
comments when generating output. This option can be used
multiple times.
-f, --find-links <url> URL for finding packages, which will be added to the output.
-l, --local If in a virtualenv that has global access, do not output
globally-installed packages.
--user Only output packages installed in user-site.
--all Do not skip these packages in the output: pip, setuptools,
distribute, wheel
$ pip3 freeze -l numpy==1.11.2
$ pip3 freeze --user $ pip3 freeze --all numpy==1.11.2 pip==9.0.1 setuptools==20.10.1
List
$ pip3 list --format=columns Package Version ---------- ------- numpy 1.11.2 pip 9.0.1 setuptools 20.10.1
$ pip3 list --format=legacy numpy (1.11.2) pip (9.0.1) setuptools (20.10.1)
$ pip3 list --format=freeze biopython==1.68 numpy==1.11.2 pip==9.0.1 setuptools==20.10.1
$ pip3 list --format=json
[{"name": "biopython", "version": "1.68"}, {"name": "numpy", "version": "1.11.2"}, {"name": "pip", "version": "9.0.1"}, {"name": "setuptools", "version": "20.10.1"}]
Show
$ pip3 show numpy Name: numpy Version: 1.11.2 Summary: NumPy: array processing for numbers, strings, records, and objects. Home-page: http://www.numpy.org Author: NumPy Developers Author-email: numpy-discussion@scipy.org License: BSD Location: /usr/local/python-3.5.2/lib/python3.5/site-packages Requires:
Check
$ pip3 check numpy No broken requirements found.
Search
Search PyPI for packages.
It searches from package name and the description.
$ pip3 search biopython
A3MIO (0.0.1) - A3M/A2M I/O for BioPython
bcbio-gff (0.6.4) - Read and write Generic Feature Format (GFF) with
Biopython integration.
Bio_Eutils (1.65) - Standalone version of the Biopython Eutils modules
bioext (0.9.5) - Misc utilities and definitions not included or hidden in
biopython
biopython-extensions (0.18.5) - Misc utilities and definitions not included or hidden in
BioPython
biopython (1.68) - Freely available tools for computational molecular
biology.
INSTALLED: 1.68 (latest)
bioscripts.convert (0.4) - Biopython scripts for converting molecular sequences
between formats.
gbgb (0.1.0) - Goodbye, GenBank is a package for use with Biopython that
gives feature annotations from GenBank records a new and
better life.
Wheel
Build wheels from your requirements.
Like "Build gemspec file from required gem dependency packages"?
Tried with a python package. But it had build error.
$ git clone git@github.com:rebase-helper/rebase-helper.git
$ cd rebase-helper
$ pip3 wheel -r requirements.txt
ERROR: 'pip wheel' requires the 'wheel' package. To fix this, run: pip install wheel
$ sudo pip3 install wheel
$ pip3 list | grep wheel
wheel 0.29.0
$ pip3 wheel -r requirements.txt
...
Building wheels for collected packages: backports.lzma, copr, lxml
Running setup.py bdist_wheel for backports.lzma ... error
...
gcc -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -I/Users/jun.aruga/include -I/opt/local/include -I/usr/local/include -I/usr/local/python-3.5.2/include/python3.5dm -c backports/lzma/_lzmamodule.c -o build/temp.macosx-10.9-x86_64-3.5-pydebug/backports/lzma/_lzmamodule.o
backports/lzma/_lzmamodule.c:115:10: fatal error: 'lzma.h' file not found
#include <lzma.h>
^
1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for backports.lzma
...
Hash
Umm.
$ pip3 hash -h Usage: pip hash [options] <file> ... Description: Compute a hash of a local package archive. These can be used with --hash in a requirements file to do repeatable installs.
virtualenv
virtualenv is a tool to create isolated Python environments. It is like ruby's Bundler, rbenv, RVM, perl's Carton.
Install and use virtualenv.
See the doc [3].
Install
$ sudo pip3 install virtualenv $ pip3 list | grep virtualenv virtualenv 15.1.0
How to use
Create environment for each python version.
Create the directory to keep scripts for virtualenv.
$ mkdir ~/virtualenv
Own setuptools, pip and wheel are installed in the environment.
$ virtualenv --python=/usr/local/python-3.5.2/bin/python3 ~/virtualenv/python3.5.2 Running virtualenv with interpreter /usr/local/python-3.5.2/bin/python3 Using base prefix '/usr/local/python-3.5.2' New python executable in /Users/jun.aruga/virtualenv/python3.5.2/bin/python3 Also creating executable in /Users/jun.aruga/virtualenv/python3.5.2/bin/python Installing setuptools, pip, wheel.../usr/local/python-3.5.2/lib/python3.5/site-packages/virtualenv.py:901: ResourceWarning: unclosed file <_io.BufferedReader name=6> call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT) done.
There are 2 kind of managing the environment. For example, Ruby rbenv, RVM have the environment for each ruby version. But bundler has it for each project.
I prefer each project than each version.
$ source ~/virtualenv/python3.5.2/bin/activate (python3.5.2) $ (python3.5.2) $ which pip /Users/jun.aruga/virtualenv/python3.5.2/bin/pip (python3.5.2) $ which python /Users/jun.aruga/virtualenv/python3.5.2/bin/python (python3.5.2) $ pip list Package Version ---------- ------- pip 9.0.1 setuptools 29.0.1 wheel 0.29.0
Create environment for each python project.
I prefer this way.
$ virtualenv --python=`which python3` `pwd`/vendor/virtualenv
See this article for detail: Think about the Python environment like Ruby Bundler - Another Japan in the World
tox
Tox is a generic virtualenv management and test command line tool.
See the document [4].
It is like bundle exec rake test.
setup.py
It is like Makefile.
