Another Japan in the World

Jun Aruga's blog, who lived in Japan, Singapore, live in Czech Republic. Mainly about IT and travel.

Install TensorFlow from source on Fedora 27

I tried to install TnsorFlow from source seeing the official page [1].

The source is on GitHub [2].

Install build dependency RPM packages

$ sudo dnf install python-devel

Install a build tool Bazel.

Bazel is necessary to build TensorFlow from the source.

I did it seeing [3]. It is not registered on the Fedora official repository.

$ sudo dnf copr enable vbatts/bazel

$ sudo dnf install bazel

$ rpm -q bazel
bazel-0.12.0-1.fc27.x86_64

Build TensorFlow from the source

This time I tried to install it to virtualenv Python environment.

$ git clone git@github.com:tensorflow/tensorflow.git

$ cd tensorflow

$ python3 --version
Python 3.6.2

$ python3 -m venv ./venv

It seems that the configure script does not have options. At least I could not see those by ./configure --help. Maybe there is another way to specify the options. Answering Yes/No interactively is not so easy. This time I did as default typing Enter key.

(venv) $ ./configure

Invalid python path: y cannot be found.
Please specify the location of python. [Default is /home/jaruga/git/tensorflow/venv/bin/python]:

Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]: 
jemalloc as malloc support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: 
Google Cloud Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: 
Hadoop File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: 
Amazon S3 File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Apache Kafka Platform support? [Y/n]: 
Apache Kafka Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [y/N]: 
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GDR support? [y/N]: 
No GDR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with VERBS support? [y/N]: 
No VERBS support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: 
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: 
No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]: 
Clang will not be downloaded.

Do you wish to build TensorFlow with MPI support? [y/N]: 
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: 

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.
  --config=mkl          # Build with MKL support.
  --config=monolithic   # Config for mostly static monolithic build.
Configuration finished

Install the build dependency.

(venv) $ pip install -U six numpy wheel

Build TensorFlow. Below "bazel build" command creates build_pip_package script file. The build time is really long. It took 8993 sec (= 2h 29 min).

(venv) $ bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
...
INFO: Elapsed time: 8993.363s, Critical Path: 116.11s
INFO: Build completed successfully, 8971 total actions

Below command creates package file by created script file build_pip_package. There are many warnings. This step might have issue on my environment.

(venv) $ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Sat Apr 21 00:01:24 CEST 2018 : === Using tmpdir: /tmp/tmp.x0LXyVKURO
~/git/tensorflow/bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles ~/git/tensorflow
~/git/tensorflow
/tmp/tmp.x0LXyVKURO ~/git/tensorflow
Sat Apr 21 00:01:25 CEST 2018 : === Building wheel
warning: no files found matching '*.dll' under directory '*'
warning: no files found matching '*.lib' under directory '*'
warning: no files found matching '*' under directory 'tensorflow/aux-bin'
warning: no files found matching '*.h' under directory 'tensorflow/include/tensorflow'
warning: no files found matching '*' under directory 'tensorflow/include/Eigen'
warning: no files found matching '*' under directory 'tensorflow/include/external'
warning: no files found matching '*.h' under directory 'tensorflow/include/google'
warning: no files found matching '*' under directory 'tensorflow/include/third_party'
warning: no files found matching '*' under directory 'tensorflow/include/unsupported'
~/git/tensorflow
Sat Apr 21 00:01:36 CEST 2018 : === Output wheel file is in: /tmp/tensorflow_pkg

Below command pip install installs TensorFlow by created package file tensorflow-1.8.0rc0-cp36-cp36m-linux_x86_64.whl.

(venv) $ pip install /tmp/tensorflow_pkg/tensorflow-1.8.0rc0-cp36-cp36m-linux_x86_64.whl
...
Successfully installed absl-py-0.2.0 astor-0.6.2 gast-0.2.0 grpcio-1.11.0 tensorboard-1.7.0 tensorflow-1.8.0rc0 termcolor-1.1.0
(venv) $ pip list | grep tensorflow
tensorflow             1.8.0rc0

Run a script to verify

Below command to verify was failed.

(venv) $ python -c "
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
"

Traceback (most recent call last):
  File "/home/jaruga/git/tensorflow/tensorflow/python/platform/self_check.py", line 25, in <module>
    from tensorflow.python.platform import build_info
ImportError: cannot import name 'build_info'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 2, in <module>
  File "/home/jaruga/git/tensorflow/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/home/jaruga/git/tensorflow/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/home/jaruga/git/tensorflow/tensorflow/python/pywrap_tensorflow.py", line 25, in <module>
    from tensorflow.python.platform import self_check
  File "/home/jaruga/git/tensorflow/tensorflow/python/platform/self_check.py", line 27, in <module>
    raise ImportError("Could not import tensorflow. Do not import tensorflow "
ImportError: Could not import tensorflow. Do not import tensorflow from its source directory; change directory to outside the TensorFlow source tree, and relaunch your Python interpreter from there.

Ah the error message shows the reason of the error clearly.

Do not import tensorflow from its source directory; change directory to outside the TensorFlow source tree,
(venv) $ cd /tmp

(venv) $ python -c "
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
"
b'Hello, TensorFlow!'

Okay!!

Install TensorFlow on Fedora 27

This article is about how to install TensorFlow binary package on Fedora Linux. I referred the install document for Ubuntu Linux on official page [1]

$ cat /etc/fedora-release 
Fedora release 27 (Twenty Seven)

$ python3 --version
Python 3.6.2

$ cd YOUR_PROJECT_DIR

$ python3 -m venv ./venv

$ source venv/bin/activate

(venv) $ easy_install -U pip

(venv) $ pip list
Package    Version
---------- -------
pip        10.0.1
setuptools 28.8.0

$ pip install --upgrade tensorflow

(venv) $ pip list
Package     Version    
----------- -----------
absl-py     0.2.0      
astor       0.6.2      
bleach      1.5.0      
gast        0.2.0      
grpcio      1.11.0      
html5lib    0.9999999  
Markdown    2.6.11     
numpy       1.14.2     
pip         10.0.1     
protobuf    3.5.2.post1
setuptools  28.8.0     
six         1.11.0     
tensorboard 1.7.0      
tensorflow  1.7.0      
termcolor   1.1.0      
Werkzeug    0.14.1     
wheel       0.31.0  

Run a script to verify.

(venv) $ python -c "
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
"

2018-04-20 23:34:27.043810: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
b'Hello, TensorFlow!'

The warning is not to support CPU extension AVX [2] and FMA [3]. It is possible to suppress the warning by setting a environment variable according [4].