TB3 clCaffe

Status

Done

Tested with caffe device_query and benchmarking the Alexnet example.

Reference

clCaffe SDK

Note: If the installation of any of these components goes wrong, remove its build directory and start over with its creation. There is no cmake clean operation. It is best to start over.

A Note

This step is misleadingly involved. It is also finicky. It took me several trials to get it right. Once done, it doesn't appear that difficult. It is easy to miss a detail that puts you on a bad path. Furthermore, there is a lot of glue here to stitch together components from diverse environments, for example, ROS catkin, classic Ubuntu, rpm, Debian, and OpenCL. Furthermore, its not clear how many of the support items are needed. I found I needed some of "Inference Optimization" and "YOLO2 Model Support".

Pre Install Space

Summer 2018

 $ df
     Filesystem     1K-blocks     Used Available Use% Mounted on
     /dev/sda7       92535808 19411140  68401008  23% /

February 2019 its 19% before install

OpenCL drivers for Intel platforms on Linux

[Downloaded http://registrationcenter-download.intel.com/akdlm/irc_nas/11396/SRB4.1_linux64.zip OpenCl SDK 4.1]

 $ cd

 $ mkdir opencl.4.1

 $ cd opencl.4.1/

 $ unzip ../Downloads/SRB4.1_linux64.zip

STOP Dont use rpm below

 $ ll *rpm
     -rw-rw-r-- 1 eepp eepp  8199300 Mar 13  2017 intel-opencl-cpu-r4.1-61547.x86_64.rpm
     -rw-rw-r-- 1 eepp eepp    74416 Mar 13  2017 intel-opencl-devel-r4.1-61547.x86_64.rpm
     -rw-rw-r-- 1 eepp eepp 24763688 Mar 13  2017 intel-opencl-r4.1-61547.x86_64.rpm

Using rpm does not appear to be a good choice.

GO I used Alien instead. The steps are:

Use Alien to convert the rpm to deb.
Then use dpkg to install the deb
How to install an rpm package on Ubuntu from HowToGeek

 $ sudo apt-get install alien dpkg-dev debhelper build-essential

 $ sudo alien intel-opencl-r4.1-61547.x86_64.rpm
 $ sudo dpkg -i intel-opencl_0r4.1-61548_amd64.deb

 $ sudo alien intel-opencl-devel-r4.1-61547.x86_64.rpm
 $ sudo dpkg -i intel-opencl-devel_0r4.1-61548_amd64.deb

 $ sudo alien intel-opencl-cpu-r4.1-61547.x86_64.rpm
 $ sudo dpkg -i intel-opencl-cpu_0r4.1-61548_amd64.deb

Compile & Install Prerequisites for Ubuntu

 $ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
 
 $ sudo apt-get install --no-install-recommends libboost-all-dev

 $ sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev

 $ sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

vienacl

 $ mkdir -p $HOME/code

 $ cd $HOME/code

 $ git clone https://github.com/viennacl/viennacl-dev.git

 $ cd viennacl-dev

 $ mkdir build && cd build

 $ cmake -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DCMAKE_INSTALL_PREFIX=$HOME/local -DOPENCL_LIBRARY=/opt/intel/opencl/libOpenCL.so ..

 $ make -j4
     ...
     [100%] Linking CXX shared library libviennacl.so
     cd /home/eepp/code/viennacl-dev/build/libviennacl && /usr/bin/cmake -E cmake_link_script CMakeFiles/viennacl.dir/link.txt --verbose=1
     /usr/bin/c++  -fPIC   -shared -Wl,-soname,libviennacl.so -o libviennacl.so CMakeFiles/viennacl.dir/src/backend.cpp.o CMakeFiles/viennacl.dir       /src/blas1.cpp.o CMakeFiles/viennacl.dir/src/blas1_host.cpp.o CMakeFiles/viennacl.dir/src/blas1_opencl.cpp.o CMakeFiles/viennacl.dir/src/blas2.cpp.o CMakeFiles/viennacl.dir/src/blas2_host.cpp.o CMakeFiles/viennacl.dir/src/blas2_opencl.cpp.o CMakeFiles/viennacl.dir/src/blas3.cpp.o CMakeFiles/viennacl.dir/src/blas3_host.cpp.o CMakeFiles/viennacl.dir/src/blas3_opencl.cpp.o /opt/intel/opencl/libOpenCL.so -Wl,-rpath,/opt/intel/opencl 
     make[2]: Leaving directory '/home/eepp/code/viennacl-dev/build'
     [100%] Built target viennacl
     make[1]: Leaving directory '/home/eepp/code/viennacl-dev/build'
     /usr/bin/cmake -E cmake_progress_start /home/eepp/code/viennacl-dev/build/CMakeFiles 0

 $ make install

isaac

 $ cd $HOME/code

 $ git clone https://github.com/intel/isaac

 $ cd isaac

 $ mkdir build && cd build

 $ cmake -DCMAKE_INSTALL_PREFIX=$HOME/local ..

 $ make -j4
     ....
     [ 90%] Linking CXX executable test-reduce-2d
     [ 90%] Built target test-reduce-2d
     Scanning dependencies of target bench-blas
     [ 91%] Building CXX object bench/CMakeFiles/bench-blas.dir/blas.cpp.o
     [ 93%] Linking CXX executable test-reduce-1d
     [ 93%] Built target test-reduce-1d
     Scanning dependencies of target example-indexing
     [ 94%] Building CXX object examples/CMakeFiles/example-indexing.dir/indexing.cpp.o
     [ 95%] Linking CXX executable test-fusion
     [ 95%] Built target test-fusion
     [ 97%] Linking CXX executable example-indexing
     [ 97%] Built target example-indexing
     [ 98%] Linking CXX executable test-element-2d
     [ 98%] Built target test-element-2d
     [100%] Linking CXX executable bench-blas
     [100%] Built target bench-blas

 $ make install

mkl

mlk tgz from Intel
The following ended up in the Downloads folder: l_mkl_2019.2.187.tgz (Summer 2018 l_mkl_2018.3.222.tgz)
I had problems related to running the install.
- Run install_GUI.sh (not install.sh). This interface is clearer.
- Running a root put all the libraries and other collateral in the correct place. The paths ended up being as described by the clCaffe installation in the following section.

First create a root passwd so I could run the install script as root.

 $ sudo passwd root
     [sudo] password for eepp: 
     Enter new UNIX password: 
     Retype new UNIX password: 
     passwd: password updated successfully

Now install

 $ cd

 $ mkdir mkl

 $ cd mkl

 ## summer 2018 $ tar xzvf ../Downloads/l_mkl_2018.3.222.tgz

 $ tar xzvf ../../Downloads/l_mkl_2019.2.187.tgz

 $ cd l_mkl_2019.2.187/

 $ su root
  Password:

 $ . install_GUI.sh

 $ ls /opt/intel/mkl/lib/intel64_lin/
     libmkl_avx2.so         libmkl_gnu_thread.so     libmkl_tbb_thread.a
     libmkl_avx512_mic.so   libmkl_intel_ilp64.a     libmkl_tbb_thread.so
     libmkl_avx512.so       libmkl_intel_ilp64.so    libmkl_vml_avx2.so
     libmkl_avx.so          libmkl_intel_lp64.a      libmkl_vml_avx512_mic.so
     libmkl_blas95_ilp64.a  libmkl_intel_lp64.so     libmkl_vml_avx512.so
     libmkl_blas95_lp64.a   libmkl_intel_thread.a    libmkl_vml_avx.so
     libmkl_core.a          libmkl_intel_thread.so   libmkl_vml_cmpt.so
     libmkl_core.so         libmkl_lapack95_ilp64.a  libmkl_vml_def.so
     libmkl_def.so          libmkl_lapack95_lp64.a   libmkl_vml_mc2.so
     libmkl_gf_ilp64.a      libmkl_mc3.so            libmkl_vml_mc3.so
     libmkl_gf_ilp64.so     libmkl_mc.so             libmkl_vml_mc.so
     libmkl_gf_lp64.a       libmkl_rt.so             locale
     libmkl_gf_lp64.so      libmkl_sequential.a
     libmkl_gnu_thread.a    libmkl_sequential.so

 $ exit

Install steps by window

"Install as root"
Entered password
Accepted license
Pressed "Install" button after viewing Summary
Pressed "Finish" button in the Complete window

TBD: Add summary snapshot ~/Picture/ Install puts mkl in /opt/intel. The only thing there before was opencl.

clCaffe

 $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/mkl/lib/intel64_lin/

 $ cd ~/code

 $ git clone https://github.com/01org/caffe clCaffe

 $ cd clCaffe

 $ git checkout inference-optimize
     clCaffe$ git checkout inference-optimize
     Already on 'inference-optimize'
     Your branch is up-to-date with 'origin/inference-optimize'.

 $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/mkl/lib/intel64_lin/

 $ mkdir build && cd build

 $ export ISAAC_HOME=$HOME/local
 
 $ cmake .. -DUSE_GREENTEA=ON -DUSE_CUDA=OFF -DUSE_INTEL_SPATIAL=ON -DBUILD_docs=0 -DUSE_ISAAC=ON -DViennaCL_INCLUDE_DIR=$HOME/local/include -DBLAS=mkl -DOPENCL_LIBRARIES=/opt/intel/opencl/libOpenCL.so -DOPENCL_INCLUDE_DIRS=/opt/intel/opencl/include
     ...
     -- Generating done
     -- Build files have been written to: /home/eepp/code/clCaffe/build

 $ make -j4
     ...
     In file included from /home/eepp/code/clCaffe/include/caffe/greentea/greentea.hpp:30:0,
                      from /home/eepp/code/clCaffe/include/caffe/common.hpp:25,
                      from /home/eepp/code/clCaffe/include/caffe/blob.hpp:8,
                      from /home/eepp/code/clCaffe/include/caffe/caffe.hpp:7,
                      from /home/eepp/code/clCaffe/python/caffe/_caffe.cpp:16:
     /opt/intel/opencl/include/CL/cl.h:1436:1: note: declared here
      clEnqueueTask(cl_command_queue  /* command_queue */,
     ^
     [100%] Linking CXX shared library ../lib/_caffe.so
     Creating symlink /home/eepp/code/clCaffe/python/caffe/_caffe.so -> /home/eepp/code/clCaffe/build/lib/_caffe.so
     [100%] Built target pycaffe

 $ export CAFFE_ROOT=$HOME/code/clCaffe

 $ ./tools/caffe device_query -gpu all
     I0705 16:11:47.471750 12530 common.cpp:556] Total devices: 2
     I0705 16:11:47.471897 12530 common.cpp:557] CUDA devices: 0
     I0705 16:11:47.471906 12530 common.cpp:558] OpenCL devices: 2
     I0705 16:11:47.471911 12530 common.cpp:582] Device id:                     0
     I0705 16:11:47.471916 12530 common.cpp:584] Device backend:                OpenCL
     I0705 16:11:47.471932 12530 common.cpp:586] Backend details:               Intel(R) Corporation: OpenCL 2.0 
     I0705 16:11:47.471941 12530 common.cpp:588] Device vendor:                 Intel(R) Corporation
     I0705 16:11:47.471946 12530 common.cpp:590] Name:                          Intel(R) HD Graphics
     I0705 16:11:47.471951 12530 common.cpp:592] Total global memory:           6573984973
     I0705 16:11:47.471958 12530 common.cpp:582] Device id:                     1
     I0705 16:11:47.471962 12530 common.cpp:584] Device backend:                OpenCL
     I0705 16:11:47.471971 12530 common.cpp:586] Backend details:               Intel(R) Corporation: OpenCL 2.0 
     I0705 16:11:47.471982 12530 common.cpp:588] Device vendor:                 Intel(R) Corporation
     I0705 16:11:47.471987 12530 common.cpp:590] Name:                          Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
     I0705 16:11:47.472146 12530 common.cpp:592] Total global memory:           8227966976

Cool

Now add the CAFFE_ROOT export to the .bashrc file.

 $ ./tools/caffe time -model ../models/bvlc_alexnet/deploy.prototxt -gpu 0

TB3 clCaffe Benchmarking the Alexnet Example

Double Cool

Opens

TBD Not sure what else is needed besides the following.

Inference Optomization

 $ sudo pip install scikit-image

 $ sudo pip install protobuf

 $ export PYTHONPATH=$PYTHONPATH:/opt/clCaffe/python/

Add this to the ~/.bashrc file.

NOT DONE Summet 2018

Repeated February 2019

Seems to be required for: TB3_ROS_OpenCL_Caffe#Environment_Setup

YOLO2 Model Support

 $ cd ~/code/clCaffe

 $wget https://pjreddie.com/media/files/yolo-voc.weights -O models/yolo/yolo416/yolo.weights

 $ python models/yolo/convert_yolo_to_caffemodel.py

 $ cd models/yolo/

 $ python ../../tools/inference-optimize/model_fuse.py --indefinition yolo416/yolo_deploy.prototxt --outdefinition yolo416/fused_yolo_deploy.prototxt --inmodel yolo416/yolo.caffemodel --outmodel yolo416/fused_yolo.caffemodel

Interesting: Darknet ROS

Repeated February 2019

Seems to be required for: TB3_ROS_OpenCL_Caffe#Environment_Setup

Current Install Space

Summer 2018

 $ df
     Filesystem     1K-blocks     Used Available Use% Mounted on
     ...
     /dev/sda7       92535808 27649300  60162848  33% /

31% February 2019

TB3 clCaffe

Contents

Status

Reference

A Note

Pre Install Space

OpenCL drivers for Intel platforms on Linux

Compile & Install Prerequisites for Ubuntu

vienacl

isaac

mkl

clCaffe

Opens

Inference Optomization

YOLO2 Model Support

Current Install Space

Navigation menu

Personal tools

Namespaces

Variants

Views

Actions

Search

Zdome

Navigation

Tools