Difference between revisions of "Install Turtlebot3 Hood Waffle Foxy alsora ros2-tensorflow 20210615"

From wikidb
Jump to: navigation, search
(Created page with "= Documentation = * [https://github.com/alsora/ros2-tensorflow ROS 2 Tensorflow] * [https://www.tensorflow.org/install TensorFlow] TensorFlow site * [https://linuxhint.com/in...")
 
 
(106 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
= Documentation =
+
= Experience =
 +
 
 +
I started here with the Tensorflow requirements for Foxy and got into trouble. So I changed my strategy to a vanilla Tensorflow and other required components install. There may be overlap and the order my need fine tuning
 +
 
 +
= References =
 +
 
 +
== Alsora ==
 +
 
 +
Alsora was chosen because it appeared prominently in searches. I had to start someplace and then adjust as I became familiar with tensorFlow Object Detection.
  
 
* [https://github.com/alsora/ros2-tensorflow ROS 2 Tensorflow]
 
* [https://github.com/alsora/ros2-tensorflow ROS 2 Tensorflow]
−
* [https://www.tensorflow.org/install TensorFlow] TensorFlow site
+
 
−
* [https://linuxhint.com/install_tensorflow_ubuntu/ Install TensorFlow on Ubuntu] Linux Hint
+
It is also the only TensorFlow link provided by Awesome Robots reference page. Again, probably a good place to start.
−
* [https://medium.com/swlh/tutorial-how-to-install-tensorflow-2-0-on-ubuntu-20-04-10cf6fe28de1 Tutorial - How to Install TensorFlow 2.0 on Ubuntu 20.04] Medium
+
* [https://fkromer.github.io/awesome-ros2/ Awesome Robot Operating System 2 (ROS 2)]
 +
 
 +
Kandi - Explore reusable Libraries curated by Popular Use Cases
 +
 
 +
* [https://kandi.openweaver.com/python/alsora/ros2-tensorflow ros2-tensorflow | #Computer Vision | ROS2 nodes for computer vision tasks in Tensorflow]
 +
 
 +
Other Projects
 +
 
 +
* [https://alsora.github.io/projects/ Alberto Soragna Projects]
 +
 
 +
In
 +
 
 +
* '''TBD''' move [https://landof.dev/awesome/robot-operating-system-20/ Awesome Robot Operating System 2.0]
 +
* '''TBD''' move [https://awesomelists.net/42/1215-Application%20layer/7629-intel-ros2_intel_movidius_ncs intel / ros2_intel_movidius_ncs]
 +
 
 +
== Others ==
 +
 
 +
* [https://index.ros.org/r/darknet_ros/#foxy YOLO ROS: Real-Time Object Detection for ROS]
 +
* [https://pythonrepo.com/repo/Ar-Ray-code-YOLOX-ROS-python-deep-learning YOLOX + ROS(1, 2) object detection package]
 +
* [https://developer.nvidia.com/blog/implementing-robotics-applications-with-ros-2-and-ai-on-jetson-platform-2/ Implementing Robotics Applications with ROS 2 and AI on the NVIDIA Jetson Platform]
 +
 
 +
== Move ==
 +
 
 +
'''TBD''' Move the following to more appropriate page.
 +
 
 +
* '''move''' [https://linuxhint.com/install_tensorflow_ubuntu/ Install TensorFlow on Ubuntu] Linux Hint
 +
* '''move'''[https://medium.com/swlh/tutorial-how-to-install-tensorflow-2-0-on-ubuntu-20-04-10cf6fe28de1 Tutorial - How to Install TensorFlow 2.0 on Ubuntu 20.04] Medium
 +
 
 +
= Requirements =
 +
 
 +
== Required Dependencies ==
 +
 
 +
* Foxy: Installed
 +
 
 +
== Rosdep Dependencies ==
 +
 
 +
=== OpenCV Python ===
 +
 
 +
* [https://pypi.org/project/opencv-python/ pip install of opencv-python]
 +
 
 +
  pip -V
 +
      pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
 +
 
 +
v 19.3 is minimum.
 +
 
 +
  $ pip install opencv-python
 +
 
 +
It was already installed.
 +
 
 +
=== Tensorflow ===
 +
 
 +
* [[Hood Ubuntu Vanilla Tensorflow Install and Test 20210708]]
 +
 
 +
=== Vision Msgs ===
 +
 
 +
ROS Foxy's vision-msgs package is required. However, the ROS key had expired causing some confusion.
 +
 
 +
==== References ====
 +
 
 +
* [https://github.com/alsora/ros2-tensorflow ROS 2 Tensorflow] See Requirements section
 +
* [https://github.com/ros-perception/vision_msgs ROS Perception Vision Msgs]
 +
* [https://discourse.ros.org/t/ros-gpg-key-expiration-incident/20669 ROS GPG Key Expiration Incident] Carefully read Tiryoh's comment carefully from Kat Scott's posting.
 +
* [https://answers.ros.org/question/384358/cannot-install-any-more-foxy-or-ubuntu-packages/ Cannot Install Any More Foxy or Ubuntu Packages] My ticket
 +
 
 +
==== Install ROS Key ====
 +
 
 +
If your ROS key has expired
 +
 
 +
  $ curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
 +
 +
  $ sudo apt update
 +
 
 +
==== Install vission-msgs ====
 +
 
 +
$ sudo apt install ros-foxy-vision-msgs
 +
 
 +
== Optional Dependencies ==
 +
 
 +
=== Tensorflow Object Detection Models ===
 +
 
 +
I tried going to the following link given by Alsora. The link is broken.
 +
 
 +
* [https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md Object Detection Installation]
 +
 
 +
I may have installed the models when I was testing a Udacity Python3 script. See the "Executing Lesson 3 from the Udacity Course" section on page [[Hood Ubuntu Vanilla Tensorflow Install and Test 20210708]]
 +
 
 +
  $ pip install -U tensorflow_datasets
 +
 
 +
=== Tensorflow Slim ===
 +
 
 +
* [https://github.com/google-research/tf-slim Tensorflow Slim]
 +
 
 +
  $ pip install --upgrade tf_slim
 +
 
 +
=== Docker ===
 +
 
 +
To use Docker
 +
 
 +
[[Hood ROS 2 Foxy Docker 20210814]]
 +
 
 +
= Build =
 +
 
 +
== Create Foxy Tensoryflow work space ==
 +
 
 +
  $ git clone https://github.com/alsora/ros2-tensorflow.git $HOME/ros2-tensorflow
 +
        Cloning into '/home/eepp/ros2-tensorflow'...
 +
        ...
 +
 +
  $ mkdir -p $HOME/tf_ws/src
 +
 +
  $ cd $HOME/tf_ws
 +
 +
  $ ln -s $HOME/ros2-tensorflow/ros2-tensorflow src
 +
 
 +
== Install dependencies ==
 +
 
 +
  $ rosdep install --from-paths src --ignore-src --rosdistro foxy -y
 +
 
 +
* This resulted in errors. I found the resolution here:
 +
* [https://answers.ros.org/question/232795/cant-resolve-package-dependencies-when-building-ros-for-ubuntu-1604/ Can't resolve package dependencies when building ros for Ubuntu 16.04]
 +
 
 +
  $ rosdep install --from-paths src --ignore-src --rosdistro foxy -y -r
 +
 
 +
== Install Object Detection Models ==
 +
 
 +
  $ sudo apt-get install -y protobuf-compiler python-lxml python-tk
 +
        ...
 +
        ...
 +
 +
  $ pip install --user Cython contextlib2
 +
        ...
 +
       
 +
The next 3 installs were already satisfied. No additional action.
 +
 
 +
  $ pip install --user Cython jupyter
 +
 +
  $ pip install --user Cython matplotlib
 +
 +
  $ pip install --user Cython Pillow
 +
 
 +
The following failed.
 +
 
 +
  cd usr/local/lib/python3.8/dist-packages/tensorflow/models/research
 +
 
 +
Corrected
 +
 
 +
  cd /usr/local/lib/python3.8/dist-packages/tensorflow/models/research
 +
 
 +
The following failed.
 +
 
 +
  $ protoc object_detection/protos/*.proto --python_out=.
 +
 
 +
Corrected
 +
 
 +
  $ sudo protoc object_detection/protos/*.proto --python_out=.
 +
 
 +
Missed this the first time I installed
 +
 
 +
  $ echo 'export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.8/dist-packages/tensorflow/models/research' >> $HOME/.bashrc
 +
 +
  $ tail ~/.bashrc
 +
        export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.8/dist-packages/tensorflow/models/research
 +
 
 +
== Install tf_slim ==
 +
 
 +
tf_slim installed earlier. See Optional Dependencies above.
 +
 
 +
 
 +
== Build and install the ros2-tensorflow package ==
 +
 
 +
Double check the symbolic link
 +
 
 +
  $ cd
 +
 +
  $ ll tf_ws/src/
 +
        total 8
 +
        drwxrwxr-x 2 eepp eepp 4096 Aug 12 23:43 ./
 +
        drwxrwxr-x 3 eepp eepp 4096 Aug 12 23:43 ../
 +
        lrwxrwxrwx 1 eepp eepp  42 Aug 12 23:43 ros2-tensorflow -> /home/eepp/ros2-tensorflow/ros2-tensorflow/
 +
 
 +
  $ cd tf_ws
 +
 
 +
The following fails
 +
 
 +
  $ colcon build
 +
 +
* [https://answers.ros.org/question/364060/colcon-fails-to-build-python-package-error-in-egg_base Colcon fails to build Python package: "error in 'egg_base']
 +
 
 +
Corrected by using symlink-install switch
 +
 
 +
  $ colcon build --symlink-install
 +
        Starting >>> ros2_tf_core
 +
        Starting >>> tf_interfaces
 +
        Finished <<< ros2_tf_core [1.33s]                                           
 +
        Finished <<< tf_interfaces [11.9s]                     
 +
        Starting >>> tf_classification_py
 +
        Starting >>> tf_detection_py                       
 +
        Finished <<< tf_classification_py [1.52s]                         
 +
        Finished <<< tf_detection_py [1.52s]
 +
        Starting >>> ros2_tensorflow
 +
        Finished <<< ros2_tensorflow [1.49s]                 
 +
 +
        Summary: 5 packages finished [15.1s]
 +
 
 +
= Tracking Down the ssd_mobilenet Model - an Addendum =
 +
 
 +
== References ==
 +
 
 +
* [https://github.com/tensorflow/models Tensorflow Model Garden]
 +
* [https://github.com/IntelAI/models/tree/master/benchmarks/object_detection/tensorflow/ssd-mobilenet ssd_mobilenet]
 +
* [https://arxiv.org/pdf/1704.04861.pdf MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications] PDF Paper by Google
 +
 
 +
* [https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/inspect_checkpoint.py inspect_checkpoint.py] '''temp'''
 +
 
 +
Found the URL in (the following steps not tested)
 +
  ~/ros2-tensorflow/ros2-tensorflow/tf_detection_py/tf_detection_py/models.py
 +
 +
  cd ~/Downloads
 +
 +
  $ wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
 +
        --2021-09-03 17:11:32--  http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
 +
        Resolving download.tensorflow.org (download.tensorflow.org)... 172.217.14.208, 2607:f8b0:400a:801::2010
 +
        Connecting to download.tensorflow.org (download.tensorflow.org)|172.217.14.208|:80... connected.
 +
        HTTP request sent, awaiting response... 200 OK
 +
        Length: 76534733 (73M) [application/x-tar]
 +
        Saving to: ‘ssd_mobilenet_v1_coco_2017_11_17.tar.gz’
 +
 +
        ssd_mobilenet_v1_co 100%[===================>]  72.99M  9.19MB/s    in 7.9s   
 +
 +
        2021-09-03 17:11:41 (9.25 MB/s) - ‘ssd_mobilenet_v1_coco_2017_11_17.tar.gz’ saved [76534733/76534733]
 +
 +
  $ cd /usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/
 +
 +
  $ sudo tar -xf ~/Downloads/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
 +
 
 +
* add read and execute for "other" to directories and read to "other" files
 +
 
 +
* '''Depreciated''': [[Turtlebot3 Hood Waffle Foxy Tensorflow Tracking Down ssd_mobilenet 20210821]] messy brain dead process
 +
 
 +
= Usage =
 +
 
 +
'''Note''': Set up environment by doing this in each terminal.
 +
 
 +
  $ source ~/tf_ws/install/local_setup.sh
 +
 
 +
'''Note''': Complete execution logs provided below because of all the warnings.
 +
 
 +
== Object Detection Task 1 ==
 +
 
 +
'''Terminal 1'''
 +
 
 +
  $ ros2 run tf_detection_py server
 +
        2021-08-28 13:48:23.345601: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-28 13:48:23.345637: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
 +
        2021-08-28 13:48:26.952001: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-28 13:48:26.952056: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
 +
        2021-08-28 13:48:26.952101: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
 +
        2021-08-28 13:48:26.952523: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
 +
        To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
 +
        2021-08-28 13:48:26.994763: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
 +
        2021-08-28 13:48:27.013943: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 1895505000 Hz
 +
        [INFO] [1630172911.870803088] [detection_server]: Load model completed!
 +
        [INFO] [1630172915.140851798] [detection_server]: Warmup completed! Ready to receive real images!
 +
 
 +
'''Terminal 2'''
 +
 
 +
  $  ros2 run tf_detection_py client_test
 +
        2021-08-28 13:54:10.878133: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-28 13:54:10.878167: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
 +
        [INFO] [1630173252.719989642] [client_test]: received vision info vision_msgs.msg.VisionInfo(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=0, nanosec=0), frame_id=''), method='TensorFlow Mobilenet network for object detection. Trained on COCO dataset. Produces boxes.', database_location='/usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/data/mscoco_label_map.pbtxt', database_version=0)
 +
        [INFO] [1630173252.838113727] [client_test]: Detected object 23 with score 0.977092444896698
 +
        [INFO] [1630173252.838639380] [client_test]: Detected object 1 with score 0.9131530523300171
 +
        [INFO] [1630173252.839119233] [client_test]: Detected object 18 with score 0.6583045721054077
 +
 
 +
* '''Note''': We found objects 23, 1, and 18. I just need to map these indices to object labels.
 +
* '''Note''': Both cannot load libcudart.so library. My NUC does not have a NVIDIA card so the library is not installed.
 +
 
 +
== Object Detection Task 2 ==
 +
 
 +
Plug in webcam and do the follow source in each terminal
 +
 
 +
  $ source ~/tf_ws/install/local_setup.sh
 +
 
 +
'''TERMINAL 1'''
 +
 
 +
  $ ros2 run tf_detection_py server
 +
        2021-08-28 13:48:23.345601: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-28 13:48:23.345637: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
 +
        2021-08-28 13:48:26.952001: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-28 13:48:26.952056: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
 +
        2021-08-28 13:48:26.952101: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
 +
        2021-08-28 13:48:26.952523: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
 +
        To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
 +
        2021-08-28 13:48:26.994763: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
 +
        2021-08-28 13:48:27.013943: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 1895505000 Hz
 +
        [INFO] [1630172911.870803088] [detection_server]: Load model completed!
 +
        [INFO] [1630172915.140851798] [detection_server]: Warmup completed! Ready to receive real images!
 +
 
 +
'''TERMINAL 2'''
 +
 
 +
  $ ros2 run image_tools cam2image --ros-args -p frequency:=2.0
 +
        [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
 +
        [INFO] [1630174558.757006217] [cam2image]: Publishing image #1
 +
        [INFO] [1630174559.255885363] [cam2image]: Publishing image #2
 +
        [INFO] [1630174559.755881035] [cam2image]: Publishing image #3
 +
        [INFO] [1630174560.255851063] [cam2image]: Publishing image #4
 +
        [INFO] [1630174560.755879077] [cam2image]: Publishing image #5
 +
        [INFO] [1630174561.255887359] [cam2image]: Publishing image #6
 +
        ...
 +
        ...
 +
 
 +
'''TERMINAL 3'''
 +
 
 +
'''Note''': detections_image below
 +
 
 +
  $ ros2 topic list
 +
        /clicked_point
 +
        /detections
 +
        /detections_image
 +
        /flip_image
 +
        /goal_pose
 +
        /image
 +
        /initialpose
 +
        /parameter_events
 +
        /rosout
 +
        /tf
 +
        /tf_static
 +
        /vision_info
 +
 
 +
  $ ros2 run image_tools showimage image:=detections_image
 +
        [WARN] [1630175001.504205227] [rcl]: Found remap rule 'image:=detections_image'. This syntax is deprecated. Use '--ros-args --remap image:=detections_image' instead.
 +
        [WARN] [1630175001.504586499] [rcl]: Found remap rule 'image:=detections_image'. This syntax is deprecated. Use '--ros-args --remap image:=detections_image' instead.
 +
        [INFO] [1630175001.879732533] [showimage]: Subscribing to topic 'image'
 +
        [INFO] [1630175001.973765362] [showimage]: Received image #map
 +
        Received image #map
 +
        [INFO] [1630175002.460577395] [showimage]: Received image #map
 +
        Received image #map
 +
 
 +
'''Note''': There is a 59% chance that I am a person. What can I say.
 +
 
 +
[[File: Image_detect_20210828.png | 400px]]
 +
 
 +
=== Dig Deeper ===
 +
 
 +
* [[Investigate Object Detection in Alsora Foxy TensorFlow 20220107]]
 +
** [https://github.com/alsora/ros2-tensorflow ROS 2 Tensorflow]
 +
 
 +
=== rviz option ===
 +
 
 +
'''TERMINAL 3'''
 +
 
 +
  $ rviz2
 +
        [INFO] [1630174421.214625347] [rviz2]: Stereo is NOT SUPPORTED
 +
        [INFO] [1630174421.214776678] [rviz2]: OpenGl version: 3 (GLSL 1.3)
 +
        [INFO] [1630174421.254860420] [rviz2]: Stereo is NOT SUPPORTED
 +
        [INFO] [1630174578.194036847] [rviz2]: Stereo is NOT SUPPORTED
 +
 
 +
* '''Note''': "Image" was added to the "Displays"
 +
* '''Note''': the Image "Topic" was set to "detections_image"
 +
 
 +
[[File: Image_detect_rviz_20210828.png | 700px]]
 +
 
 +
== Image Classification Task ==
 +
 
 +
'''Note''': Failed
 +
'''TBE''': Fix
 +
 
 +
= Notes and Logs =
 +
 
 +
The install Tensorflow Foxy install went badly. So I went for a simpler vanilla Tensorflow install that did not integrate with ROS2.
 +
* [[Turtlebot3 Waffle NUC Alsora Foxy Tensorflow Install 20210708 Notes and Logs]]
 +
* [[Turtlebot3 Waffle NUC Alsora Build Foxy Tensorflow WS 20210810 Notes and Logs]]
 +
* [[Turtlebot3 Waffle NUC Alsora Build Foxy Tensorflow WS Again 21200812 Notes and Logs]]
 +
* [[Turtlebot3 Waffle NUC Alsora Foxy Tensorflow Track Down Execution Errors 20210821 Notes and Logs]]
 +
* [[Turtlebot3 Waffle NUC Vanilla Tensorflow Install and Test 20210730 Notes and Logs]]
 +
* [[Turtlebot3 Waffle NUC ROS Key and vision-msgs Install 20210811 Notes and Logs]]
 +
 
 +
=  Depreciated =
 +
 
 +
== Object Detection Task ==
 +
 
 +
'''Terminal 1'''
 +
 
 +
This was run under Docker. I noted that the ssd_mobilenet_v1_coco_2017_11_17 downloaded
 +
 
 +
  $ source tf_ws/install/local_setup.sh
 +
 +
  root@hood:~# ros2 run tf_detection_py server
 +
        2021-08-21 19:12:31.941588: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-21 19:12:31.941645: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
 +
        >> Downloading ssd_mobilenet_v1_coco_2017_11_17 100.0%Succesfully downloaded ssd_mobilenet_v1_coco_2017_11_17 76534733 bytes.
 +
        Trying to extract archive /usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
 +
        2021-08-21 19:12:47.081207: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1';  dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-21 19:12:47.081271: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
 +
        2021-08-21 19:12:47.081327: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
 +
        2021-08-21 19:12:47.081894: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural  Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
 +
        To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
 +
        2021-08-21 19:12:47.125438: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
 +
        [INFO] [1629573171.671236404] [detection_server]: Load model completed!
 +
        [INFO] [1629573174.817241482] [detection_server]: Warmup completed! Ready to receive real images!
 +
 +
 
 +
Need to dig into these errors
 +
 
 +
'''Terminal 2'''
 +
 
 +
  $ source tf_ws/install/local_setup.sh
 +
 +
  $ ros2 run tf_detection_py client_test
 +
        2021-08-13 00:12:29.450154: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home /eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-13 00:12:29.450190: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
 +
        Downloading data from http://farm9.staticflickr.com/8036/7946162344_ee2e1a814e_z.jpg
 +
          8192/Unknown - 0s 0us/step[INFO] [1628827952.451718517] [client_test]: service not available, waiting again...
 +
        [INFO] [1628827953.455543297] [client_test]: service not available, waiting again...
 +
        [INFO] [1628827954.459358797] [client_test]: service not available, waiting again...
 +
        [INFO] [1628827955.463214479] [client_test]: service not available, waiting again...
 +
        [INFO] [1628827956.466590696] [client_test]: service not available, waiting agai
 +
 
 +
Kind of wanted to start.
 +
 
 +
'''Note''': Both cannot load libcudart.so library. My NUC does not have a NVIDIA card so the library is not installed.
 +
 
 +
== Clue from Running Under Docker ==
 +
 
 +
'''TERMINAL 1'''
 +
 
 +
First start up Docker. Notice that ssd_mobilenet_v1_coco_2017_11_17 gets loaded which it doesn't when we are not in that virtual environment.
 +
 
 +
  $ bash ros2-tensorflow/docker/run.sh
 +
 +
  root@hood:~# ros2 run tf_detection_py server
 +
        2021-08-21 19:12:31.941588: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-21 19:12:31.941645: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
 +
        >> Downloading ssd_mobilenet_v1_coco_2017_11_17 100.0%Succesfully downloaded ssd_mobilenet_v1_coco_2017_11_17 76534733 bytes.
 +
        Trying to extract archive /usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
 +
        2021-08-21 19:12:47.081207: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1';  dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
 +
        2021-08-21 19:12:47.081271: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
 +
        2021-08-21 19:12:47.081327: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
 +
        2021-08-21 19:12:47.081894: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural  Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
 +
        To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
 +
        2021-08-21 19:12:47.125438: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
 +
        [INFO] [1629573171.671236404] [detection_server]: Load model completed!
 +
        [INFO] [1629573174.817241482] [detection_server]: Warmup completed! Ready to receive real images!

Latest revision as of 11:25, 29 January 2022

Experience

I started here with the Tensorflow requirements for Foxy and got into trouble. So I changed my strategy to a vanilla Tensorflow and other required components install. There may be overlap and the order my need fine tuning

References

Alsora

Alsora was chosen because it appeared prominently in searches. I had to start someplace and then adjust as I became familiar with tensorFlow Object Detection.

It is also the only TensorFlow link provided by Awesome Robots reference page. Again, probably a good place to start.

Kandi - Explore reusable Libraries curated by Popular Use Cases

Other Projects

In

Others

Move

TBD Move the following to more appropriate page.

Requirements

Required Dependencies

  • Foxy: Installed

Rosdep Dependencies

OpenCV Python

 pip -V
     pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

v 19.3 is minimum. 

 $ pip install opencv-python

It was already installed.

Tensorflow

Vision Msgs

ROS Foxy's vision-msgs package is required. However, the ROS key had expired causing some confusion.

References

Install ROS Key

If your ROS key has expired 

 $ curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -

 $ sudo apt update

Install vission-msgs

$ sudo apt install ros-foxy-vision-msgs

Optional Dependencies

Tensorflow Object Detection Models

I tried going to the following link given by Alsora. The link is broken.

I may have installed the models when I was testing a Udacity Python3 script. See the "Executing Lesson 3 from the Udacity Course" section on page Hood Ubuntu Vanilla Tensorflow Install and Test 20210708 

 $ pip install -U tensorflow_datasets

Tensorflow Slim

 $ pip install --upgrade tf_slim

Docker

To use Docker

Hood ROS 2 Foxy Docker 20210814

Build

Create Foxy Tensoryflow work space

 $ git clone https://github.com/alsora/ros2-tensorflow.git $HOME/ros2-tensorflow
       Cloning into '/home/eepp/ros2-tensorflow'...
       ...

 $ mkdir -p $HOME/tf_ws/src

 $ cd $HOME/tf_ws

 $ ln -s $HOME/ros2-tensorflow/ros2-tensorflow src

Install dependencies

 $ rosdep install --from-paths src --ignore-src --rosdistro foxy -y

 $ rosdep install --from-paths src --ignore-src --rosdistro foxy -y -r

Install Object Detection Models

 $ sudo apt-get install -y protobuf-compiler python-lxml python-tk
       ...
       ...

 $ pip install --user Cython contextlib2
       ...

The next 3 installs were already satisfied. No additional action. 

 $ pip install --user Cython jupyter

 $ pip install --user Cython matplotlib

 $ pip install --user Cython Pillow

The following failed. 

 cd usr/local/lib/python3.8/dist-packages/tensorflow/models/research

Corrected 

 cd /usr/local/lib/python3.8/dist-packages/tensorflow/models/research

The following failed. 

  $ protoc object_detection/protos/*.proto --python_out=.

Corrected 

 $ sudo protoc object_detection/protos/*.proto --python_out=.

Missed this the first time I installed 

 $ echo 'export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.8/dist-packages/tensorflow/models/research' >> $HOME/.bashrc

 $ tail ~/.bashrc
       export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.8/dist-packages/tensorflow/models/research

Install tf_slim

tf_slim installed earlier. See Optional Dependencies above.


Build and install the ros2-tensorflow package

Double check the symbolic link 

 $ cd

 $ ll tf_ws/src/
       total 8
       drwxrwxr-x 2 eepp eepp 4096 Aug 12 23:43 ./
       drwxrwxr-x 3 eepp eepp 4096 Aug 12 23:43 ../
       lrwxrwxrwx 1 eepp eepp   42 Aug 12 23:43 ros2-tensorflow -> /home/eepp/ros2-tensorflow/ros2-tensorflow/

 $ cd tf_ws

The following fails 

 $ colcon build

Corrected by using symlink-install switch 

 $ colcon build --symlink-install
       Starting >>> ros2_tf_core
       Starting >>> tf_interfaces
       Finished <<< ros2_tf_core [1.33s]                                             
       Finished <<< tf_interfaces [11.9s]                       
       Starting >>> tf_classification_py
       Starting >>> tf_detection_py                        
       Finished <<< tf_classification_py [1.52s]                           
       Finished <<< tf_detection_py [1.52s]
       Starting >>> ros2_tensorflow
       Finished <<< ros2_tensorflow [1.49s]                   

       Summary: 5 packages finished [15.1s]

Tracking Down the ssd_mobilenet Model - an Addendum

References

Found the URL in (the following steps not tested) 

 ~/ros2-tensorflow/ros2-tensorflow/tf_detection_py/tf_detection_py/models.py

 cd ~/Downloads

 $ wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
       --2021-09-03 17:11:32--  http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
       Resolving download.tensorflow.org (download.tensorflow.org)... 172.217.14.208, 2607:f8b0:400a:801::2010
       Connecting to download.tensorflow.org (download.tensorflow.org)|172.217.14.208|:80... connected.
       HTTP request sent, awaiting response... 200 OK
       Length: 76534733 (73M) [application/x-tar]
       Saving to: ‘ssd_mobilenet_v1_coco_2017_11_17.tar.gz’

       ssd_mobilenet_v1_co 100%[===================>]  72.99M  9.19MB/s    in 7.9s    

       2021-09-03 17:11:41 (9.25 MB/s) - ‘ssd_mobilenet_v1_coco_2017_11_17.tar.gz’ saved [76534733/76534733]

 $ cd /usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/

 $ sudo tar -xf ~/Downloads/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
  • add read and execute for "other" to directories and read to "other" files

Usage

Note: Set up environment by doing this in each terminal. 

 $ source ~/tf_ws/install/local_setup.sh

Note: Complete execution logs provided below because of all the warnings.

Object Detection Task 1

Terminal 1 

 $ ros2 run tf_detection_py server
       2021-08-28 13:48:23.345601: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-28 13:48:23.345637: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
       2021-08-28 13:48:26.952001: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-28 13:48:26.952056: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
       2021-08-28 13:48:26.952101: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
       2021-08-28 13:48:26.952523: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
       To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
       2021-08-28 13:48:26.994763: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
       2021-08-28 13:48:27.013943: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 1895505000 Hz
       [INFO] [1630172911.870803088] [detection_server]: Load model completed!
       [INFO] [1630172915.140851798] [detection_server]: Warmup completed! Ready to receive real images!

Terminal 2 

 $  ros2 run tf_detection_py client_test
       2021-08-28 13:54:10.878133: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-28 13:54:10.878167: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
       [INFO] [1630173252.719989642] [client_test]: received vision info vision_msgs.msg.VisionInfo(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=0, nanosec=0), frame_id=), method='TensorFlow Mobilenet network for object detection. Trained on COCO dataset. Produces boxes.', database_location='/usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/data/mscoco_label_map.pbtxt', database_version=0)
       [INFO] [1630173252.838113727] [client_test]: Detected object 23 with score 0.977092444896698
       [INFO] [1630173252.838639380] [client_test]: Detected object 1 with score 0.9131530523300171
       [INFO] [1630173252.839119233] [client_test]: Detected object 18 with score 0.6583045721054077
  • Note: We found objects 23, 1, and 18. I just need to map these indices to object labels.
  • Note: Both cannot load libcudart.so library. My NUC does not have a NVIDIA card so the library is not installed.

Object Detection Task 2

Plug in webcam and do the follow source in each terminal 

 $ source ~/tf_ws/install/local_setup.sh

TERMINAL 1 

 $ ros2 run tf_detection_py server
       2021-08-28 13:48:23.345601: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-28 13:48:23.345637: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
       2021-08-28 13:48:26.952001: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-28 13:48:26.952056: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
       2021-08-28 13:48:26.952101: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
       2021-08-28 13:48:26.952523: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
       To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
       2021-08-28 13:48:26.994763: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
       2021-08-28 13:48:27.013943: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 1895505000 Hz
       [INFO] [1630172911.870803088] [detection_server]: Load model completed!
       [INFO] [1630172915.140851798] [detection_server]: Warmup completed! Ready to receive real images!

TERMINAL 2 

 $ ros2 run image_tools cam2image --ros-args -p frequency:=2.0
       [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
       [INFO] [1630174558.757006217] [cam2image]: Publishing image #1
       [INFO] [1630174559.255885363] [cam2image]: Publishing image #2
       [INFO] [1630174559.755881035] [cam2image]: Publishing image #3
       [INFO] [1630174560.255851063] [cam2image]: Publishing image #4
       [INFO] [1630174560.755879077] [cam2image]: Publishing image #5
       [INFO] [1630174561.255887359] [cam2image]: Publishing image #6
       ...
       ...

TERMINAL 3

Note: detections_image below 

 $ ros2 topic list
       /clicked_point
       /detections
       /detections_image
       /flip_image
       /goal_pose
       /image
       /initialpose
       /parameter_events
       /rosout
       /tf
       /tf_static
       /vision_info
 
 $ ros2 run image_tools showimage image:=detections_image
       [WARN] [1630175001.504205227] [rcl]: Found remap rule 'image:=detections_image'. This syntax is deprecated. Use '--ros-args --remap image:=detections_image' instead.
       [WARN] [1630175001.504586499] [rcl]: Found remap rule 'image:=detections_image'. This syntax is deprecated. Use '--ros-args --remap image:=detections_image' instead.
       [INFO] [1630175001.879732533] [showimage]: Subscribing to topic 'image'
       [INFO] [1630175001.973765362] [showimage]: Received image #map
       Received image #map
       [INFO] [1630175002.460577395] [showimage]: Received image #map
       Received image #map

Note: There is a 59% chance that I am a person. What can I say.

Image detect 20210828.png

Dig Deeper

rviz option

TERMINAL 3 

 $ rviz2
       [INFO] [1630174421.214625347] [rviz2]: Stereo is NOT SUPPORTED
       [INFO] [1630174421.214776678] [rviz2]: OpenGl version: 3 (GLSL 1.3)
       [INFO] [1630174421.254860420] [rviz2]: Stereo is NOT SUPPORTED
       [INFO] [1630174578.194036847] [rviz2]: Stereo is NOT SUPPORTED
  • Note: "Image" was added to the "Displays"
  • Note: the Image "Topic" was set to "detections_image"

Image detect rviz 20210828.png

Image Classification Task

Note: Failed TBE: Fix

Notes and Logs

The install Tensorflow Foxy install went badly. So I went for a simpler vanilla Tensorflow install that did not integrate with ROS2.

Depreciated

Object Detection Task

Terminal 1

This was run under Docker. I noted that the ssd_mobilenet_v1_coco_2017_11_17 downloaded 

 $ source tf_ws/install/local_setup.sh 

 root@hood:~# ros2 run tf_detection_py server
       2021-08-21 19:12:31.941588: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-21 19:12:31.941645: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
       >> Downloading ssd_mobilenet_v1_coco_2017_11_17 100.0%Succesfully downloaded ssd_mobilenet_v1_coco_2017_11_17 76534733 bytes.
       Trying to extract archive /usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
       2021-08-21 19:12:47.081207: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1';  dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-21 19:12:47.081271: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
       2021-08-21 19:12:47.081327: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
       2021-08-21 19:12:47.081894: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural  Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
       To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
       2021-08-21 19:12:47.125438: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
       [INFO] [1629573171.671236404] [detection_server]: Load model completed!
       [INFO] [1629573174.817241482] [detection_server]: Warmup completed! Ready to receive real images!

Need to dig into these errors

Terminal 2 

 $ source tf_ws/install/local_setup.sh

 $ ros2 run tf_detection_py client_test
       2021-08-13 00:12:29.450154: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home /eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-13 00:12:29.450190: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
       Downloading data from http://farm9.staticflickr.com/8036/7946162344_ee2e1a814e_z.jpg
          8192/Unknown - 0s 0us/step[INFO] [1628827952.451718517] [client_test]: service not available, waiting again...
       [INFO] [1628827953.455543297] [client_test]: service not available, waiting again...
       [INFO] [1628827954.459358797] [client_test]: service not available, waiting again...
       [INFO] [1628827955.463214479] [client_test]: service not available, waiting again...
       [INFO] [1628827956.466590696] [client_test]: service not available, waiting agai

Kind of wanted to start.

Note: Both cannot load libcudart.so library. My NUC does not have a NVIDIA card so the library is not installed.

Clue from Running Under Docker

TERMINAL 1

First start up Docker. Notice that ssd_mobilenet_v1_coco_2017_11_17 gets loaded which it doesn't when we are not in that virtual environment. 

 $ bash ros2-tensorflow/docker/run.sh

 root@hood:~# ros2 run tf_detection_py server
       2021-08-21 19:12:31.941588: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-21 19:12:31.941645: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
       >> Downloading ssd_mobilenet_v1_coco_2017_11_17 100.0%Succesfully downloaded ssd_mobilenet_v1_coco_2017_11_17 76534733 bytes.
       Trying to extract archive /usr/local/lib/python3.8/dist-packages/tensorflow/models/research/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz
       2021-08-21 19:12:47.081207: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1';  dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/tf_ws/install/tf_interfaces/lib:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-21 19:12:47.081271: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
       2021-08-21 19:12:47.081327: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
       2021-08-21 19:12:47.081894: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural  Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
       To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
       2021-08-21 19:12:47.125438: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
       [INFO] [1629573171.671236404] [detection_server]: Load model completed!
       [INFO] [1629573174.817241482] [detection_server]: Warmup completed! Ready to receive real images!
Retrieved from "http://zdome.net/wiki/index.php?title=Install_Turtlebot3_Hood_Waffle_Foxy_alsora_ros2-tensorflow_20210615&oldid=7857"