Difference between revisions of "Investigate Object Detection in Alsora Foxy TensorFlow 20220107"

From wikidb
Jump to: navigation, search
(→‎Nodes and Topics)
(→‎explore tf_detection_py)
 
(10 intermediate revisions by the same user not shown)
Line 66: Line 66:
 
         [INFO] [1630175002.460577395] [showimage]: Received image #map
 
         [INFO] [1630175002.460577395] [showimage]: Received image #map
 
         Received image #map
 
         Received image #map
−
 
+
 
'''showimage''' window
 
'''showimage''' window
  
Line 94: Line 94:
  
 
[[File: rosgraphObjectDetection20220117.png | 800px]]
 
[[File: rosgraphObjectDetection20220117.png | 800px]]
 +
 +
This shows the image directly from the webcam
 +
 +
  $ ros2 run image_tools showimage image:=image
 +
 +
 +
== Image Tools ==
 +
 +
* [https://index.ros.org/p/image_tools/#foxy ROS Index image_tools foxy]
 +
* [https://github.com/ros2/demos/tree/master/image_tools/src ros2 demos image_tools src]
 +
 +
 +
  $ pwd
 +
        /opt/ros/foxy/lib/image_tools
 +
 +
  $ ll
 +
        total 148
 +
        drwxr-xr-x  2 root root  4096 Mar 11  2021 ./
 +
        drwxr-xr-x 92 root root 45056 Aug 12 19:53 ../
 +
        -rwxr-xr-x  1 root root 47480 Feb 25  2021 cam2image*
 +
        -rwxr-xr-x  1 root root 47480 Feb 25  2021 showimage*
 +
 +
  $ ps -ux
 +
        eepp        4587  2.0  0.5 1010504 92124 pts/1  SLl+ 13:27  0:07 /opt/ros/foxy/lib/image_tools/cam2image --ros-args -p frequency:=2.0
 +
        eepp        6455  0.2  0.6 1367832 106256 pts/3  SLl+ 15:34  0:17 /opt/ros/foxy/lib/image_tools/showimage image:=detections_image
 +
 +
== explore tf_detection_py ==
 +
 +
  $ps -au
 +
        eepp        5472 72.9  6.1 3551620 1005320 pts/0 Sl+  13:29  3:02 /usr/bin/python3 /home/eepp/tf_ws/install/tf_detection_py/lib/tf_detection_py/server
 +
 +
Executables
 +
 +
  $ pwd
 +
        /home/eepp/tf_ws/install/tf_detection_py/lib/tf_detection_py
 +
 +
  $ ls
 +
        catsNdogs  client_test  face_detection  server
 +
 +
Source
 +
 +
  $ pwd
 +
        home/eepp/tf_ws/src/ros2-tensorflow
 +
  $ ls
 +
        ros2_tensorflow  tf_classification_py  tf_interfaces
 +
        ros2_tf_core    tf_detection_py
 +
 +
Libraries
 +
 +
  $ pwd
 +
        /usr/local/lib/python3.8/dist-packages/tensorflow/models/research
 +
 +
  $ ls
 +
        adversarial_text    deeplab                marco            seq_flow_lite
 +
        attention_ocr      deep_speech            nst_blogpost      slim
 +
        audioset            delf                  object_detection  vid2depth
 +
        autoaugment        efficient-hrl          pcl_rl
 +
        cognitive_planning  lfads                  README.md
 +
        cvt_text            lstm_object_detection  rebar
 +
 +
  $ typora README.md &
 +
        [3] 12643
 +
 +
  $ cd object_detection/
 +
 +
  $ ls
 +
        anchor_generators                        __init__.py
 +
        box_coders                              inputs.py
 +
        builders                                inputs_test.py
 +
        colab_tutorials                          legacy
 +
        configs                                  matchers
 +
        CONTRIBUTING.md                          meta_architectures
 +
        core                                    metrics
 +
        data                                    model_hparams.py
 +
        data_decoders                            model_lib.py
 +
        dataset_tools                            model_lib_tf1_test.py
 +
        dockerfiles                              model_lib_tf2_test.py
 +
        eval_util.py                            model_lib_v2.py
 +
        eval_util_test.py                        model_main.py
 +
        exporter_lib_tf2_test.py                model_main_tf2.py
 +
        exporter_lib_v2.py                      models
 +
        exporter_main_v2.py                      model_tpu_main.py
 +
        exporter.py                              packages
 +
        exporter_tf1_test.py                    predictors
 +
        export_inference_graph.py                protos
 +
        export_tflite_graph_lib_tf2.py          README.md
 +
        export_tflite_graph_lib_tf2_test.py      samples
 +
        export_tflite_graph_tf2.py              ssd_mobilenet_v1_coco_2017_11_17
 +
        export_tflite_ssd_graph_lib.py          test_data
 +
        export_tflite_ssd_graph_lib_tf1_test.py  test_images
 +
        export_tflite_ssd_graph.py              tpu_exporters
 +
        g3doc                                    utils
 +
        inference
 +
 +
  $ typora README.md &
 +
        [3] 12784

Latest revision as of 15:48, 17 January 2022

References

Object Detection Usage Task 2

Copied from

Plug in webcam and do the follow source in each terminal 

 $ source ~/tf_ws/install/local_setup.sh

TERMINAL 1 

 $ ros2 run tf_detection_py server
       2021-08-28 13:48:23.345601: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-28 13:48:23.345637: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
       2021-08-28 13:48:26.952001: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/eepp/tf_ws/install/tf_interfaces/lib:/home/eepp/image_ws/install/stereo_image_proc/lib:/home/eepp/image_ws/install/realsense_ros/lib:/home/eepp/image_ws/install/realsense_msgs/lib:/home/eepp/image_ws/install/realsense_camera_msgs/lib:/home/eepp/image_ws/install/image_view/lib:/home/eepp/image_ws/install/image_rotate/lib:/home/eepp/image_ws/install/image_proc/lib:/home/eepp/image_ws/install/depth_image_proc/lib:/home/eepp/turtlebot3_ws/install/turtlebot3_msgs/lib:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/opt/ros/foxy/opt/yaml_cpp_vendor/lib:/opt/ros/foxy/opt/rviz_ogre_vendor/lib:/opt/ros/foxy/lib/x86_64-linux-gnu:/opt/ros/foxy/lib
       2021-08-28 13:48:26.952056: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
       2021-08-28 13:48:26.952101: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (hood): /proc/driver/nvidia/version does not exist
       2021-08-28 13:48:26.952523: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
       To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
       2021-08-28 13:48:26.994763: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
       2021-08-28 13:48:27.013943: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 1895505000 Hz
       [INFO] [1630172911.870803088] [detection_server]: Load model completed!
       [INFO] [1630172915.140851798] [detection_server]: Warmup completed! Ready to receive real images!

TERMINAL 2 

 $ ros2 run image_tools cam2image --ros-args -p frequency:=2.0
       [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
       [INFO] [1630174558.757006217] [cam2image]: Publishing image #1
       [INFO] [1630174559.255885363] [cam2image]: Publishing image #2
       [INFO] [1630174559.755881035] [cam2image]: Publishing image #3
       [INFO] [1630174560.255851063] [cam2image]: Publishing image #4
       [INFO] [1630174560.755879077] [cam2image]: Publishing image #5
       [INFO] [1630174561.255887359] [cam2image]: Publishing image #6
       ...
       ...

TERMINAL 3

Note: detections_image below 

 $ ros2 topic list
       /clicked_point
       /detections
       /detections_image
       /flip_image
       /goal_pose
       /image
       /initialpose
       /parameter_events
       /rosout
       /tf
       /tf_static
       /vision_info
 
 $ ros2 run image_tools showimage image:=detections_image
       [WARN] [1630175001.504205227] [rcl]: Found remap rule 'image:=detections_image'. This syntax is deprecated. Use '--ros-args --remap image:=detections_image' instead.
       [WARN] [1630175001.504586499] [rcl]: Found remap rule 'image:=detections_image'. This syntax is deprecated. Use '--ros-args --remap image:=detections_image' instead.
       [INFO] [1630175001.879732533] [showimage]: Subscribing to topic 'image'
       [INFO] [1630175001.973765362] [showimage]: Received image #map
       Received image #map
       [INFO] [1630175002.460577395] [showimage]: Received image #map
       Received image #map

showimage window

Note: There is a 59% chance that I am a person. What can I say.

Image detect 20210828.png

Nodes and Topics

Terminal 4 

 $ ros2 node list
       /cam2image
       /detection_server
       /showimage

 $ ros2 topic list
       /detections
       /detections_image
       /flip_image
       /image
       /parameter_events
       /rosout
       /vision_info

 $ rqt_grap

RosgraphObjectDetection20220117.png

This shows the image directly from the webcam 

 $ ros2 run image_tools showimage image:=image


Image Tools


 $ pwd
       /opt/ros/foxy/lib/image_tools

 $ ll
       total 148
       drwxr-xr-x  2 root root  4096 Mar 11  2021 ./
       drwxr-xr-x 92 root root 45056 Aug 12 19:53 ../
       -rwxr-xr-x  1 root root 47480 Feb 25  2021 cam2image*
       -rwxr-xr-x  1 root root 47480 Feb 25  2021 showimage*

 $ ps -ux
       eepp        4587  2.0  0.5 1010504 92124 pts/1   SLl+ 13:27   0:07 /opt/ros/foxy/lib/image_tools/cam2image --ros-args -p frequency:=2.0
       eepp        6455  0.2  0.6 1367832 106256 pts/3  SLl+ 15:34   0:17 /opt/ros/foxy/lib/image_tools/showimage image:=detections_image

explore tf_detection_py

 $ps -au
       eepp        5472 72.9  6.1 3551620 1005320 pts/0 Sl+  13:29   3:02 /usr/bin/python3 /home/eepp/tf_ws/install/tf_detection_py/lib/tf_detection_py/server

Executables 

 $ pwd
       /home/eepp/tf_ws/install/tf_detection_py/lib/tf_detection_py

 $ ls
       catsNdogs  client_test  face_detection  server

Source 

 $ pwd
       home/eepp/tf_ws/src/ros2-tensorflow
 $ ls
       ros2_tensorflow  tf_classification_py  tf_interfaces
       ros2_tf_core     tf_detection_py

Libraries 

 $ pwd
       /usr/local/lib/python3.8/dist-packages/tensorflow/models/research

 $ ls
       adversarial_text    deeplab                marco             seq_flow_lite
       attention_ocr       deep_speech            nst_blogpost      slim
       audioset            delf                   object_detection  vid2depth
       autoaugment         efficient-hrl          pcl_rl
       cognitive_planning  lfads                  README.md
       cvt_text            lstm_object_detection  rebar

 $ typora README.md &
       [3] 12643

 $ cd object_detection/

 $ ls
       anchor_generators                        __init__.py
       box_coders                               inputs.py
       builders                                 inputs_test.py
       colab_tutorials                          legacy
       configs                                  matchers
       CONTRIBUTING.md                          meta_architectures
       core                                     metrics
       data                                     model_hparams.py
       data_decoders                            model_lib.py
       dataset_tools                            model_lib_tf1_test.py
       dockerfiles                              model_lib_tf2_test.py
       eval_util.py                             model_lib_v2.py
       eval_util_test.py                        model_main.py
       exporter_lib_tf2_test.py                 model_main_tf2.py
       exporter_lib_v2.py                       models
       exporter_main_v2.py                      model_tpu_main.py
       exporter.py                              packages
       exporter_tf1_test.py                     predictors
       export_inference_graph.py                protos
       export_tflite_graph_lib_tf2.py           README.md
       export_tflite_graph_lib_tf2_test.py      samples
       export_tflite_graph_tf2.py               ssd_mobilenet_v1_coco_2017_11_17
       export_tflite_ssd_graph_lib.py           test_data
       export_tflite_ssd_graph_lib_tf1_test.py  test_images
       export_tflite_ssd_graph.py               tpu_exporters
       g3doc                                    utils
       inference

 $ typora README.md &
       [3] 12784
Retrieved from "http://zdome.net/wiki/index.php?title=Investigate_Object_Detection_in_Alsora_Foxy_TensorFlow_20220107&oldid=7842"