Operations - Image Streaming (Make it See)

This section describes how to see what your Duckiebot sees, in other words, gain access to the camera stream. Three methods are shown, one to access the images as the Duckiebot sees them, and two to access images as humans see them.

What you will need

• A correctly assembled Duckiebot: Testing - Duckiebot Hardware Components

• (for most methods) A functional DTS installation: Duckietown Shell (dts) installation

What you will get

• Seeing images as the Duckiebot sees them

• Knowledge on the Duckietown image viewer

The Duckietown Image Viewer

One of the easiest ways to see what your Duckiebot sees is by using the Image Viewer.

Fig. 26 The Duckietown “Image Viewer” helps visualize the Duckiebot’s camera stream.

To open the Image Viewer, run:

dts duckiebot image_viewer DUCKIEBOT_NAME

Note the keys in the table below.

Key	Function
X	Increase the Frame Rate
Z	Decrease the Frame Rate
Space	Capture an image
R	Refresh the window
T	Open the Debug Console

Through the browser: the Duckietown Dashboard Mission Control page

If you followed the instructions in Setup - Duckiebot Dashboard, you should have access to the Duckiebot Dashboard. You can check the image stream by accessing the Mission control page on your Duckiebot Dashboard.

The bottom of the page shows the camera block. You should be able to see the camera feed in the camera block, as shown in the image below.

Fig. 27 The Dashboard Robot > Mission Control page shows basic sensor and actuator data streams.

By default, the camera stream is throttled down to 8 frames per second. This is to minimize the resources used by your browser while streaming images from the robot. Feel free to increase the data stream frequency in the Properties tab of the camera block.

Tip

If you see a black image in the camera block, make sure that you removed the protective cap that covers the camera lens of your Duckiebot.

Attention

There is a known bug for which the image stream will flicker when using Mozilla Firefox. Change browser, e.g., to Chrome bypass this.

Viewing the Image Stream on Your Laptop through rqt image view

The camera image is streaming from your Duckiebot by default on startup.

To see it, open a terminal on your laptop and run:

dts start_gui_tools ROBOT_NAME

This will start a container with access to the ROS messages of the Duckiebot, including the image stream from the camera.

Your terminal has now turned into a command line interface running inside of that container within the Duckiebot. You can exit the container back to your normal terminal interface at any time by running the exit command.

To view the camera stream, run:

rqt_image_view

The command will open a window where you can view the image.

You will have to select the camera_node/image/compressed topic from the drop-down menu:

Fig. 28 The rqt image view window with dropdown menu - select the camera_node/image/compressed topic.

How to save a picture from rqt_image_view

On the top right of the rqt_image_view window, there is a button to save the current frame to an image. But don’t save it yet. A little extra setup is needed to be able to view that file later.

Create a folder on you laptop for where you would like to have the image saved to, say ~/duckiebot_images/. Then, launch the start_gui_tools with the following command:

dts start_gui_tools --mount ~/duckiebot_images/:/duckiebot_images [DUCKIEBOT_NAME]

Then, run rqt_image_view again, and use the top-right “Save as image” button to save to the /duckiebot_images folder. To find that folder, you might need to navigate to Computer and select / directory in the pop-up dialogue.

And the saved image from your robot’s view should appear in the folder you created!

How to see images as the robot sees them

While in the gui tools terminal, run:

rostopic list

If using the baseline ROS image from Duckietown, you will see list of ROS topics. Without getting in the details of ROS at this stage (check the learning experiences for additional information), you will find a topic with camera_node/image/compressed in its name. Run:

rostopic echo TOPIC_NAME

to see the data (messages) being transmitted over that channel (topic). Press CTRL-C to stop the stream and go back to the terminal.

Troubleshooting

Troubleshooting

SYMPTOM

I can see messages being sent from my Duckiebot when looking at the DUCKIEBOT_NAME/sensor/camera/front_center/jpeg DTPS topic, after following Duckietown Postal Service (DTPS), but I do not see an image.

RESOLUTION

Make sure that the duckiebot-interface container is running by checking the Portainer page of the Dashboard (opened by running dts duckiebot dashboard DUCKIEBOT_NAME --page portainer) or by running:

docker -H DUCKIEBOT_NAME.local ps

The exact name of the container will depend on your Duckiebot’s version. If you do not see the duckiebot-interface container, update your Duckiebot by running:

dts duckiebot update DUCKIEBOT_NAME

Troubleshooting

SYMPTOM

I cannot see an image after refreshing the window and I cannot see messages being sent from my Duckiebot when looking at the DUCKIEBOT_NAME/sensor/camera/front_center/jpeg DTPS topic, after following Duckietown Postal Service (DTPS).

RESOLUTION

Contact support.

Troubleshooting

SYMPTOM

I see a black image.

RESOLUTION

Make sure that the protective cap for your Duckiebot’s camera lens has been removed.

Troubleshooting

SYMPTOM

The images are out of focus.

RESOLUTION

The focus for your Duckiebot’s camera can be manually adjusted by rotating the mechanical focus ring on the lens. When dealing with hardware, exercise care and minimize the use of force. Occasionally, cameras come with the lens glued in place. If the lens does not rotate, you may need to break the glue.