Load Ground Truth Signals to Label
The Ground Truth Labeler app provides options for labeling two types of signals.
Image signals are image-based. You can load these signals from sources such as videos or image sequences.
Point cloud signals are lidar-based. You can load these signals from sources such as a sequence of point cloud files.
In this example, you load a video and a point cloud sequence into the app. These signals are taken from a camera sensor and a lidar sensor mounted to a vehicle. The signals represent the same driving scene.
Load the timestamps for the point cloud sequence. The timestamps are a
duration vector that is in the same folder as the sequence. To load the
timestamps, you must temporarily add this folder to the MATLAB® search path.
pcSeqFolder = fullfile(toolboxdir('driving'),'drivingdata','lidarSequence'); addpath(pcSeqFolder) load timestamps.mat rmpath(pcSeqFolder)
The app also provides an option to specify timestamps for video sources. The video used in this example does not have a separate timestamps file, so when you load the video, you can read the timestamps directly from the video source.
Open Ground Truth Labeler App
To open the Ground Truth Labeler app, at the MATLAB command prompt, enter this command.
The app opens to an empty session.
Alternatively, you can open the app from the Apps tab, under Automotive.
Load Signals from Data Sources
The Ground Truth Labeler app enables you to load signals from multiple types of data sources. In the app, a data source is a file or folder containing one or more signals to label.
For the video, the data source is an MP4 file that contains a single video.
For the point cloud sequence, the data source is a folder containing a sequence of point cloud data (PCD) files. Together, these files represent a single point cloud sequence.
Other data sources, such as rosbags, can contain multiple signals that you can load. For more details on the relationship between sources and signals, see Sources vs. Signals in Ground Truth Labeling.
Load the video into the app.
On the app toolstrip, click Import > Add Signals.
The Add/Remove Signal dialog box opens with the Source Type parameter set to
Videoand the Timestamps parameter set to
In the File Name parameter, browse for this video file.
<matlabroot>is the full path to your MATLAB installation folder, as returned by the
Click Add Source. The video loads into the app, and the app reads the timestamps directly from the video. The source table displays the information about the video data source.
Load Point Cloud Sequence
Load the point cloud sequence into the app.
With the Add/Remove Signal dialog box still open and the video loaded, set the Source Type parameter to
Point Cloud Sequence. The dialog box displays new options specific to loading point cloud sequences.
In the Folder Name parameter, browse for the
lidarSequencefolder, which contains the sequence of point cloud data (PCD) files to load.
Set the Timestamps parameter to
From Workspace. In the Import From Workspace dialog box, select the
timestampsvariable that you loaded for the point cloud sequence. Click OK.
Click Add Source. The point cloud sequence loads into the app, and the app reads the timestamps from the
timestampsvariable. The source table displays the information about the data source for the point cloud sequence.
Verify Information About Loaded Signals
The table at the bottom of the Add/Remove Signal dialog box displays information about the loaded signals. Verify that the table displays this information for the loaded signals.
The Signal Name column displays the signal names generated by the app. For the video, the signal name is the file name of the data source with the prefix
video_and with no file extension. For the point cloud sequence, the signal name is the name of the source folder.
The Source column displays the full file paths to the signal data sources.
The Signal Type column displays the type of each signal. The video is of type
Image. The point cloud sequence is of type
The Time Range column displays the duration of the signals based on the loaded timestamp data. Both signals are approximately 10 seconds long.
For the point cloud sequence, if you left Timestamps set to
Use Default, then the Time Range
value for the sequence ranges from 0 to 33 seconds. This range is based on the 34
PCD files in the folder. By default, the app sets the timestamps of a point cloud
sequence to a
duration vector from 0 to the number of valid
point cloud files minus 1. Units are in seconds. If this issue occurs, in the table,
select the check box for the point cloud sequence row. Then, click Delete
Selected, load the signal again, and verify the signal information
After verifying that the signals loaded correctly, click OK. The app loads the signals and opens to the first frame of the last signal added, which for this example is the point cloud sequence.
Configure Signal Display
When you first load the signals, the app displays only one signal at a time. To display the signals side-by-side, first, on the Label tab of the app toolstrip, click Display Grid. Then, move the pointer to select a 1-by-2 grid and click the grid.
The video and point cloud sequence display side-by-side.
To view the video and point cloud sequence together, in the slider below the signals, click the Play button . The video plays more smoothly than the point cloud sequence because the video has more frames over approximately the same amount of time and therefore a higher frame rate.
By default, the app plays all frames from the signal with the highest frame rate. This signal is called the main signal. For all other signals, the app displays the frame that is time-aligned with the currently displaying frame of the main signal. To configure which signal is the main signal, use the options in the Playback Control Settings dialog box. To open this dialog box, below the slider, click the clock settings button . For more details about using these options to control the display of signal frames, see Control Playback of Signal Frames for Labeling.
After loading the signal and viewing the frames, you can now create label definitions and label the data, as described in Label Ground Truth for Multiple Signals.