Fly PX4 Multicopter and Perform Perception-Guided Indoor Flight

R2026b

This example shows how to use a Simulink^® model that starts the flight of a PX4^® multicopter and detects a blue-colored object on the ground during indoor flight. A Raspberry Pi Zero 2W companion computer with a Pi Camera processes image data, and a Pixhawk® 6C flight controller controls the multicopter.

In this example, you will learn how to use the model to:

Obtain an image from the downward-facing camera while the multicopter is airborne
Perform analysis of the image to detect any blue colored objects
Hover the multicopter over the detected blue object

Caution

This example is intended for indoor flight in a controlled environment and is provided for educational purposes only.

Usage restrictions:

Use this example only for educational and evaluation purposes.
Operate the multicopter in a safe, controlled indoor environment.
Do not use this workflow for outdoor, BVLOS (Beyond Visual Line of Sight), or autonomous surveillance operations.
Comply with all applicable aviation, export control, and local regulatory requirements.
Do not provide this example to restricted parties or use it in restricted regions where prohibited by applicable regulations.
Familiarity with PX4 autopilot systems and multicopter operation is recommended before attempting this workflow.
The user is responsible for operating the system, complying with applicable regulations, and assessing operational risks.

Prerequisites

Before you run this example, complete these topics:

If you are new to Simulink, watch the Simulink Quick Start video.
If you have not done so already, complete the steps starting from Hardware Setup for QAV250 Multicopter.

Required Products

To open, simulate, and deploy this example, install these products:

MATLAB^®
Simulink
Embedded Coder^®
Simulink Coder™
UAV Toolbox
Computer Vision Toolbox
Image Processing Toolbox
Simulink 3D Animation™ - For more information, see Satellite Scenario Viewer (Simulink)
UAV Toolbox Support Package for PX4 Autopilots
Simulink Support Package for Raspberry Pi Hardware

Required Hardware

To run this example on hardware, use:

QAV250 multicopter frame with Pixhawk 6C flight controller
Raspberry Pi Zero 2W companion computer
Raspberry Pi Camera Module
PMW3901 optical flow sensor
Telemetry Radios/Modems
ST VL53L1X Time-of-Flight rangefinder
RC transmitter and receiver
Charged flight battery and propeller guards

Open Example Project

At the MATLAB Command Window, open the example project:

PX4MultiCopterPerceptionAndControlStart

Perception and Control project

The example project contains the flight-control and perception workflows used for indoor multicopter operation. The project separates the flight-control model deployed to the Pixhawk 6C and the perception-and-guidance model deployed to the Raspberry Pi.

Note

The command PX4MultiCopterPerceptionAndControlStart starts a new project every time you run the command. Any modifications that you made in a previous project session do not appear in the new project.

Simulate the Model in 3D Simulator

To simulate the model, click the Run button on the Simulink model toolbar.
The lower-left corner of the model window displays status while Simulink prepares to run the model on the host computer.
Observe the multicopter behavior in the 3D Simulator window. The multicopter takes off and starts flying to the right direction until the camera on the multicopter detects a blue color object. On detecting a blue color object, the multicopter stops moving and hovers above the blue object..

Prepare Indoor Environment to Fly multicopter

Place a blue colored object on the ground in the test area.
Position the vehicle so that the object enters the camera field of view during flight..
Connect the telemetry radio paired with the Pixhawk 6C Mini to the host computer. In QGC, verify that the vehicle is connected and ready for flight.
Ensure that the indoor area has sufficient lighting and a textured floor surface for optical flow sensing.
Confirm that all these pre-arm checks pass.
- Confirm that all mechanical fasteners are secure.
- Ensure that the battery is fully charged and properly mounted.
- Remove propellers for initial testing.
- Verify that QGroundControl shows Ready status for all pre-arm checks.
- Ensure that the RC transmitter is powered on and bound.
- Confirm that a kill switch is configured and tested.
- Verify that the Raspberry Pi application is running.
Verify that the RC transmitter is connected and that the Arm, Land, and Kill switches function as expected. These switch assignments are configured in the controller model.
Verify that the vehicle is ready for flight in QGroundControl.

Before Flight

Verify the following before deploying and flying:

Vehicle reports Ready in QGroundControl.
Optical flow data is valid.
Rangefinder data is valid.
Raspberry Pi application is running.
Camera is streaming images.
Arm, Land, and Kill switches are verified.
Propeller guards are installed.

Deploy Controller Model

Note

Before deploying the controller model, verify that the PX4 flight controller and Raspberry Pi are powered on and connected. In QGroundControl, confirm that the Raspberry Pi is communicating with PX4 and reports a valid connection status.

In the Simulink project, from the Project Shortcuts tab, click Step_1_Deploy_Controller.

Project Shortcuts tab showing Step_2_Deploy_Perception

Flight control model

The lower-left corner of the model window displays status while Simulink prepares, downloads, and runs the model on the Pixhawk 6C hardware.

After successfully deploying the model, the PX4 multicopter takes off from the ground and starts moving to the right direction until the multicopter detects the blue colored object. On detecting the object, the multicopter stops moving and starts hovering over the object.

Deploy Perception Model

Note

Before deploying the perception model, verify that the PX4 flight controller and Raspberry Pi are powered on and connected. In QGroundControl, confirm that the Raspberry Pi is communicating with PX4 and reports a valid connection status.

In the Simulink project, from the Project Shortcuts tab, click Step_2_Deploy_Perception.
In the Simulink model, on the Hardware tab, choose one of these options:
- Click Monitor & Tune to deploy the perception model to the Raspberry Pi and tune parameters while the model is running.
- Click Deploy to Hardware to deploy and run the perception model without parameter tuning.
The lower-left corner of the model window displays status while Simulink prepares, downloads, and runs the model on the Raspberry Pi hardware.

After successfully deploying both models, the multicopter takes off and performs perception-guided flight. The Raspberry Pi processes camera images and sends guidance commands to the Pixhawk, which controls the vehicle using these commands together with onboard sensor data.

Modify the Track in 3D Simulator Window

You can modify the track and test environment in the 3D Simulator window to test different flight scenarios and perception algorithms.

In the MATLAB Command Window enter the following command to open the Competition Track Builder.
```
openTrackBuilder
```
In the Competition Track Builder app, modify the track layout as required. For example, create a custom track by drawing a new path and adding visual markers.
Click Update Virtual World to apply the changes to the 3D Simulator environment.
A sample GIF image is shown here.
Observe the multicopter behavior and modify the track as needed to evaluate different navigation and perception scenarios.