Apriltags Follower demonstration

This page describes all you have to know to use the ApriltagFollower demonstration, in simulation and in real conditions.

The ApriltagFollower demonstration's goal is to make an uav follow an ​Apriltag autonomously and without using motion capture system.

The ultimate goal of the demonstration is to have two different uavs flying together : one following a line and holding an Apriltag and another following the first uav thanks to the Apriltag. But this goal has not been reached for now, the control law used to follow the Apriltag is not robust enough to compensate the wind from the second uav.

This demonstration looks like that :

How does the demonstration work ?

The demonstration is composed of two elements : the software running on the uav which make the uav follow the Apriltag (program called ApriltagFollower_uav) and the program processing a picture and returning the Apriltag pose in space (program called apriltags_detector). The main reason for the two different components is that the AR.Drone uav is not powerful enough to take care of the picture processing so we use another computation board (a Raspberry Pi).

Simulation

In simulation you have the two softwares on the same machine (your computer). The communication between the two softwares use socket but as your on the same computer, the local network is used.

The picture shows the components and the communication :

In real

In real, things are different as one software runs on the uav and the other one on the Raspberry Pi. The communication between the two components still use sockets but this time it goes trough a network. As the only wire between the uav and the Raspberry Pi is an USB wire, we use Ethernet-over-USB feature to be able to use sockets. The network uses the usb0 interface of the uav and the Raspberry Pi.

The picture shows this difference :

Folder organization

The ApriltagFollower main folder is composed of three sub-folders :

• apriltags : this folder contains the sources of a program mandatory to make this demonstration work. Please see the apriltags/README.md file for more informations
• simulator : this folder contains the sources for the simulator
• uav : this folder contains the sources for the uav program (simulation and real)

Compilation

apriltags program

Before going further, you have to compile the apriltags program. To do it, go in the apriltags folder and launch the make.sh file :

cd apriltags
./make.sh

If it fails, you have to read the specific compilation instructions of the apriltags component. You can find those instructions in the apriltags/README.md file (or direct link : ​here).

Simulator and uav programs

Normally the components for this demonstration are automatically compiled when Flair framework is compiled.

If not or if you want to compile the components yourself, here are the instructions.

For the simulator :

cd $FLAIR_ROOT/flair-hds/src/demos/ApriltagFollower/simulator
$FLAIR_ROOT/flair-dev/scripts/cmake_codelite.sh
cd build
make -j4

For the uav :

cd $FLAIR_ROOT/flair-hds/src/demos/ApriltagFollower/uav
$FLAIR_ROOT/flair-dev/scripts/cmake_codelite.sh
cd build
make -j4
cd ../build_arm
make -j4

Launching simulation

BEFORE LAUNCHING THE SIMULATION MAKE SURE YOUR FLAIR VERSION IS UPPER THAN 0.1.2 OTHERWISE IT WON'T WORK !

In order to launch the simulation you have to launch several components :

• Flair Ground Station
• Dualshock3, to be able to control an uav using Dualshock3 controller
• The apriltags_detector program
• The simulator
• An uav for the uav holding the Apriltag
• An uav for the uav following the Apriltag

Each program has to be launched in a separate terminal.

Flair Ground Station

$FLAIR_ROOT/flair-bin/tools/scripts/launch_flairgcs.sh

Dualshock3 controller

If the Bluetooth is configured on your computer (see this page) you can launch the Bluetooth version :

$FLAIR_ROOT/flair-bin/tools/scripts/dualshock3_local_bt.sh

If not, use the USB wire instead :

$FLAIR_ROOT/flair-bin/tools/scripts/dualshock3_local_usb.sh

Important note : by default, the Dualshock3 is connected to the port 20000. It suits with the uav holding the Apriltag, not the one following it.

apriltags_detector program

Go in the apritags folder and simply launch the apriltags_detector program :

cd $FLAIR_ROOT/flair-hds/src/demos/ApriltagFollower/apriltags
./apriltags_detector

Important note : this program has to be launched BEFORE the uav following the Apriltag ! Otherwise it will crash...

Simulator

The simulator script manages two uavs, you don't have to launch a simulator for each one.

cd $FLAIR_ROOT/flair-hds/src/demos/ApriltagFollower/simulator/build/bin
./simulator_x4.sh

Uav with the tag

The uav program for the uav holding the tag does not matter : you just have to be able to control it using the dualshock3 controller. So you can for instance use the uav from the CircleFollower demonstration :

cd $FLAIR_ROOT/flair-src/demos/CircleFollower/uav/build/bin
./x4.sh

This uav will be controlled using the controller. So wait to takeoff the follower uav before moving.

Uav following the tag

The uav program for the uav following the tag, you have to use the ApriltagFollower uav :

cd $FLAIR_ROOT/flair-hds/src/demos/ApriltagFollower/uav/build/bin
./ApriltagFollower.sh

You don't have a controller to control this uav. So you first have to take it off using the ground station and clicking on "takeoff" button. Once it's done, takeoff the other uav. After the two uavs have taken off, you can start to follow the Apriltag. Simply click on "start_following" on the ground station. If you move the other uav using the controller, the follower should move to follow it.

Launching real demonstration

The real demonstration is a little bit more complicated to launch has there are several more configuration steps and specific hardware...

Specific uav and raspberry pi

First thing different for the ApriltagFollower : you can't use the uav you want ! Indeed, because of the Raspberry Pi connexion you have to use specific uavs, which have a modified kernel to support Ethernet-over-USB feature. In the laboratory there are two uavs modified for this demonstration, stored in a special area. So if you want an uav, you have to ask Guillaume for one.

Once you have the uav, you will need to get a Raspberry Pi 3 board. In theory you have one board with the uav. If not, things will be a little bit more complicated as you will have to find a board and flash the correct distribution with the correct programs...

Okay, you have an uav and a raspberry pi, now you have to connect them together ! If you are lucky, it's already done, if not, take a look to the next section.

Creating a SD-card

If you have to create a SD-card for the Raspberry Pi, this section is for you. If not, you can skip this section and go to the next one.

We have saved an image of the operating system for the Raspberry Pi so you don't have to reinstall all from scratch. Download the image and uncompress it :

wget https://uav.hds.utc.fr/src/rpi/latest/2018-10-26_apriltagfollower_demonstration_rpi.img.tgz
tar xzf 2018-10-26_apriltagfollower_demonstration_rpi.img.tgz

Then plug the SD-card and launch the flashing process. We assume your SD-card is on the mmcblk0 port :

dd if=2018-10-26_apriltagfollower_demonstration_rpi.img bs=4M | pv -pterb | sudo dd of=/dev/mmcblk0 bs=4M

Extra information about network

If you use your image for the Raspberry Pi, you may encounter some troubles with network communication.

If you can't connect to the voliere router, please add these few words in the /boot/cmdline.txt file of the Raspberry Pi system :

cfg80211.ieee80211_regdom=GB

The voliere router hds_platforms_roam use the channel number 13. But this channel does not exist in America, only in Europe. So you have to telle your Raspberry Pi that you are in Europe (America by default).

The Raspberry Pi USB network is configured as follow (interface name : usb0, ip address : 192.168.7.36) :

# Ethernet/RNDIS gadget (g_ether)
# ... or on host side, usbnet and random hwaddr
allow-hotplug usb0
iface usb0 inet static
	address 192.168.7.36
	netmask 255.255.255.0
	network 192.168.7.0
	gateway 192.168.7.1

If you want to change the configuration you will need to change a few extra things to make the communication between the programs on the uav and on the Raspberry Pi work.

Connecting the Raspberry Pi and the uav

If you have the Raspberry Pi in one hand and the uav in another hand, this section will explain to you how to connect them together. If the two elements are already connected together, you can directly go to the next section.

First of all, you need a specific wire to connect the Raspberry Pi and the uav. The wire is a modified double usb male and looks like that :

So you have 4 elements to plug. Each usb port go in a female, the one going on the uav is indicated. Please be careful to plug the correct usb on the correct side otherwise it won't work. Below is a picture to connect the uav usb wire :

Then plug the two wires going on the Raspberry Pi. You can plug the usb port where you want. The small other connector goes in a specific place, as you can see in the picture below. Please be careful when you plug this connector, if you plug it anywhere else you will destroy the Raspberry Pi !

The last connector goes under the uav, like the picture below. You can't get wrong.

Power on the uav

Nothing complicated, you just have to plug the battery and to verify that the Raspberry Pi is also powered on (a red and a green light must be on).

MANDATORY STEP : Wait 1 minute (a real minute, if not, it will fail !) to let the Raspberry Pi time to boot and then you have to unplug the USB on the Raspberry Pi and to replug it.

On the computer

You just have to launch the ground station and the dualshock3 controller :

Ground station :

$FLAIR_ROOT/flair-bin/tools/scripts/launch_flairgcs.sh

Dualshock3 controller, replacing the uav_ip by the real ip of the uav (192.168.147.xxx) :

$FLAIR_ROOT/flair-bin/tools/scripts/dualshock3_usb.sh uav_ip

On the Raspberry Pi

First of all, you have to connect to the Raspberry Pi (the IP address may differ) :

ssh pi@192.168.147.214
password : raspberry

Next you have to check that the USB connection with the uav is working :

ifconfig

If you can see a connexion named "usb0" it's okay, you can go further. If not, unplug and replug the USB connector from the Raspberry Pi. If you still can't see the "usb0" interface, then you have a big problem... Ask Guillaume, the demonstration will not work without this interface.

The only program you have to launch on the Raspberry Pi is the apriltags_detector program. Depending on where it is installed, just launch it :

cd somewhere/apriltags
./apriltags_detector

On the uav

On the uav you have to launch the uav program.

First, upload it if it's not done (replace the uav_ip_address and folder_location by your values) :

scp $FLAIR_ROOT/flair-hds/src/demos/ApriltagFollower/uav/build_arm/bin/* root@uav_ip_address:folder_location

Connect to the uav :

ssh root@uav_ip_address

Go to the correct folder and launch the uav :

cd folder_location
./ApriltagFollower_ardrone2.sh

It may crash because the dsp program is not present. You can get it from the flair-hds repository.

scp $FLAIR_ROOT/flair-hds/bin/lib/dsp/dspcv_dsp.out root@uav_ip_address:folder_location

Launch the demonstration !

Okay, everything is setup correctly. Before taking off the uav, make sure you have an Apriltag to follow in the area (hold by a static uav, a person, no matter).

It's time to takeoff, press the START button of your controller.

Go manually in about 1 meter from the Apriltag and launch the following process, using the SQUARE button.

Now you can smoothly move the tag for a beautiful demonstration ;) Do not move the tag too fast as the follower will not see it anymore.

Here is a summary of the controller buttons :