Position tracking without GPS

In an GPS denied environment one option to track the own position is with a camera which is mounted to see the ground directly from above. With a quadcopter this could be archived with a gimbal.
Two other preconditions are that the ground has good visible structures and there is sufficient light.

With OpenCV there are different optical flow methods available. The first approach used
Lucas-Kanade to track the points in the picture. To detect the movement of the cam it
used just a median of max 100 points. But this approach did not work very well.

The second approach was to build with max 21 points some triangles. There are a lot of
advantages with this triangle approach.

• Check possible if the triangle has the same ratio on the sites
• Calculate the middle of the triangle to calculate the translation
• Calculate the rotation of the triangle
• Calculate the scale of the triangle

In the screencast you can see that it works quite well.

The video to track was recorded with the smartphone and the screencast was done on a laptop of course.

But as you can see on the video, not only the movement can be tracked also the orientation and the distance.
Of course the movement related to the ground has to calculated together with the distance and the rotation.

Because of accumulating errors this navigation is not as exact as with GPS, but dependingon the scenario it might be good enough.

MAVLink tunnel through MQTT

On the opensource project uasmqtt now it’s possible to ‘tunnel’ MAVLink packets. That means it’s not only to control the smartphone from the ground and get it’s data. Through the same message queue mechanism MAVLink packets can send and received.

To get a proof of concept the AircamQC app was extended to get a bluetooth connection to the Quanum Nova/CX-20 APM quadrocopter. Also a small utility was created to listen on a TCP socket and transfer these data to the MQTT broker (TCPmavlink2uasmqtt).

At the picture you can see the working example. The picture itself was taken through the AircamQC app and triggered by the AircamQC Viewer app.

MAVLink packets are different in it’s size and have a starting id (0xFE). The payload length up to 254 bytes is following direct after the starting id. There are also some other importand status bytes so that a packet size is calculated by 6 bytes header + payload + 2 bytes checksum.

The drawback on this approach is the performance. It’s really slow for some reason. A better approach is not to interpred the MAVLink packet stream but to create proper topic structure and json content.
Then the smartphone app can act like a ‘companion computer’ and talk with the copter through MAVLink with all necessary information at the right time, like the heardbeat. But to another aircraft or groundstation the smartphone can talk in it’s own MQTT ‘language’ which should not be too time critical.

The experiment set-up is shown in the following picture.

Experiences with a new platform

For further development on smartphone apps it’s helpful to use a copter with buildin position hold and open source flight control.

Quanum Nova/CX-20 is small, the same size as a DJI phantom, and includes APM flightcontrol. Flight time with a Samsung Galaxy S3 as payload was about 5-6 minutes.

An advantage of the Nova/CX-20 is, that a lot of parts of the phantom also fit into it. For example the props, the prop nuts, the battery and also the cam gear.

Some tips if you buy a Quanum Nova/CX-20.
If it’s a PNF (Plug and fly = without receiver), you should watch the Hobby King video for setup.
Before every flight check if the prop nuts are tight. Otherwise you may loose a prop while flying.

For communication with mavlink tools. The usb connector on the Nova works with a default of 115200 baud, but the serial connection on the flight control has a default of 57600 baud. You can use it by soldering 4 wires on it (telemetry tutorial).

If you want to connect a bluetooth module on it, you should set the speed on the bluetooth module properly.

Not every application is helpful to set the baud rate on the bluetooth module. Minicom works but on OS X and Linux, “screen” was not really helpful and also Arduino IDE was not working.

Open source project for unmanned aerial system on MQTT base

Introduce of a new open source project which uses MQTT with Json data format to communicate between devices. At the moment it contains the parts uavMap to display position and Android app to transfer theses information and also take pictures on demand.

https://github.com/mobile2power/uasmqtt

Why use Message Queue Telemetry Transport with Json data format?

With MQTT you have a broker. Every device can send data and listen to new data complete independently. Means that MQTT has some benefits:

• With Json data format it’s open and independend from programming language
• Easy for testing. No problem to write a Mock and connect it to the broker.
• It’s scalable. Means for example you can have a couple of UAV in the air and control it all with one application. Every UAV can also listen to other UAV to prevent collision.
• With SSL and login or other authentification it’s secured by design.
• There are some mechanisms if connection is broken and in case of a timeout.

But there are also some drawbacks

• TCP has more overhead than UDP. Performance is lower.
• Not possible to have a direct device to device communication. Single point of failure and again lower performance.

But usually these drawbacks are not that critical. Only if you want to transmit live video it might be better to use another technology.

About the applications

The first app for the Android Smartphone just collects sensor values for location (GPS, Pressure Sensor) and attitude (Gyros, Accelerator). It uses Eclipse Paho mqtt client to communicate with the broker and Gson to generate json data format for location, attitude and view angle of the cam.
The app also registers a listener for the broker to get the trigger for the cam. Every photo is stored locally on the Smartphone with Geo tagging. Additional a json file is stored with the additional information about attitude and pressure based altitude.

The second application is a processing app to show a map with the actual position and orientation of the Smartphone. The pressure based altitude is shown as a number and a rectangle displays the covered area by the camera.
On every taken picture the rectangle marker stays in the position to see, what is already covered. Used libraries are UnfoldingMap, Qatja and again Gson.

Configuration

A really simple approach is to connect the Smartphone via tethering to a laptop with uavMap and the MQTT broker on it. But the wlan range has a limit about 50-70 m without the use of directional antenna.

It’s better to have 2 mobile internet connections running and MQTT broker on an internet server. Then there is no range limitation except the accesss to the mobile internet.

Test of MQTT performance for device communication

As alternativ to the selfmade UDP based protocol to transfer video, position and all the other data from a smartphone, the TCP based MQTT has some advantages on the paper:

• Encrypted communication possible (SSL)
• Proven protocol. Early version started more than 10 years ago.
• Standard protocol which means there are a couple of implementations
• Similar approach to ROS (robot operating system)
• Free server/broker available

But to check if the performance is sufficient there has to be a test. For the test the HiveMQ is used local on the Mac and also the receiving program which is created with processing (lib qatja used for MQTT). On the other side a small android app is created which use eclipse paho library to transfer camera preview images (10 fps).

The performance is not bad and the implementation was easy to do with these nice libs.

SimpleFPV copter controlled by smartphone and gamepad

Both apps, SimpleFPV and MobileFPV, are prepared to transmit control information back to the vehicle. That means the cam preview is transfered from the vehicle smartphone to the smartphone next to the user and control information from the user is transfered back to the vehicle smartphone. There is no need to have a link with traditional rc components, but for safety reasons it’s possible to have one.

In the above movie the control chain is: Gamepad via bluetooth to smartphone. From the smartphone via wifi direct to the smartphone within the quadrocopter, from the smartphone via bluetooth to a arduino pro mini. The arduino pro mini is connected by wire to the copter flight control.

As mentioned a traditional rc receiver is also connected to the arduino for safety. It’s possible to switch between smartphone and rc operation at the rc transmitter.

SimpleUAV source code available

If you want to play a bit around with the source code of SimpleUAV it’s now possible. The whole source code can be downloaded from http://www.mobile2power.de/simpleuav.html

Let me know if you have cool modifications or want it to be integrate into your own projects.

MobileFPV and MobileRC app available for free

If you ever wanted to test MobileFPV or MobileRC with two Smartphones through data link, it’s now more easy than before. You can just download it from googles play store.

There is also no need to register with your email address to get a connection token. You can just download the connection server from the download section on http://www.mobile2power.de and start it on your private server.

MobileFPV flight with video goggles

There was another testflight with MobileFPV App and two smartphones. This time one is used as video receiver and goggle. The other smarthone was cam and video transmitter. The video was compressed to 5% per jpeg picture and send 10 times per second. With this poor quality it was possible to get a delay between 1/3 and 2/3 second on a cheap mobile 3g data link.
It’s not really an enjoyment to control the quadrocopter, but possible. For assistance vehicle attitude and orientation as soon as altitude, home direction, speed and distance is displayed in the video goggle.

The transmission used kind of internet proxy located at Berlin. Flight was at Karlsruhe, which means the video and telemetry data travel about 500 km two times (Karlsruhe->Berlin->Karlsruhe).
Because of the use of udp instead of tcp handover points for mobile internet should not be an issue. But a test of handover is still pending.

Short FPV flight with MobileFPV

New test on MobileFPV app. Now the app allows to reduce the data tranfer to 2% jpeg compression and only send 10 frames per second. This is qoor video quality, but fast enough within a 7,2/5,76 Mbit per second mobile internet connection to control a quadcopter on a first person view base.
Usually the app try to get a direct connection to the opposite device, but when this fails a kind of proxy server is used to transfer the data from one device to the other. This was also the case in the above test szenario. That means the data are send from KA to Berlin to the proxy server and back, which is more than 1000 km.
In the video you can see that it’s not that easy to control the quadcopter. On the location there was some wind and the quadcopter was not as stable as it is usually.