How high did you fly?

[Llaves]

(There's a new web version of the program described below. The links are here. The general explanation in this post is still valid, but jump to the link to actually compute the true elevations and graph the results)

Regulations around the world restrict drone operations to 120-150m above ground level (AGL) for the majority of users. I've found that it can be very hard to gauge how high I'm flying when the terrain is widely varying, particularly when there are steep rises or drop-offs, which describes the topography for most of my flying. The telemetry graph in the app shows the elevation relative to the launch point, generally measured by barometric altimeter. This graphs tells very little about actual altitude AGL when the terrain elevation changes significantly along the flight path.

Here's the telemetry plot for a recent flight:

Looks pretty mundane - climb 3 meters at takeoff, then a short time later climb to 10 meters for most of the flight before returning.

But this ignores the terrain:

I'm standing at the edge of a steep drop-off over a broad canyon. Within the first 10 meters of flight the ground level has dropped 10 meters. At the lowest point the ground has dropped 25 meters, making the peak altitude 35 meters, not 10. Still within the regs, but not the altitude I'm seeing in the telemetry plot or the exported .csv file.

Here's the profile of the altitude AGL:

Now imagine flying your drone next to the Rio Grande Gorge Bridge outside Taos, NM -

Your flight is level relative to the takeoff point, the controller is telling you the altitude is 3m the whole time. But in reality you max out at 300m, far in excess of the allowed altitude.

I find the problem to be even greater when flying up a hillside. I can't really tell the horizontal distance to the hill and end up flying much higher with respect to the ground than I think I am.

You can export your flight to kml and view it in Google Earth to get a rough sense of your altitude along the flight path, but that's hard to visualize. In addition, I've found the launch point elevation estimate in the flight log to be very inaccurate. The elevation in the log is presumably the GPS elevation, which should be accurate to 10-20 meters. Yet the elevation in the log differs by 50-60m from true elevation. In one instance the difference was 120m! This probably due to the choice of model for the earth's surface. Different models can vary by 100m or more at various points on the earth's surface. Variations of 10-20 meters are common. The key point is that when plotting the height along the path, the same model has to be used.

Here's a Google Earth view of the flight used in the plots above. The orange trace is the kml file exported from the app, using the launch point elevation in the log. The red trace uses a NED10 elevation. (NED10 is a USGS elevation dataset with 10 meter resolution.) The paths differ by 60m. The red path starts on the ground, the orange starts 60m above the ground.

Visualization Tools

You can use this python script to generate the various plots and tables to gauge you ability to know how high you are flying AGL.

First step is to export your flight log to csv using the export button in the upper right of the flight view.
You then run the script:
python ProcessFlightPath.py <flight_log.csv> <output files root name> <Label for kml and plot> --dataset "ned10m" --plot

<flight_log.txt,csv> is the name of the csv file exported from DLB  
<output files root name> is the name without extension for the output files. The script will produce csv, kml, and png files  
<Label for kml and plot>  this name is used for the title of the path in GE and to label the plot  
--dataset "net10m" tells the program to use the more accurate NED10 dataset (valid only for flights in the US.)
           Leave this off to use the worldwide Aster 30m dataset  
--plot - generate a plot as a png file. Leave this off for no plot.  

You may need to install the pandas, requests, and matplotlib libraries. You can do this with
>pip install pandas requests matplotlib

The .csv output file contains a subset of the columns of the input .csv, along with new columns for the NED10 ground level, Distance Along Path, Flight Altitude, and Height Above Ground. The file contains every 10th row of the input file to reduce the number of elevation look-ups.

The output .kml file differs from the one the app will output only to the extent of using a NED10 lookup of the launch point elevation.

If your included the --plot argument, you will also get a .png file with the altitude AGL.

If you are using Excel to view the .csv file generated by the script, Visual Basic for Apps script will add the two plots to your workbook.

If you don't know how to add a VBA script, this is one of many web pages that explains how.

[arpanghosh8453]

@Llaves Thank you so much again for the detailed guide :)

[Llaves]

I've made it even easier to to visualize the AGL on your flights.
This html file.file does all the work right in your browser. You will need a proxy to the OpenTopoData API service. The README on my github repo explains how to set up a proxy.

For now I'm sharing the URL for my proxy. If it gets overloaded I will shut it down, but for now feel free to use it. If you set up a proxy, think about sharing the URL in this thread.
Proxy URL = https://creative-melisandra-droneteam-fb5207f5.koyeb.app

[robo-w]

This overlay would be definitely a great feature for C0 drones. C1+ drones show during the flight the height over ground; Is this data present in the flight records as well? Visualizing this data would help already - if present in the data.

[robo-w]

I checked the produced JSON from dji-log-parser, and found for the home point:

  "home": {
    "latitude": xx.yyy,
    "longitude": xx.yyy,
    "altitude": 368.0,
    "heightLimit": 3000.0,
    "isHomeRecord": true,
    "goHomeMode": "FixedHeight",
    ...
  }

and for the frame:

  "osd": {
    "flyTime": 188.0,
    "latitude": xx.zzz,
    "longitude": xx.zzz,
    "height": 116.0,
    "heightMax": 118.0,
    "vpsHeight": 12.8,
    "altitude": 484.0,
    ...
  }

which means, for the home point there is the ASL present in the data; but for the current position there is only the height over startpoint and ASL altitude in the data - although the remote controller has the information of height above ground during the flight.

[Llaves]

My knowledge of the drone world is pretty limited. Consumer/prosumer DJI drones do not measure height above ground, they measure and display height above launch point. (Some of the drones use lidar and stereo imaging to estimate altitude up to 25m.) This is what is exported to the csv and kml files by OpenDroneLog. The programs described in this discussion take that information and the lat/lon of the flight path to create plots of true height above ground.

[arpanghosh8453]

Hey @Llaves I gave it a try and the HTML file was so quick and easy! Good work. I used the server you provided :)

The issue I had with some logs that if there are missing GPS data for any point, then it just bugs out instead of skipping that point for the record (leaving gap in data plot is fine)

{
    "error": "Unable to parse location 'NaN,NaN' in position 36. Latitude must be between -90 and 90. Provide locations in lat,lon order.",
    "status": "INVALID_REQUEST"
}

[arpanghosh8453]

@Llaves I had some more ideas,

how about an external tool (script) that takes the KML file directly (it's a text file anyway), uses open topo data or https://api.open-elevation.com/api/v1/lookup?locations={lat},{lon} to get the topology elevation for each co-ordinate in batches of 100 (improve this with grouping points within 2 meters), then find out the height difference from the takeoff point for each point (subtract the takeoff point absolute altitude from all other), then subtract the relative elevation from the elevation of takeoff point for each position - I think that will give proper AGL

Basically relative height from takeoff - relative elevation wrt takeoff = AGL

That way, it takes the KML file and outputs an KML file that can be integrated with the Google maps etc seamlessly for focused analysis of specific logs. What do you think of this approach?

[Llaves]

can you send me a sample CSV with NaN entries?

[arpanghosh8453]

@Llaves try this one.

DJIFlightRecord_2023-06-20_15-59-26_.csv

also, tell me your thoughts about the KML approach.

[Llaves]

There is now a suite of related web pages for processing the csv output of ODL.

FlightPathProcessor accepts the csv file, looks up the ground elevation, and produces graphs of the path over terrain as well as a graph of elevation above ground level.

FlightChartViewer - accepts the csv produced by FlightPathProcessor and reproduces the charts without having to fetch the elevation data again.

FlightPathSRT accepts the csv produced by FlightPathProcessor and produces a subtitle file to use with a screen recording from the controller. The subtitle contains a timestamp and altitude above ground level. This might help you associate a camera view with the altitude.

[Llaves]

are there are speed units allowed?

[arpanghosh8453]

@Llaves There are 4 units for the speed, kmh, ms, fts and mph

The issues of missing data arises due to the problem of the upstream parser, nothing to do with ODL code itself. I think height_m or height_ft is more reliable than VPS height (which is capped at 25m due to limitation of sensor).

The error in the altitude can be due to technical fault or corrupt log - the CSV file provided was from a problematic log from an enterprise drone, not a consumer one. Consumer drones behave better in this regard.

[Llaves]

I realize the missing items in the csv come upstream. I just wonder what was going on with the flight recording.

The "Prefer_VPS_height" means when the VPS_height is less than 10m, use VPS instead of altitude_m. This is roughly the limit of reliable VPS height.

I'll add all the other unit options in the next day or two

[arpanghosh8453]

@Llaves Yes, the units can be mixed in the interface as well as CSV exports. You can read the assignments either from the Metadata or the column headers.

You have to apply unit transformation accordingly!

[Llaves]

All three programs list in this comment have been updated to accept all the units and headers in the ODL .csv file. Let me know if I've missed one.