Welcome to PhantomPilots.com

Sign up for a weekly email of the latest drone news & information

[TOOL][WIN] Offline TXT FlightRecord to CSV Converter

Discussion in 'Pro/Adv Discussion' started by ferraript, Mar 6, 2016.

  1. Trafcon

    Joined:
    Jan 11, 2017
    Messages:
    7
    Likes Received:
    0
     
  2. Trafcon

    Joined:
    Jan 11, 2017
    Messages:
    7
    Likes Received:
    0
    AMEN !!!!!!!
     
  3. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    I bought Mavic Pro (Platinum), did a few TXT conversions and found out that some fixes have to be done, so new version is out

    batterySN values in both RECOVER and DETAILS section were fine for older types of the drones, but reversed for the newer ones - it should be fine now

    on my very first flight I had compass error with situation when Mavic switched to Atti and didn't use GPS signals
    looking in the logs I realized few things, for example, that GPS values are still there and they look legit,
    but also that although the Mavic was moving at the time, CUSTOM.hSpeed showed 0 (or very low values),
    which means that this field represents only what Mavic thinks at the current circumstances

    so there is new column CALC.hSpeed, calculated from the GPS coordinates (sometimes the values are little bit off, because GPS coordinates don't change precisely, but in general it will give you good values)
    CALC.hSpeed.running_max is now based on this new CALC.hSpeed column
    new column CUSTOM.hSpeed.running_max is based on "official" CUSTOM.hSpeed column
     
    Hans 75, speleo and sar104 like this.
  4. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,305
    Likes Received:
    946
    Location:
    Indy, USA
    @ferraript
    have you by chance checked values using "mi/h" anytime lately? I have updated to the current most recent version(2018/04/21) of the tool and seem to have ran into issues in all hSpeed areas. In testing the data from a Mavic Pro settings are set to display in "feet" and "mph". None of the D column or CUSTOM.hSpeed [mi/h] display correctly. The H column appears to display some of the values double the amount. I have not yet tried using metric as imperial setting are the usual for me. Has anyone else mentioned such issues that you are aware of? And, is there a fix for this? Thanks.

    *Edit: The metric displays in metric, but also does not display the values correctly as well as almost double amount in some cases.
     
    #564 flyNfrank, Apr 29, 2018
    Last edited: Apr 29, 2018
  5. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    long time no see :)
    if you can, attach some txt log that gives you bad values
    and also write here at least one example of what values exactly you don't like and what do you expect them to be so I can evaluate it
     
  6. DasPhantom

    Joined:
    May 20, 2018
    Messages:
    3
    Likes Received:
    1
    Hi, I ended up here looking for a way to add a telemetry overlay to my P3S videos. Thank you very much for the tool to do the conversion offline!

    Now my only problem is which data profile to choose in DashWare (which works fine for me since I use Win7). In an older video on YT, the guy selects Flytrex. But for me this generates an error:

    "Error locating required column(s): You must map a data column to the "<Required> / Data Running Time Seconds" data type."

    So which one of the many profiles would work with the current output of the tool? Thanks in advance.
     
    embayweather likes this.
  7. JohnF

    Joined:
    Jun 3, 2018
    Messages:
    1
    Likes Received:
    0

    Given that you won't publish the algorithm can you at least add an option to output the data in a .GPX format? That's the format I ultimately need it in and have written an app to convert the .CSV file to .GPX. I'd much rather have this as a one step process.
     
  8. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,305
    Likes Received:
    946
    Location:
    Indy, USA
    Good to see you too. I apologize for not responding back before now. I have been working on so much other stuff. As you might remember I have always been a freak about researching the data of each of my flights. And since discovering the issue I mentioned above I simply stopped flying since the data was not displaying correctly. Let me get back into this and refresh my memory with and I will post again soon. :smiley:
     
  9. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    it should be doable, let me think about it
    by the way, I also create GPX files for myself, but I use CSVs generated by DatCon (from DAT logs) as the source, because TXT logs often have large gaps

    good luck :D
     
  10. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,305
    Likes Received:
    946
    Location:
    Indy, USA
    incorrect2.jpg
    ******************************************************************************************** incorrect.jpg


    Ok, so I did not look at all the categories for errors. There are 3 errors above that I would call common as this same issue has come up before from previous firmware updates to the GO4 app. I don't know why DJI has to make changes to these type of files but it probably explains why they have some of the issues they do from time to time. If I were to predict when these changes took place I would guess it changed when the Platinum came onto the scene.

    @ferraript , thanks for looking into this!

    *EDIT: I just found the Battery Cell #1, #2, #3, data amounts are off and need attention.
    Also, the SMART_BATTERY.volumeConsume (EE category) appear to be way off.

    I'm sorry for being all over the place with this info. But I'm at this point I hope there is an easy fix like there was in the past. I will edit more if I happen to stumble upon any additional.
     
    #570 flyNfrank, Jun 5, 2018
    Last edited: Jun 6, 2018
  11. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    it's no problem at all to report problems/errors
    fixing is easy part, but I need to distinguish which logs need to be adjusted

    SMART_BATTERY.volumeConsume ... I don't even know what this is, so I can't tell if the values are ok or not - if I look into logs from my P3A or Mavic, they don't make any sense to me
    Battery Cell #1, #2, #3 ... these values look fine for logs I've got

    CALC values were added just recently and I explained that 1. they are for reference only, 2. they are calculated from GPS coordinates, so they won't be precise all the time
    so there is nothing I can/will do about them
    maybe I will remove CALC.hSpeed.running_max column as it is really useless

    with CUSTOM.hSpeed (or any other potential columns) it's the problem I started to explain
    for example for my logs these values are fine
    and I suppose these values were also fine for your logs, but in one moment they started to be off
    so if you want me to fix this, I will need cooperation of you and other users to find this moment, so I can distinguish, whether I read/write the values just like I did before or I need to use new method (in this specific case to multiply them by 10)
    some of the possible detection signs could be logVersion (11th byte of TXT log), OSD.flycVersion or DJI GO version
    and of course I need to know if it's only iOS relevant or it's valid for Androids too

    if it's too complicated for you, then easier way could be to send me 2 consecutive logs, where first log's values are still fine, but second log's values are off
    and I will try to adjust my tool according to information I'll get from them
     
  12. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,305
    Likes Received:
    946
    Location:
    Indy, USA
    If you can provide an email address I can send you a good(old) & not good(new) files.

    I spent a little more time looking throughout the converted version of my most recent flight and there are other red flags popping up, some columns no longer have data, and most columns appear to have NO problems. Because of this is why I should send the files to you.

    What bothers me about this is why has nobody else mentioned this before I discovered it? Is others just not having any issues, or not paying attention and noticing a revision is needed?

    Btw, this is on Android. My good(old) file is from app version 4.1.8. The not good(new) file is from app version 4.1.22.
     
  13. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    PM sent

    >why has nobody else mentioned this before I discovered it?
    no idea, maybe most of the people use DatCon and CsvView as they provide full flight data (I also prefer DatCon as TXT logs have large gaps, especially when AC is far away) and luckily DJI is not changing the structure/format of DAT logs like they are doing in DJI GO
     
  14. BudWalker

    Joined:
    Jun 24, 2015
    Messages:
    1,282
    Likes Received:
    469
    Location:
    Santa Margarita, Ca
    Another reason is that the columns most people use don't have problems. An exception is the recent problem with the the individual battery cell voltages. There were a lot of people that noticed that which lead to the fix you did.

    TXTlogToCSSVtool is widely used. Especially for the lost drone scenario.
     
    sar104 likes this.
  15. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    new (major) version is out
    - CALC.hSpeed.running_max removed as it was useless
    - the output filename is now automatically assigned according to the input filename
    - problem reported by flyNfrank should be solved now
    - added possibility to export GPX file (also in command line version - parameter G) as requested by JohnF
    - new group of records - CAMERA_INFO displays information about what camera is doing, SD card info, etc...
    - new group of records - MC_PARAM displays DJI GO settings of failSafe and Obstacle Avoidance (2 notices: 1. this record is present in the TXT log on every 100th row so the real delay can be more than 10 seconds; 2. I tested it myself and sometimes it doesn't report correct data, so we can't completely rely on it)

    notice: we are now at 274 columns so if there is someone like me who used to convert CSV to XLS, well, this is now not possible as XLS can hold 255 columns only :(

    BudWalker: FYI
     
    #575 ferraript, Jun 10, 2018
    Last edited: Jun 10, 2018
    sar104 and BudWalker like this.
  16. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,305
    Likes Received:
    946
    Location:
    Indy, USA
    Perfect!

    Everything works great. Thanks so much for taking the time to all that you did. It's much appreciated.
     
  17. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    glad to hear that
    lets hope I didn't break something for iOS users :D
     
  18. Framestamp

    Joined:
    Sep 6, 2017
    Messages:
    3
    Likes Received:
    3
    Thanks for all the work you put into this tool !

    Is there any chance the stored video file name / ID would be exposed as a column ?
    Also, is there an option to output 9 decimal precision (instead of the default 6) using the command line ?

    Thanks.
     
  19. DasPhantom

    Joined:
    May 20, 2018
    Messages:
    3
    Likes Received:
    1
    Quoting myself, since I still haven't found the solution. If no replies mean that it simply doesn't work with DashWare, please inform me, if there is a better way - thanks a lot!
     
  20. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    381
    Likes Received:
    185
    Location:
    Slovakia/Kosice
    no, this information is not present in the logs

    what for? you are pushing it...
    I chose 6 decimal digits because it gives precision 16 cm (6")
    even GPS is not so precise, so there is absolutely no sense of using more decimal digits

    I didn't answer your question because I hoped somebody else would
    I don't use Dashware, can't help you, sorry
    I think somebody found a way, it's somewhere in this thread, so keep looking...
     
Loading...
Similar Threads - [TOOL][WIN] Offline TXT Forum Date
offline Litchi Waypoint upload Software Jul 29, 2018
Offline maps on P4+ - since update Phantom 4 Help Jul 23, 2018
Mapbox - Offline satelite images? Phantom 4 Discussion Jul 8, 2018
DJI Go 4.2.16 and MAPBOX - Offline Control (Go, Litchi, etc..) Jul 5, 2018
DJI Go mod offline waypoint mission (Including spark) Control (Go, Litchi, etc..) May 20, 2018