Requesting P3A log analysis

I downloaded the DAT files provided by Spike Ike in first post.



I'm not watching an interface, i used "open" to read a packet file (.pcap) stored previously.
And to get the .pcap file, I converted .DAT:
Code:
./comm_dat2pcap.py -vvv -u 3 -e -d FLY100.DAT



Wireshark can't understand DJI protocol by itself. You have to install dissectors:
dji-firmware-tools/README.md at master · o-gs/dji-firmware-tools · GitHub

The export plugin is included in these dissectors. It adds a new option to "Tools" menu.



There is a button battery on the GPS board. I would conclude that this battery supports real time clock, as most drones are somehow able to know current time even with no GPS lock.
It is just a guess though, I never tried messing with the battery.

EDIT: And in case you'll be wondering if converting/opening/exporting has hanged - no, it really goes that slow. Just let it run in the background and watch some YT.

Thank you for that detailed explanation. I've never used Wireshark for that purpose. Is there an advantage to interpreting the DAT file that way rather than using DatCon?
 
Is there an advantage to interpreting the DAT file that way rather than using DatCon?

I'd say these two tools go head-to-head nowadays. Which one you want depends on your use case.

I'm mostly diagnosing hardware, so I want to see the exact packets send by each module, and filter them by specific field values. Wireshark is better for that purpose. Oh yes - I did not mentioned that, but with Wireshark you can look not only at the .DAT logs, but also at communication between modules within the drone (if you tap to any of the serial lines).

But if a flat list of flight log fields is what you want, DatCon is a lot faster - it often takes 10 minutes to open a large log in Wireshark, DatCon does things instantly. And in recent months a lot of new fields were added to DatCon, so I don't think you miss any data by looking only there.

Both tools will allow you to draw nice graphs, but Wireshark has a nice build-in functionality for that.
Both tools have KML export (though the one in Wireshark is far more advanced ATM, @BudWalker needs to catch up there ;) ).
Both tools allow to look at all the fields, though DatCon often formats and scales the fields better, while Wireshark often leaves them in raw form.
In Wireshark you can look at unknown or damaged packets, DatCon just ignores these (though I think both tools can read all Ph3 packets now, so there shouldn't be any unknowns).

I am rarely using DatCon nowadays, so I may be missing some new options there, or be biased in general - I liked Wireshark even before I got into drones :)
 
Last edited:
I'd say these two tools go head-to-head nowadays. Which one you want depends on your use case.

I'm mostly diagnosing hardware, so I want to see the exact packets send by each module, and filter them by specific field values. Wireshark is better for that purpose. Oh yes - I did not mentioned that, but with Wireshark you can look not only at the .DAT logs, but also at communication between modules within the drone (if you tap to any of the serial lines).

But if a flat list of flight log fields is what you want, DatCon is a lot faster - it often takes 10 minutes to open a large log in Wireshark, DatCon does things instantly. And in recent months a lot of new fields were added to DatCon, so I don't think you miss any data by looking only there.

Both tools will allow you to draw nice graphs, but Wireshark has a nice build-in functionality for that.
Both tools have KML export (though the one in Wireshark is far more advanced ATM, @BudWalker needs to catch up there ;) ).
Both tools allow to look at all the fields, though DatCon often formats and scales the fields better, while Wireshark often leaves them in raw form.
In Wireshark you can look at unknown or damaged packets, DatCon just ignores these (though I think both tools can read all Ph3 packets now, so there shouldn't be any unknowns).

I am rarely using DatCon nowadays, so I may be missing some new options there, or be biased in general - I liked Wireshark even before I got into drones :)

Interesting. I use DatCon to export all fields to CSV and then I analyze them in other software. DatCon also computes a number of very useful fields that are not in the DAT file. CsvView does a nice job of graphical representation of the original data though. In what way is the Wireshark kml better than the DatCon kml? For my purposes if it has lat/long/altitude that’s all I need.
 
In what way is the Wireshark kml better than the DatCon kml? For my purposes if it has lat/long/altitude that’s all I need.

DatCon only exports a static path at ground level. And for Wireshark - look at the file I attached. It shows 3D drone moving over the map, plus control sticks. Just remember to set all the "speed" options to slowest in Google Earth, otherwise the simulation will be way too fast. And disable automatic switching to view from earth level.
 
DatCon only exports a static path at ground level. And for Wireshark - look at the file I attached. It shows 3D drone moving over the map, plus control sticks. Just remember to set all the "speed" options to slowest in Google Earth, otherwise the simulation will be way too fast. And disable automatic switching to view from earth level.

DatCon exports a 3D path as an option. No stick inputs though - that’s a nice visualization feature. That said, your file doesn’t work at all in my version of GE Pro. Also I prefer to correlate stick response to flight path quantitatively rather than eyeballing stick motion compared to track, but obviously that’s just personal preference.
 
your file doesn’t work at all in my version of GE Pro.
Oh. Pity. I though it is universal format.

I prefer to correlate stick response to flight path quantitatively rather than eyeballing stick motion compared to track
Sticks is not the only feature, you have 3D model in space. You can see the drone, where it's heading and how it's rotated (or at least how the drone "thinks" he's rotated), in every second.
 
.....

Looking at the packets directly, besides the fact that GPS battery must be bad (the drone thinks it is 2013 for a first few seconds), everything looks ok.
.....
This is normal. Until there are enough satellites and the DOP gets low enough the date will be some where in the year 2013.
 
....
Both tools allow to look at all the fields, though DatCon often formats and scales the fields better, while Wireshark often leaves them in raw form.
....
DatCon (and CsvView) allows you to choose how it parses the .DAT via the Parsing Options on the menu bar.

Choosing Engineered (for reverse engineered) is equivalent of the way it's always been done with the proper scaling, biasing, nice labels, extra calcs etc.

Choosing DatDefined uses the record definitions in the .DAT file itself and is more like Mefistolis's converter DatDefined actually does a little more by using record definitions found in other .DATs if it can. DatDefined has way more data but it's raw - no biasing, scaling, nice labels etc.

Choosing the Engineered then DatDefined option attempts to give you the best of both the above options.
 
Last edited:
Howdy. I'm here to give a long overdue update. The more I learn about phantom analysis the more I realize how little I know about it. I don't think it requires a degree from MIT, but I would not call it simple. I have been busy, and the weather uninspiring, but I got a chance to fly for just a bit. I was very cautious at first, but eventually I had him up over 300ft. and out more than a half mile. No issues, everything went fine. I had a relative with me who knew very little about quads, and he was interested. He was most impressed with it's speed. I touched down briefly, then I lifted off again with the intention of doing a fly-by. I was gonna fly 50 yards away or so, and do a short sprint to give him a better idea of how quick a phantom flies. I intended to go up to about 25ft. and a safe distance away, but close enough for him to get a better perspective. Well, I raised up to about 10 feet and headed away at 1/3 stick to stage for this fly-by. At around 25 yards away I heard a very loud screech and Japser went into the same violent tumble as my original crash. Again, it happened so fast that it looked like he was hit by a shotgun. Again, he was flying perfect, and this happened with no warning. As sad as it was, I was immensely grateful that it happened at 8 ft up and at a brisk jogging speed, rather than the 350 by 3000ft, or whatever I was at just a couple of minuets before. I got it inside (I was close to home) looked it over and found no real damage. But I had a new clue; the screech. It was very loud and irritating, like nails on a chalk board. My thoughts were drawn to the motors. I looked them over closely and didn't notice anything amiss. But as I spun them one seemed just a tiny bit harder to rotate than the others. It was so little that I was not sure that it anything. Tiny. I decided to pull the motor out and really put it under the microscope. After I pulled it out, I inspected it closely. Nothing looked off, and the more I spun it the more it felt like the other motors. It was harder to turn, but like by 0.02%. I was shining a small flashlight inside and moving the motor around as I inspected it when something funny happened. I turned it upside down and was looking inside for any damage, or maybe a broken winding. I went to spin the motor as I had been doing to look all around inside and it was stuck. What?!? I tried again and it spun normally. I kept searching, and eventually found a tint little rock fragment. It was hard to see, near the bottom and tucked into the windings. i grabbed a minuscule screwdriver and attempted to dislodge it. It was stubborn, but after a couple prods it moved. But it didn't fall out. It kinda slid under the armature, around the bottom of the motor. That is when I realized it was magnetized. I couldn't see it at that point. Then I tried to separate the motor. It really didn't want to come apart. I removed the clip and spacer, but it refused to budge. Eventually I got it apart after a lot of prying and tugging. Definitely harder than it should have been. There was a couple deep scratches on the magnets, each jaggedly spiraling around for maybe 200 degrees. Looks like that intruder hid in my motor, waiting for just the right jostle. Then he would jump out, wedge himself into the rigging, crashing my bird. Next he would slink off again, diapering into the shadows. Filthy coward. Anyway, I got a new motor and reassembled everything. I got a chance to put it in the air for four or five minuets the other day. It was misting, and more rain was on the way, but I chanced a quick shake down run. No issues. So it looks like (hopefully) we solved this murder mystery. And that little chip of rock would have gotten away with it, if it weren't for us meddling kids!
 
Howdy. I'm here to give a long overdue update. The more I learn about phantom analysis the more I realize how little I know about it. I don't think it requires a degree from MIT, but I would not call it simple. I have been busy, and the weather uninspiring, but I got a chance to fly for just a bit. I was very cautious at first, but eventually I had him up over 300ft. and out more than a half mile. No issues, everything went fine. I had a relative with me who knew very little about quads, and he was interested. He was most impressed with it's speed. I touched down briefly, then I lifted off again with the intention of doing a fly-by. I was gonna fly 50 yards away or so, and do a short sprint to give him a better idea of how quick a phantom flies. I intended to go up to about 25ft. and a safe distance away, but close enough for him to get a better perspective. Well, I raised up to about 10 feet and headed away at 1/3 stick to stage for this fly-by. At around 25 yards away I heard a very loud screech and Japser went into the same violent tumble as my original crash. Again, it happened so fast that it looked like he was hit by a shotgun. Again, he was flying perfect, and this happened with no warning. As sad as it was, I was immensely grateful that it happened at 8 ft up and at a brisk jogging speed, rather than the 350 by 3000ft, or whatever I was at just a couple of minuets before. I got it inside (I was close to home) looked it over and found no real damage. But I had a new clue; the screech. It was very loud and irritating, like nails on a chalk board. My thoughts were drawn to the motors. I looked them over closely and didn't notice anything amiss. But as I spun them one seemed just a tiny bit harder to rotate than the others. It was so little that I was not sure that it anything. Tiny. I decided to pull the motor out and really put it under the microscope. After I pulled it out, I inspected it closely. Nothing looked off, and the more I spun it the more it felt like the other motors. It was harder to turn, but like by 0.02%. I was shining a small flashlight inside and moving the motor around as I inspected it when something funny happened. I turned it upside down and was looking inside for any damage, or maybe a broken winding. I went to spin the motor as I had been doing to look all around inside and it was stuck. What?!? I tried again and it spun normally. I kept searching, and eventually found a tint little rock fragment. It was hard to see, near the bottom and tucked into the windings. i grabbed a minuscule screwdriver and attempted to dislodge it. It was stubborn, but after a couple prods it moved. But it didn't fall out. It kinda slid under the armature, around the bottom of the motor. That is when I realized it was magnetized. I couldn't see it at that point. Then I tried to separate the motor. It really didn't want to come apart. I removed the clip and spacer, but it refused to budge. Eventually I got it apart after a lot of prying and tugging. Definitely harder than it should have been. There was a couple deep scratches on the magnets, each jaggedly spiraling around for maybe 200 degrees. Looks like that intruder hid in my motor, waiting for just the right jostle. Then he would jump out, wedge himself into the rigging, crashing my bird. Next he would slink off again, diapering into the shadows. Filthy coward. Anyway, I got a new motor and reassembled everything. I got a chance to put it in the air for four or five minuets the other day. It was misting, and more rain was on the way, but I chanced a quick shake down run. No issues. So it looks like (hopefully) we solved this murder mystery. And that little chip of rock would have gotten away with it, if it weren't for us meddling kids!
Which motor had the problem?
 
Thank you for that detailed explanation. I've never used Wireshark for that purpose. Is there an advantage to interpreting the DAT file that way rather than using DatCon?
I've read your post several times and I cannot figure out what you are doing - I must be missing something obvious. So you have the two DAT files somewhere, but which interface is Wireshark watching and how is watching an interface showing you packets of data from those files? And you mention that a function was added to export the flight log position data to a kml. A function in which application? Which GPS battery is bad? It's been a while since I was that baffled by a post on this subject. Can you elaborate?
Hi,
I am trying to use
./comm_dat2pcap.py -vvv -u 3 -e -d FLY100.DAT
to get the pcap file from the DAT file from my DJI spark and getting crc error.
Do you know the seed value I need to pass for spark in comm_dat2pcap.py script?
Need urgent help for a project :)

Zaidul
 
Hi,
I am trying to use
./comm_dat2pcap.py -vvv -u 3 -e -d FLY100.DAT
to get the pcap file from the DAT file from my DJI spark and getting crc error.
Do you know the seed value I need to pass for spark in comm_dat2pcap.py script?
Need urgent help for a project :)

Zaidul

I think you need to direct that question to @quaddamage - that's his technique.
 
I'm not familiar with Spark and its logs. I know that DUML communication format is the same even for latest drones, but FLY*.DAT may have different format or its own encryption.
 
I'm not familiar with Spark and its logs. I know that DUML communication format is the same even for latest drones, but FLY*.DAT may have different format or its own encryption.
I think it is just the seed we need to pass is different in alc_pkt55_hdr_checksum(seed, packet, plength) function. It is just asking for checksum error.
 

Members online

Forum statistics

Threads
143,054
Messages
1,467,297
Members
104,919
Latest member
BobDan