RAW is processed, JPG is not

[Tomas Wangen]

Hi. I read many answers and I do get you. I am a professional photographer (and I´d like to think I am casue I do a good living out of that and have won some important prices, and I say this just to set that I do know about what I am talking about as you do) and I have to say I agree with you 100% I have the same problem. My RAW or DNG file comes out processed, exactly like the JPG should look like. I have chatted with DJI and of course, that´s a dead end, as many answers here that say that´s normal. I have shot with many cameras and drones and have never seen this problem (again, even if many says is normal and not a problem).

The only solution I can find and even more if you have sent the drone back and got another one, is just to overexpose your shots. Contrast and saturation might not be correct, but at least the starting point between the light and shadows will be "correct". Any thoughts?

Follow the normal principles for photography and you will get good results.

That is:

* Expose-to-the-right but please do not over-expose. Those highlights will not come back
* Make sure you use the base ISO unless light does not permit
* Follow a regular post-processing workflow with extra care for color noise (those photosites are awfully close on this sensor).

That's it.

 

[RicardoUK]

Follow the normal principles for photography and you will get good results.

That is:

* Expose-to-the-right but please do not over-expose. Those highlights will not come back
* Make sure you use the base ISO unless light does not permit
* Follow a regular post-processing workflow with extra care for color noise (those photosites are awfully close on this sensor).

That's it.

Wow your post is a bit funny. Probably you didn´t read carefully. I do make my living of taking photographs and never take photos in manual. So I do know how to expose correctly and I know when to pump up the ISO etc. But If the DNG or RAW file from the P4P are coming out processed and underexposed even if I exposed correctly, then a temporary solution is to overexposed in order to get more light into the sensor therefore when reviewing the DNG it´s starting point will be at least ok in terms of exposure, even if it´s still contrasted and saturated. Then I can change it more freely.

 

[Tomas Wangen]

Wow your post is a bit funny. Probably you didn´t read carefully. I do make my living of taking photographs and never take photos in manual. So I do know how to expose correctly and I know when to pump up the ISO etc. But If the DNG or RAW file from the P4P are coming out processed and underexposed even if I exposed correctly, then a temporary solution is to overexposed in order to get more light into the sensor therefore when reviewing the DNG it´s starting point will be at least ok in terms of exposure, even if it´s still contrasted and saturated. Then I can change it more freely.

There is always a chance there is something wrong with your sensor if you get bad results following the guidelines above. If you post a example here (raw), we can of-couse check this.

 

[RicardoUK]

There is always a chance there is something wrong with your sensor if you get bad results following the guidelines above. If you post a example here (raw), we can of-couse check this.

I will try to upload the raw file tonight. Does guidelines are great for someone who does not know about photography and I appreciate it. That's ABC of photography. Trust me it has nothing to do with the settings. And also it has nothing to do if the RAW file can be edited in post. It´s a matter of that the RAW file is not coming out as shot. Is not coming out unprocessed. its coming out underexposed and with process applied.

 

[Tomas Wangen]

I will try to upload the raw file tonight. Does guidelines are great for someone who does not know about photography and I appreciate it. That's ABC of photography. Trust me it has nothing to do with the settings. And also it has nothing to do if the RAW file can be edited in post. It´s a matter of that the RAW file is not coming out as shot. Is not coming out unprocessed. its coming out underexposed and with process applied.

A lot of people are reading this (and maybe getting worried about the quality) so repeating the basics is always in it's place. I will have a look at the sample once it is uploaded.

 

[RicardoUK]

A lot of people are reading this (and maybe getting worried about the quality) so repeating the basics is always in it's place. I will have a look at the sample once it is uploaded.

I will upload the files later. But this is not a matter of quality. This is a matter of a process applied to a Raw files that should be applied unless one those it in post in lightroom, photoshop or other photo editor.

 

[ianwood]

Maybe someone like Tomas can post DNG v JPEG shots of the camera chart showing the extra latitude in the DNG (similar to what he did here). I am certainly not seeing any loss in the shadows. Noise? Yes. But that is to be expected. I really do think this is at most a metadata issue and is just a matter of moving sliders.

 

[gr8pics]

I will try to upload the raw file tonight. Does guidelines are great for someone who does not know about photography and I appreciate it. That's ABC of photography. Trust me it has nothing to do with the settings. And also it has nothing to do if the RAW file can be edited in post. It´s a matter of that the RAW file is not coming out as shot. Is not coming out unprocessed. its coming out underexposed and with process applied.

Yes Ricardo, thats what ive been trying to explain and document for so long now, but all the other "pros" in here keeps chewing me off.
The only remedy and the way to cheat this hyped up camera, is to overexpose the areas where it hurts the least, the sky, and then expose after the shadow areas, so you will only have to do minimal adjustments to the shadow areas in post, wheres theres already visible noise before you even start pulling the raw file.
Then replace the sky with a new one, then you can blow it out until the breaking point, where the sky meets the ground, but not beyond the limit where the light from the sky starts to bleed into the ground details.

Video is another story, and performs much better. Ive said this before, about the previous Phantoms, and i still think it applies, this is not still camera, its a decent video camera.
I think well have to wait another generation or two before we can start to classify this a something close to pro. It really doesnt take better stills than any new iPhone or Android, and maybe not even that.

No one here knows how Dji designed this camera to work, what is included in the raw file when its saved from the sensor.
I will repeat again my post #91:

Raw image format - Wikipedia
"Some camera manufacturers do additional processing before saving raw files; for example, Nikon has been criticized by astrophotographers for applying noise reduction before saving the raw file."

If Nikon can, then certainly Dji can too, any camera manufacturer can, and im not the only one in here being suspicious about it...

Of course i expected allot more from this camera, after reading articles and cross references to Sony RX 100, but its not even close, not even to MKI, as i still have myself.
Ive been hoping and waiting years for these cameras to deliver better pics, but ill have to wait more, cause this one still only delivers marginally better than the previous one, from a professional point of view, of course.

 

[danieljmoore]

Solution... Color correct the JPEG and use that XD... #pro.

 

[Tomas Wangen]

Raw image format - Wikipedia
"Some camera manufacturers do additional processing before saving raw files; for example, Nikon has been criticized by astrophotographers for applying noise reduction before saving the raw file."

If Nikon can, then certainly Dji can too, any camera manufacturer can, and im not the only one in here being suspicious about it...

If you follow the reference in the wiki, you will see that Nikon was accused of using median filter for the three groups of photosites (essentially averaging the photon counts for neighboring photosites for each color) in two modes for one camera to reduce excessive noise for very long captures. I had two D70 back in 2004 (and remember the controversy) and I never noticed the problem, but then again I don't do astrophotography.

This has nothing to do with post-processing in general before saving (averaging actual photon-counts before saving was at best as a hack). Nikon did not process the image after de-bayering and then encode it back to photon-counts before saving. Dji certainly does nothing of the sort (as it would serve absolutely no purpose whatsoever while at the same time add a considerable processing chain).

Speculation of this without any kind of data backing it up serves no purpose that I can see.

That being, there might very well be an issue with one or more sensors in the field, but I have already demonstrated that there was technically nothing wrong or unexpected with OPs raw-capture posted earlier (on page 2).

 

[RicardoUK]

Solution... Color correct the JPEG and use that XD... #pro.

Terrible idea if you want to print in large format and magazine, due to the fact that JPG although flat and almost no contrast it is low res and for PRINTING it´s useless.

 

[RicardoUK]

Yes Ricardo, thats what ive been trying to explain and document for so long now, but all the other "pros" in here keeps chewing me off.
The only remedy and the way to cheat this hyped up camera, is to overexpose the areas where it hurts the least, the sky, and then expose after the shadow areas, so you will only have to do minimal adjustments to the shadow areas in post, wheres theres already visible noise before you even start pulling the raw file.
Then replace the sky with a new one, then you can blow it out until the breaking point, where the sky meets the ground, but not beyond the limit where the light from the sky starts to bleed into the ground details.

Video is another story, and performs much better. Ive said this before, about the previous Phantoms, and i still think it applies, this is not still camera, its a decent video camera.
I think well have to wait another generation or two before we can start to classify this a something close to pro. It really doesnt take better stills than any new iPhone or Android, and maybe not even that.

No one here knows how Dji designed this camera to work, what is included in the raw file when its saved from the sensor.
I will repeat again my post #91:

Raw image format - Wikipedia
"Some camera manufacturers do additional processing before saving raw files; for example, Nikon has been criticized by astrophotographers for applying noise reduction before saving the raw file."

If Nikon can, then certainly Dji can too, any camera manufacturer can, and im not the only one in here being suspicious about it...

Of course i expected allot more from this camera, after reading articles and cross-references to Sony RX 100, but its not even close, not even to MKI, as i still have myself.
Ive been hoping and waiting years for these cameras to deliver better pics, but ill have to wait more, cause this one still only delivers marginally better than the previous one, from a professional point of view, of course.

I understand perfectly where you coming from and the issue. Is exactly my issue. I really hope (maybe stupidly) that since Phantom 4, or Phantom 3 Pro RAW files did not performed this way on the RAW files) that in a firmware upgrade they might fix it. You, a couple of other people in the answers and I know what are we talking about.

I will be trying just to expose well and then whatever are the good settings for that shot then I will over expose by 2 steps. That way I will rescue more shadows and still don´t think will blow out the sky. It´s an extra useless step while I hope and wait for a fix in a firmware.

 

[danieljmoore]

Terrible idea if you want to print in large format and magazine, due to the fact that JPG although flat and almost no contrast it is low res and for PRINTING it´s useless.

Meh, it'll be fine lol. No one will notice.

 

[RicardoUK]

Meh, it'll be fine lol. No one will notice.

I will noticie hehe and if I notice it on my own photos that´s enough to be wrong.

 

[Andrei V]

I understand perfectly where you coming from and the issue. Is exactly my issue. I really hope (maybe stupidly) that since Phantom 4, or Phantom 3 Pro RAW files did not performed this way on the RAW files) that in a firmware upgrade they might fix it. You, a couple of other people in the answers and I know what are we talking about.

I will be trying just to expose well and then whatever are the good settings for that shot then I will over expose by 2 steps. That way I will rescue more shadows and still don´t think will blow out the sky. It´s an extra useless step while I hope and wait for a fix in a firmware.

What you're both complaining about has nothing to do with DJI and everything to do with how Adobe Camera Raw sets the initial settings for processing a Phantom 4 DNG file. For the record, I don't believe ACR has a specific profile for the P4P camera, but does have specific profiles for the P4 and P3. For the P4P, ACR is probably just applying a default profile or relying on DJI's metadata in the DNGs themselves.

As others have tried to explain, the darker, higher-contrast initial rendition of the P4P DNG files is just a gamma curve applied when the DNG file is debayered by ACR. It is not an inherent property of the DNG file itself. At most, the gamma curve is metadata, but you can adjust it to your taste by sliding a few sliders around in your raw processing software. All the information in the shadows the camera is capturing is still there. If that wasn't the case, you'd get better results in terms of dynamic range and shadow noise shooting JPEGs rather then DNGs. So far, neither you nor gr8pics have shown that.

Rather, you've complained that by default the gamma curve ACR applies is more contrasty then you're used to from other cameras. And this crushes the shadows/blacks. And when you raise the shadows/blacks, you get noise in the shadows. Is that same noise not there if you shoot the same scene with the same exposure settings but in JPEG rather than DNG?

I fear that you may mislead others who read this thread. Your "solution" of overexposing the image has the tradeoff of clipping your highlights. It also is unnecessary, given that the problem you've described--to the extent it's a problem at all--isn't an actual issue with the DNG files themselves, but merely the default gamma curve applied by ACR. Obviously, you will get less shadow noise if you overexpose the image two stops. That's physics. The more light you give a sensor, the lower your random photon noise will be. But you'll also blow out your highlights, which, depending on the scene, may contain important details. It seems much easier to me to just slide a few sliders around in ACR...

At the end of the day, the P4P is still just a 1" sensor. It's going to be roughly 2.7 times noisier than a full frame DSLR sensor, all else being equal. The images it produces may be too noisy or not detailed enough for your particular work. If that's the case, you'll need to fly a camera with a larger sensor and better optics.

I own or used to own a Sony RX100 (another 1" sensor) along with several m4/3, aps-c and full frame cameras. Compared to my full frame Sony cameras, the P4P is certainly noisy. But no more so than the RX100 was.

Here's a couple of shots, taken as DNGs, and processed to taste:



Perhaps you can get that kind of quality from your cellphone. I certainly can't.

 

[Daniel W]

This has been an interesting thread to read for many reasons. The one that comes to the fore (unfortunately) is the total dismissal of someone's problems and often blatant disrespect for their problem/arguments. Given that forums are often (by extension) a request for help, comments such as banging your head against a wall are just not productive. If you get that riled, here's a tip: MOVE ON AND LOOK AT ANOTHER THREAD (or, you know, fly your drone, or take a walk, or read a book, or...). Here's a relevant clip that might inject some much-needed humour:

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

So, on to the actual problems raised. I understand completely the OP's concerns, since JPEGs always come out looking better than out-of-camera RAWs (can't believe that I'm having to even say that). That's the role of the JPEG (as well as offering compression) - to do a 'pretty ok' job of applying some saturation, sharpness noise reduction etc. So, again, I completely understand the question being raised based on the examples provided.

I've just completed some fairly comprehensive testing with my P4P and P3A, prompted by this thread. I took JPEG+RAW from both, both at the same camera settings (ISO100, F2.8, 1/100 shutter) and moved the P3A closer to allow for the wider FL. I then imported into ACR and looked at RAW Vs JPEG, P4P vs P3A and a few other matters. My results are:

-that the P4P well outperforms the P3A (whew!). Very noticeable in the DR and SNR;
-that the RAWs look like they are not modified - they're 'muddy' and flat as expected;
-that my JPEGS look better than the unprocessed RAWs. Again, whew!
-that the processed RAWs look better than the JPEGs. (one more time, whew!).

So, this matches what is expected which is P4P>P3A, JPEG>RAW (unprocessed), and RAW (processed)>JPEG.

To the OP: this would tend to make me think that you've either got a defective camera, or you have JPEG settings that are destroying them (and their ability to offer a comparison to the RAWs).

Final note: since it seems to be required to offer qualifications, I am a photographer by profession and have been for many years - I just use the drone to augment the architecture side of my business which is why I read such threads with interest.

Happy flying all!

 

[RicardoUK]

Andre and Daniel

Again. The question is not if the RAW file can be modify in post, or not. In any camera, at least on my experience, It doesnt matter if it´s a medium format file, a full size sensor or it is Phantom 3 Pro or Phantom 4, never the DNG has come out contrasted, saturated and underexposed(regardless is it´s possible to tweak it in post) never ever before I have seen ACR or any other post processing image software apply a "adjustment" to the Raw Images. Never!

We are not misleading anyone here. Maybe some of you are super happy with the RAW on your P4P´s we are not. Regardless of the megapixels or dynamic range, Phantom 4 RAW images came out flat and without any adjustment applied to it.

I know this is wrong, at least to me. Maybe not to you and I respect that, but to me it is wrong.

Here´s the link to the straight from the camera a DNG and a JPG if you guys want to see it. If after seeing them you still tell me that ACR applies and adjustment to the DNG and that´s why it doesn´t come out flat etc... then why does the JPG comes out flat and without saturation etc.... it has NOTHING to do with ACR I am 100% sure, otherwise that would be applied to any other RAW files from any other camera, and that never happened before.

Again, we are not misleading anyone. This are facts. If you are content having your DNG being applied some process, great! I am not!

The firs screen shot is the JPG and the second one is the DNG

And here is a link for the files themselves: Dropbox - RAW - JPG

Dropbox - RAW - JPG

 

[inkytog]

Did anyone see the grabs I posted comparing my 1dx mkii raw and JPEG?? Not for quality inspection but to see that the DEFAULT gamma curve added by ACR is not far off the Edited RAW file. It also shows that ACR does pull the picture style
I set in camera and 'layer' it onto the preview render in ACR. BUT it's just preview render. It hasn't changed the CR2 file from my Canon. It's just non-destructive data.

For some reason there seems to be some confusion about this?

Yes. The way your software chooses to display the DNG may seem strange. YES it looks like it's been processed. BUT it's just the base levels that your computer has decided on laying on the DNG file from the p4p.

If you scroll back you can see the default profile that ACR has applied to the p4p file. It's been embedded at the camera and brought into ACR on import. It hasn't and never would adjust the fundamental, basic digital negative file at the base level. Not at all. It just is approximating what it thinks you want the image to look like.

It's the same for any of my cameras. From the crappy GoPro, my Leica and my Canon. None are pulled in looking washed out. They all look 'ok' and sometimes - of course I don't need to do much editing.

The way you see the DNG file is just the foundation for editing. That's all.

Once you find a look. Save it as a preset and apply it on import. That's all that needs to happen.

What am I missing here?

 

[Meta4]

-that my JPEGS look better than the unprocessed RAWs. Again, whew!
-that the processed RAWs look better than the JPEGs. (one more time, whew!).

So, this matches what is expected which is P4P>P3A, JPEG>RAW (unprocessed), and RAW (processed)>JPEG.

To the OP: this would tend to make me think that you've either got a defective camera, or you have JPEG settings that are destroying them (and their ability to offer a comparison to the RAWs).

I did some testing today and found that if you have your P4pro camera set to D-Log the raw files come out looking better than the jpg files.
But if you set the camera to default values, you get the look that you expect.

 

[RicardoUK]

Did anyone see the grabs I posted comparing my 1dx mkii raw and JPEG?? Not for quality inspection but to see that the DEFAULT gamma curve added by ACR is not far off the Edited RAW file. It also shows that ACR does pull the picture style
I set in camera and 'layer' it onto the preview render in ACR. BUT it's just preview render. It hasn't changed the CR2 file from my Canon. It's just non-destructive data.

For some reason there seems to be some confusion about this?

Yes. The way your software chooses to display the DNG may seem strange. YES it looks like it's been processed. BUT it's just the base levels that your computer has decided on laying on the DNG file from the p4p.

If you scroll back you can see the default profile that ACR has applied to the p4p file. It's been embedded at the camera and brought into ACR on import. It hasn't and never would adjust the fundamental, basic digital negative file at the base level. Not at all. It just is approximating what it thinks you want the image to look like.

It's the same for any of my cameras. From the crappy GoPro, my Leica and my Canon. None are pulled in looking washed out. They all look 'ok' and sometimes - of course I don't need to do much editing.

The way you see the DNG file is just the foundation for editing. That's all.

Once you find a look. Save it as a preset and apply it on import. That's all that needs to happen.

What am I missing here?

I will do further tests because to be it doesn´t make sense. I understand what you are saying. In simpler terms is like if P4P is sugesting a starting point for the RAW file but the data and information still there and has not been modify. Even if that´s the case it shouldn´t be like that, regardless. A very dumb example would be: I buy a lolly pop and it comes full of sugar as a sugestion, if you don´t like it you can take it of hehehe