RAW is processed, JPG is not

[F3honda4me]

And this is how misinformation never dies.

 

[inkytog]

And this is how misinformation never dies.

My head hurts.
My wall has a dent in it.

 

[Tomas Wangen]

It seems a continuing misunderstanding here is the use of:

1) A log curve for video capture (where a limited-bandwidth media is used, ie not raw) to better use the available bandwidth of a linear rgb space

2) A raw capture where the original data captured by the sensor (think photon-count for each cell) is stored along with a gamma curve and other metadata.

These are not the same and solves different problems. 1) is used to avoid wasting bandwidth (essentially using it all for highlights with horrible tonality of shadows and partially also mid-tones) and 2 ) preserving the actual original captured data to allow maximum flexibility during post-processing.

If you mix these, I can see that it can be confusing.

 

[Andrei V]

Fact:
A raw file NEVER comes straight of a sensor, a sensor cant even read colors, thats why its a 3 colored filter also known as RGB added on top of the sensor, aka Debayer filter.
The DeBayer filter is used to convert raw image data into an RGB image. In a raw color image, every pixel represents a value for one basic color, instead of three as is the case for an RGB image. In order to get a real color image, the two missing colors have to be interpolated. This is exactly what this filter does.

Actually, this is not what the filter does (the filter is a physical component of the camera, it does nothing beyond filtering which wavelength of light gets through to each of the pixels). The interpolation is what the debayering algorithm (e.g. Adobe Camera Raw, if you're using Lightroom/Photoshop) on your computer does when you open a raw file.

In regards to my OP, forget the jpg vers, it is true what some of you said, that this was much flatter due to D-cinelike profile added, but my concern was only the raw file, which stil in my opinion, have to much contrast and saturation from the STARTING point.
If you start to edit in the far end, you will obviously loose the "maximum" range of "pulling" the picture into the extreme before it starts to crack, this is most visible in the shadow areas where most of the noise will be introduced and noticeable. Thats why it need to be as close to the "center" of the range as possible when you start editing, to be able to take full advantage in any direction, both highlights, and shadows.

With all due respect, I don't think the information you've provided so far supports this conclusion. In a normal implementation, contrast and saturation are not properties of the RAW file -- the raw file is just an array of individual light intensity (i.e. luma) values for each pixel. The RAW file doesn't record color information or contrast (i.e. gamma curve) in this array of luma values, though it may contain metadata based on the color/picture profile you select in camera which INFORMS your post-processing software as to what gamma curve and saturation boosts to apply after the image is debayered. But this metadata doesn't alter or limit the underlying raw luma information for each pixel, which comes straight from the sensor (with perhaps some in-camera noise reduction thrown in by certain manufacturers). This, at least, is what happens in theory.

You seem to be suggesting that DJI is somehow messing up the luma data it's recording in the raw file and thus saving a smaller dynamic range in RAW than in saves in JPEG. This, understandably, comes across as absurd to many of the contributors to this thread, because it would require massive stupidity on DJI's part to actually implement in-camera raw processing which specifically degrades the quality of the raw luma data recorded in a raw file.

Just like DJI failed to make a good D-log file, probably cause they havent got proper "how to" knowledge, i think the tweaking of the raw files can also be improved to take full advantage of 12 stops of DR, even with such small sensor.

You haven't demonstrated that this is happening. I suggest you go ahead and process the raw file by raising the shadows and lowering the highlights and then show us, how, compared to the JPEG file, the raw is retaining less detail in the highlights and/or in the shadows. If that's indeed the case, then I think you'd have a lot more people willing to concede that there's an issue with DJI's raw implementation.

But all you've done so far is show that the default processing of DJI raw files in your raw editing software gives you an image with more contrast than a JPEG shot with the d-cinelike color profile. Why do you conclude this is a property of the data in the raw file rather than a consequence of the way that your raw editing software debayers that data and the contrast (gamma) curve your software applies?

 

[Andrei V]

gr8pics,
Your observations and concerns were spot on from the first post on through. I'm not sure why some folks felt it necessary to get so adversarial.

When I got my Canon 5D mkII 7 years ago, it took me a while to discover that, while I was shooting RAW, my photos were contrasty with blown highlights, crushed shadows, oversaturated, and sharpened in-camera. After a few months of hating my pics, I got schooled on Picture Profiles (names that mean nothing) for sharpness, contrast, saturation and tone. The '0' values are not zero at all. Since then, I use the 'Neutral' (meaningless name) to -4, -4, -4 to get something flatter, still not as flat as a log profile, but good enough to work with.

Seemingly, RAW is an ambiguous term and the sharpness/contrast/ settings are relative to the engineering of the sensor and the processing. Your sample RAW pics do not look anything like any LOG profile I've seen working in professional video. They look contrasty and over-saturated, far from a flat LOG profile. Having to lift the shadows, recover highlights and desaturate in LR to achieve a flatter appearance is the opposite of a LOG workflow. I appreciate your very comprehensive and informative posts in this thread.

RAW =/= LOG

Unfortunately, the two are commonly confused on the internet, and, yes, apparently, by lots of working photographers/DPs.

A raw image - be it DNG, NEF, or whatever other format -- only records the luminance values of each pixel on the sensor.

You cannot DISPLAY a raw image without doing some post processing to it to convert this raw luminance data into an RGB or YUV image that can actually be displayed on a monitor or television. This processing includes:

* debayering -- i.e. applying an algorithm to generate, based on the configuration of the physical bayer color filter on the sensor, a red, green and blue pixel value for each pixel.
* applying a contrast or gamma curve which dictates how contrasty or flat the image looks
* applying a saturation curve which dictates how saturated the colors of your image are.

A LOG video format, by contrast, is a regular RGB or YUV file that has a log-like gamma curve applied (and thus LOOKS flatter). There's absolutely no requirement that you apply a log curve to a raw image when you process it in order to convert it to something you display. The gamma curve used to display a RAW file doesn't affect the underlying dynamic range of that file, just the way it is displayed.

By default, it appears that Adobe Camera Raw processes and displays P4P DNGs with a normal (i.e. contrasty) gamma curve rather than the flat curve that some of you may be accustomed to from other cameras. This doesn't mean the P4P DNGs themselves record less information in the highlights and shadows. If you change the gamma curve in Camera Raw (by lowering contrast, raising shadows and/or lowering highlights), you'll be able to access all the information in the DNG regardless of how it's initially displayed.

 

[gr8pics]

You haven't demonstrated that this is happening. I suggest you go ahead and process the raw file by raising the shadows and lowering the highlights and then show us, how, compared to the JPEG file, the raw is retaining less detail in the highlights and/or in the shadows. If that's indeed the case, then I think you'd have a lot more people willing to concede that there's an issue with DJI's raw implementation.

But all you've done so far is show that the default processing of DJI raw files in your raw editing software gives you an image with more contrast than a JPEG shot with the d-cinelike color profile. Why do you conclude this is a property of the data in the raw file rather than a consequence of the way that your raw editing software debayers that data and the contrast (gamma) curve your software applies?

Certainly, again, for you, and new readers.

Screenshot of 100% after processing them to look as close to each other as possible.
Dng file is even less sharpened.
So, since a raw file contains so much more data, have so much greater DR, it would look allot better than the jpg version that has been processed twice now, right?
So which one look clearly better if you look at the details and shadow areas?
Oh, and dont mention that raw saved straight from the sensor has been cropped more than the jpg.

 

[SmurfN]

you must be intentionally trolling at this point.

 

[gr8pics]

you must be intentionally trolling at this point.

Who are you addressing???

 

[Macfawlty]

Certainly, again, for you, and new readers.

Screenshot of 100% after processing them to look as close to each other as possible.
Dng file is even less sharpened.
So, since a raw file contains so much more data, have so much greater DR, it would look allot better than the jpg version that has been processed twice now, right?
So which one look clearly better if you look at the details and shadow areas?
Oh, and dont mention that raw saved straight from the sensor has been cropped more than the jpg.

They both look pretty crappy to me, nearly identically so. The RAW doesn't look like any RAW file I've seen, edited or not. It's noisy, muddy, contrasty. The discussion on this has been quite informative, but the bottom line is that the RAW file does not look good by any measure.

What's remarkable in RAW files from a DSLR like the 5DmkIII is how much can be brought out of the shadows. I'm not seeing that in gr8pics edited photo examples. I see mud and noise. Perhaps my expectations are higher than what the camera delivers?

 

[gr8pics]

I see mud and noise. Perhaps my expectations are higher than what the camera delivers?

Add me to that list.

 

[Viridis]

So we're those shot separate, or RAW+jpeg?
I see the moving car is in the same place so wondering if this is a limited bug to raw+jpeg

Sent from my F8331 using PhantomPilots mobile app

 

[gr8pics]

So we're those shot separate, or RAW+jpeg?
I see the moving car is in the same place so wondering if this is a limited bug to raw+jpeg

Sent from my F8331 using PhantomPilots mobile app

Same exposure, one is raw file, the other jpg (untouched) I just retouched the raw to accompany the jpg, you can see the file name on the tabs on top of the pics.
Jpeg vers has a tad less noise and looks cleaner.

 

[gr8pics]

Just for fun,
Dji's own example files in Dng, untouched, can you see the big difference between the old camera and the new?
No? Thats cause there arent, even with 4 times larger sensor and supposedly better optics.

 

[danieljmoore]

I also noticed on these forums, that people of "high ranks" or thousands of postings are quick to judge new members as amateurs with no knowledge about anything, please stop that, theres allot of resourceful people out there that probably have ten times more knowledge than yourself, gained over years with experience, which also would love to share and teach, but if you start jumping at them, they will just stop, even if they could advice you with personal experience on that field.<br />Dont take it personally that a "rookie" sometimes knows better than you, or even worse, get offended by it, take advantage of it instead, to gain even more knowledge yourself, ask more instead of judging, and you will be amazed to see how many people smiles back at you if you smile at them <br /><br />Have a nice one everybody.<br />Peace out!

Geez, touchy and full of yourself to an extent... Maybe this is why people are treating you so. This message is indicating that you know better than the people that you came to for discussion on this forum. That's a recipe for trollish insults.

I've never felt that way from these guys and am still pretty new here. Admittedly, I do not know as much as some of the vets here and do not openly claim my intelligence to be superior, which helps me steer clear .

On subject, I hope you get this figured out. I see where your DNG does indeed look more saturated than your JPEG.

I too have the same result when I set the settings to "flat" so to speak. The JPEG receives that process making it look dull and bland and my raw generally comes out looking a little better because that "flat" setting is not baked in. These cameras may very well add saturation and so forth, which would be a bad deal to an extent. My raw images generally come out basically as a real representation on how my naked eye sees the shot, no processing at all.

This is coming from someone who may not know better though. I make money with my work from time to time but I am no professional. I've only tested stills really. I only do video work, so I've had no issues with processing the image in Log for my line of work as these cameras do not shoot in raw for video.

 

[ianwood]

I feel bad for the people who walk away from this thread thinking RAW is post-processed like a JPEG. You can call debayer, pixel binning, etc. processing if you like but that does not make a RAW image processed like a JPEG image. There's a difference between processing the image and a setting in the metadata.

I'm pulling perfectly good DNGs off the P4P. If you don't like what you see, move a slider or two! That's the whole point of RAW!

P.S. Anyone who says "I'm a professional photographer, therefore..." might just lose most of their credibility right there!

 

[Macfawlty]

In fairness to gr8pics, he was not the one who was throwing insults around. Clarifications, interpretations, opinions are perfectly fine. It was a legitimate point of discussion regardless of ones technical interpretation RAW and how the camera handles them. Between the insults, there's a lot of good info in this thread.

With regards to the DNG, if you can lift the shadows and recover highlights in LR, that's great. I'm not sure we've settled that question as yet. I'd love to see some more examples with different images.

 

[Andrei V]

Just for fun,
Dji's own example files in Dng, untouched, can you see the big difference between the old camera and the new?
No? Thats cause there arent, even with 4 times larger sensor and supposedly better optics.
View attachment 73729 View attachment 73730

Christer, the two images clearly have been processed differently (the 1/2.3" sensor has a much more contrasty gamma curve, for one thing) so I'm not really sure what you're trying to show.

Could you take a similar picture with the P4 as you can with the P4P at base iso in bright sunshine? Sure. In those kind of conditions, you're not going to see a lot of difference between the two sensors. Especially not when when showing screenshots of your photoshop interface at 2315 × 1365 pixels rather then the images rendered out at their full resolution. But even so, if you zoom in and look at the shadows, there is clearly more detail in the trees and the dark buildings in the P4P shot then in the P4 shot.

If you're really trying to make the claim that the P4P raw images retain a lesser dynamic range than the JPEGs, then you'll need to show us a scene where there are actually blown out highlights/crushed shadows in one and not in the other. The images you've shown so far don't show this because the scenes have a reasonably limited dynamic range, and the major portion of the image is exposed in the middle, with few highlight or shadow areas where we can look to to judge the level of retained detail.

Beyond that, why are you uploading screenshots rather then the actual rendered files? This introduces an additional element that muddles the experiment, since a screenshot of your photoshop interface at 2315 × 1365 pixels does not retain the actual quality of the original file, either in terms of resolution or dynamic range.

 

[Andrei V]

I feel bad for the people who walk away from this thread thinking RAW is post-processed like a JPEG. You can call debayer, pixel binning, etc. processing if you like but that does not make a RAW image processed like a JPEG image. There's a difference between processing the image and a setting in the metadata.

I'm pulling perfectly good DNGs off the P4P. If you don't like what you see, move a slider or two! That's the whole point of RAW!

P.S. Anyone who says "I'm a professional photographer, therefore..." might just lose most of their credibility right there!

Agreed. One should be weary of appeals to authority in general -- if someone knows their stuff, that is best demonstrated through their analysis and explanation, not by some title, especially a self-ascribed one. I am certain there are plenty of very good photographers out there who don't necessarily understand all the technology they are using to capture their images. While I feel that there is value to understanding the tools one works with, photography is both a science and an art. At the end of the day, composition, lighting, emotion, capturing the decisive moment -- that's what makes for compelling photography, not how much detail your images have under pixel-peeping.

 

[inkytog]

Agreed. One should be weary of appeals to authority in general -- if someone knows their stuff, that is best demonstrated through their analysis and explanation, not by some title, especially a self-ascribed one. I am certain there are plenty of very good photographers out there who don't necessarily understand all the technology they are using to capture their images. While I feel that there is value to understanding the tools one works with, photography is both a science and an art. At the end of the day, composition, lighting, emotion, capturing the decisive moment -- that's what makes for compelling photography, not how much detail your images have under pixel-peeping.

I'd like to think a pro photographer has a basic understanding of how to post process images shot in RAW.

Just as I'd like to think someone who fixes my car knows how to be a mechanic rather than just owning a tool bag and pulling random tools out and hitting he engine until it works because he's passionate about his job.

 

[RicardoUK]

Just for fun,
Dji's own example files in Dng, untouched, can you see the big difference between the old camera and the new?
No? Thats cause there arent, even with 4 times larger sensor and supposedly better optics.
View attachment 73729 View attachment 73730

Hi. I read many answers and I do get you. I am a professional photographer (and I´d like to think I am casue I do a good living out of that and have won some important prices, and I say this just to set that I do know about what I am talking about as you do) and I have to say I agree with you 100% I have the same problem. My RAW or DNG file comes out processed, exactly like the JPG should look like. I have chatted with DJI and of course, that´s a dead end, as many answers here that say that´s normal. I have shot with many cameras and drones and have never seen this problem (again, even if many says is normal and not a problem).

The only solution I can find and even more if you have sent the drone back and got another one, is just to overexpose your shots. Contrast and saturation might not be correct, but at least the starting point between the light and shadows will be "correct". Any thoughts?