Jump to content

8-bit DNGs-how is this done?


Recommended Posts

Advertisement (gone after registration)

Still, there’s no alternative to getting the exposure exactly right, and the only way to make sure is to check the histogram.

 

Sorry Michael, but I'm afraid in this specific case with the M8, one can not completely rely on the histogram to ensure perfect exposure, since the RGB histogram is calculated from the JPEG capture and it is directly affected by the white balance results of that particular shot. I'd never be so sure that any of the R, G or B channels is clipping or not considering the M8's really erratic WB performance. That's absolutely one of those highest priority issues that needs to be addressed.

Link to post
Share on other sites

Sorry Michael, but I'm afraid in this specific case with the M8, one can not completely rely on the histogram to ensure perfect exposure, since the RGB histogram is calculated from the JPEG capture and it is directly affected by the white balance results of that particular shot. I'd never be so sure that any of the R, G or B channels is clipping or not considering the M8's really erratic WB performance. That's absolutely one of those highest priority issues that needs to be addressed.

 

I assume that the histogram is actually calculated from the JPEG thumbnail for the LCD, as you still get a histogram if you are taking in DNG only. As the thumbnail is a pretty small image, I would guess that WB notwithstanding, it will not be a terribly accurate histogram. Even if you have clipping indication 'on' it seems to me to be a fairly rough and ready guide. My previous Sony R1, could as an option, show zebra striping on the EVF preview, if you were at risk of clipping on any areas. This is another "wouldn't it be nice feature" if we could have that as a button push option on playing the image thumbnail on the LCD. Very quick and visual.

 

Wilson

Link to post
Share on other sites

I'm guessing that the Jpeg shown when shooting DNGs is the embedded one that Wilson mentions. I use Microsoft Expression (formally iView Media Pro) to catalogue the shots and that uses the embeded Jpeg. Quite often the Jpeg and the DNG file look quite different when opening the DNG in ACR with the default values. Most times of course they look very similar.

 

So while the Jpeg is a very good guide it's not really totally reliable. The same occured on the Canon 5D previews, quite often a shot would show clipping, but that clipping could be recovered in processing.

Link to post
Share on other sites

I assume that the histogram is actually calculated from the JPEG thumbnail for the LCD, as you still get a histogram if you are taking in DNG only. As the thumbnail is a pretty small image, I would guess that WB notwithstanding, it will not be a terribly accurate histogram. Even if you have clipping indication 'on' it seems to me to be a fairly rough and ready guide. My previous Sony R1, could as an option, show zebra striping on the EVF preview, if you were at risk of clipping on any areas. This is another "wouldn't it be nice feature" if we could have that as a button push option on playing the image thumbnail on the LCD. Very quick and visual.

 

Wilson

 

It is interesting to note that the histogram seems to be calculated from the embedded jpg initially, and after a short while is replaced by the calculated jpeg. The highlight markings "jump" to a larger value on the LCD after a short while.

Link to post
Share on other sites

Sorry Michael, but I'm afraid in this specific case with the M8, one can not completely rely on the histogram to ensure perfect exposure, since the RGB histogram is calculated from the JPEG capture and it is directly affected by the white balance results of that particular shot. I'd never be so sure that any of the R, G or B channels is clipping or not considering the M8's really erratic WB performance. That's absolutely one of those highest priority issues that needs to be addressed.

Yes, auto WB has its issues and the histogram is not completely accurate. Still, if the histogram shows no sign of clipping, in all likelihood there is no clipping. The converse is not necessarily true.

Link to post
Share on other sites

At the risk of sticking one's toe in a water with much bigger fish in it, here's a thought: as Sean has educated us on the value of the lower contrast lenses (bright day shooting), might one's attitude toward exposure be affected by what one is shooting? Shooting in bright light (southern Calif) is very different than northern light (possibly more cloudy, less contrast....). Not trying to open a big can of worms, but there is a difference.

 

The Leica rendering seems to have a control in the subtleties of its rendering, especially for the darker tones, where it just keeps finding information, delightfully so. Thus might the slight shift in the curve (and thus emphasis) to the lower light values, and theri restraint in pursuing the very high highlights, which are often blown out anyway, be purposefully driven to what they think needs more control?

 

Products are developed for a specific market, and the specialized ones have an approach, a taste, or a sensibility in them. Are we poking around in the deeper values of the Leica look?

 

I don't mean this to be a negative - rather, it is an appealing but slightly OT thought - perhaps, like film, the camera has a character. That's a good thing, yes?

 

Geoff G.

Link to post
Share on other sites

Advertisement (gone after registration)

Thinking well on this long issue... WITHOUT altering the basic spirit of the camera, I think that they could, someway, make available 2 or 3 different settings for the basic conversion from CCD output (14 to 8 bit lookup table..that is 2 or 3 "graphs" as we have seen, with slightly different "shapes") ... maybe this would mean to alter the "lack of excessive menu choices" that is one of the reason for I and other like M8; but it could have some sense... but I think difficult to implement in current electronics... requires memory, computing power... maybe for a M9... but in the meantime a 16 bit (or 14 or so) implementation could also be on the way...

Link to post
Share on other sites

........Your standard -1/3 EV correction setting ensures that the camera will tend to err on the side of underexposure. This is a useful strategy insofar as with sensors, there is a graceful degradation of image quality with underexposure whereas overexposure leads to clipping .........

 

Maybe we are starting to get somewhere with the original problem, which was to understand the reason why the M8 files seem to some of us to be less capable of being edited in the highlights than other digital files. The shadows are in this respect surprisingly good.

 

The question posed by several contributors was whether this highlight issue is the result of the “8-Bit” file structure and if it is then would a “16-Bit” file cause less of a problem.

 

Many opinions and explanations have been offered but I have to say that any argument presented one way or the other, based on questionable or clearly erroneous data, assumptions or methods immediately fails to convince me or have any credibility.

 

(I did learn something about photography working at Kodak for more than 40 years and I have a degree in Applied Physics which equips me quite well to deal with the technicalities, including the mathematics.)

 

Leica are in a difficult position which is entirely of their own making. They promote the M8 as having “16-bit resolution”, whatever that turns out to mean, but the data published in LFI and reproduced in this thread shows clearly that at no point in the dynamic range of the DNG output file does the resolution even approach “16-Bit”.

 

To deal with this the argument has been shifted to a demonstration that even if a “16-Bit” version of the data collected by the sensor were offered the resulting print would be indistinguishable from the one that can be obtained from the normal “8-Bit” file.

 

My response is two fold; firstly it is always possible to select a subject that supports this type of argument one needs many examples before a conclusion can be reached on such matters and secondly, which is more important, this is not actually the point being questioned. It is the ability to edit and modify the file’s data prior to printing that is at issue.

 

On this I suggest Michael’s response is the first clue – it is a trade off between dynamic range and editing ability which is being made. Leica have, it seems, maximised the dynamic range as the default position and accepted the limitations on highlight modification and recovery – which begs the original question – would a data file with greater resolution, be it “16-Bit” or some other format, minimise or eliminate the consequences of this trade off?

Link to post
Share on other sites

AGREE, this is another way to expose the same conclusion I made in my above post...they have followed the rule of "expose for the shadow, develop for the highlights" :) ... old simple rules DO have a value...:)

 

Hi Luigi,

 

That rule is valid for developing B&W negative film because density increases with development. Development can't add shadow detail but it can restrain density in the highlight areas of the negative, thereby making the negative more printable.

 

With digital, once highlight information is gone, its gone. One can try to extract more or less information from the RAW file but one can extract no more than is actually there. There's no digitally analogous way to hold back the highlights in developments.

 

So, the processes have important differences in this respect. If one wants to retain highlight detail with digital capture, he or she must do so via exposure. One question being explored in this thread, of course, is what effect Leica's 8-bit compression has on highlight transitions.

 

Cheers,

 

Sean

Link to post
Share on other sites

At the risk of sticking one's toe in a water with much bigger fish in it, here's a thought: as Sean has educated us on the value of the lower contrast lenses (bright day shooting), might one's attitude toward exposure be affected by what one is shooting? Shooting in bright light (southern Calif) is very different than northern light (possibly more cloudy, less contrast....). Not trying to open a big can of worms, but there is a difference.

 

Geoff G.

 

Certainly, that's quite true.

 

Cheers,

 

Sean

Link to post
Share on other sites

Yes, auto WB has its issues and the histogram is not completely accurate. Still, if the histogram shows no sign of clipping, in all likelihood there is no clipping. The converse is not necessarily true.

 

Hi Michael,

 

That's sometimes the case but, in fact, what I tend to see in some high contrast shooting (having used the M8 for a little over a year for weddings, among other things) is that the M8 histogram can indicate that highlights are in range when in fact they're out just a bit. Of course, its easy enough, once one knows that, to mentally adjust.

 

I also agree with your previous comments about metering. Having often worked with cameras that had no meters, I do often wonder about the complaints I sometimes hear about the M8's metering. No camera can read the photographer's mind (despite marketing hype to the contrary) and one always has to consider the exposure a camera suggests in relation to what kind of "digital negative" one wants to make. I don't let any camera set exposure for me, broadly speaking, even in AE mode, etc. The M8's meter is quite good in that it reliably tells me what I would expect to know from that kind of meter. It provides fairly reliable data but I never would allow it to be the final arbiter of exposure.

 

Cheers,

 

Sean

Link to post
Share on other sites

All this discussion is very interesting, but what I still do not understand well is the methods for highlight recovery. I am using Lightroom, or PSCS3 or C1 and all these programs allow for highlight recovery which appears at least to my non-expert eye to recover highlights in most cases. The problem photos are those where these programs do not manage a full recovery, then I get a splash of white, e.g. in clouds with sunshine behind etc. and that photo is history. So what the M8 is showing me as blown highlights is a bit to much and it appears to me that the blown highlights are not really all blown, some of them can be recovered using the above mentioned developers. I of course understand all this about correct exposure, exposing to the right and all that, but this does not help me very much in certain situations.

 

But look at the attached photos. The first one is what Camera Raw shows me before doing anything, almost all the sky is blown out and btw that is the same thing the camera showed. After moving the recovery slider up to about 25 all the red disappears and I get a sky, and clouds back. So my question is do I loose anything by doing this, am I just manipulating the highlights or also something else (I cannot see it). Of course this is not enough in some cases as I mentioned, then I take the slider all the way to 100 and still have a white blob somewhere that is un recoverable. So it seems to me that there is a lot of latitude in the DNG files, and considerably more than I have experienced before with 12 bit uncompressed NEF files from a D2X. Is this a correct impression and if so how come the 8 bit files from the M8 are better than 12 bit NEF files? And finally arent we sometimes fooled by the color on our screens (which are 8 bit) and on our camera screens? If this is correct then why is the camera not showing a correct representation of what is really happening?

 

You see I am getting a little confused here, so any and all simplifications and explanations will be very welcome.

 

JKB

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

Link to post
Share on other sites

Sorry Michael, but I'm afraid in this specific case with the M8, one can not completely rely on the histogram to ensure perfect exposure, since the RGB histogram is calculated from the JPEG capture and it is directly affected by the white balance results of that particular shot. I'd never be so sure that any of the R, G or B channels is clipping or not considering the M8's really erratic WB performance. That's absolutely one of those highest priority issues that needs to be addressed.

 

This is the same with any digital camera though. There is no histogram available that calculates exposure from the RAW data.

 

(so one can never rely on the histogram for perfect exposure. However, the RED channel should not clipped if the histo says you're ok, and I'm assuming you either know where the wb is going to fall or have set it)

Link to post
Share on other sites

All this discussion is very interesting, but what I still do not understand well is the methods for highlight recovery.

JKB

 

Many people get confused about this. All that a highlight or shadow recovery action can do is to recover information that is present in the RAW file. What many of us here are discussing relates to desired highlight information that is not actually in the RAW file. Recovery can only recover something that's there in the first place, of course.

 

Cheers,

Link to post
Share on other sites

The clipped highlights indicator usually assumes that everything above a certain value is blown, not just those that actually _are_ blown.

 

Taking an 8 bit file as an example (because I can remember the numbers :-), technically for a pixel to be blown it will have to have a value of 255 in each channel. What the clipped indicator may do is show everything above say 250 as being clipped. This doesn't mean that the pixels are necessarily blown, just that they are very close to being blown and may appear so to the naked eye without some post processing.

 

One of the advantages of shooting RAW is that it's usually very easy to recover information from these pixels.

Link to post
Share on other sites

Ok I get it, the histogram is made from a jpeg/tumbnail, but I think that is incorrect, it should show a representation of what is really captured. The problem is that when I see some clipping on my camera screen I have no way of knowing whether this is recoverable or not. Based on my above observation I have accepted a little bit of red in the review, but clearly this is not something I can rely on and I have no way of knowing whether the intensity of the light there will be above or below the threshold that is there. Well, I suppose experience will show better what the limits are, but my current impression is that this is more of a problem in the M8 than in my former D2x. But no matter, problems are for solving.

Link to post
Share on other sites

{snipped}After moving the recovery slider up to about 25 all the red disappears and I get a sky, and clouds back. So my question is do I loose anything by doing this, am I just manipulating the highlights or also something else (I cannot see it). Of course this is not enough in some cases as I mentioned, then I take the slider all the way to 100 and still have a white blob somewhere that is un recoverable.

 

Practically speaking, RAW files (and the M8 DNGs included) have more exposure latitude than JPEGs. Because you are working with more data (it's uncompressed by the time you're seeing it in the RAW converter) you can usefully push and pull the files around a lot.

 

Now, the real question no-one is asking here is how is the recovery accomplished? That will tell you what you're losing.

 

So what you lose depends on how you "recover highlights" that aren't actually blown; they're just out of range of your current raw interpretation. (Of course, if you've made a JPEG of that interpretation, then yes, they're now fixed and blown, which is why a lot of us prefer to shoot RAW).

 

You will always lose something in recovering highlights.

 

So in C1, pulling the exposure level back you will lose shadow details. If you change the "film curve" to linear, you will lose contrast or snap. If you change the contrast slider, you will lose, of course, contrast.

 

All of these things, by the way, don't change the relative midtone values, which is why I like them :) But they do change the way the lower and upper midtones move to shadows and highlights.

 

In ACR, they are doing some pretty complicated things with the channels in the colour space you're working in at the time. It works pretty well, but, like the "fill light slider" you can get gross artifacts and upper midtone changes with a heavy hand on the slider.

 

It's true that Canon's RAW files seem to take well to whatever they're doing underneath the covers. Oddly enough, many a blown shot in Canon's own DPP converter is completley ok in ACR.

 

So, sticking with C1, because I like its colour and workflow best, I actually see quite a lot of highlight recovery in the M8 files in C1--at least as good as the Canons, and better in the shadows.

 

So it seems to me that there is a lot of latitude in the DNG files, and considerably more than I have experienced before with 12 bit uncompressed NEF files from a D2X. Is this a correct impression and if so how come the 8 bit files from the M8 are better than 12 bit NEF files? And finally arent we sometimes fooled by the color on our screens (which are 8 bit) and on our camera screens? If this is correct then why is the camera not showing a correct representation of what is really happening?

 

Yes, DNGs have a lot of latitude. With the same RAW converter (c1), I've found the M8 files have at least as much recoverability as the 5d under the same exposure conditions, and more than my 1ds2--which I never expected, but it's the truth.

 

Now, folks, note I'm not talking about ACR/Lightroom, which has long been the highlight recovery champ. But even there the M8 kills the NEF stuff I've worked with, and as I mentioned to Sean, the only thing I usually see there is a bit more red sensitivity which can clip easily if not handled correctly.

 

You see I am getting a little confused here, so any and all simplifications and explanations will be very welcome.

 

JKB

Link to post
Share on other sites

Guest GuyMancusoPhoto
Hi Michael,

 

That's sometimes the case but, in fact, what I tend to see in some high contrast shooting (having used the M8 for a little over a year for weddings, among other things) is that the M8 histogram can indicate that highlights are in range when in fact they're out just a bit. Of course, its easy enough, once one knows that, to mentally adjust.

 

I also agree with your previous comments about metering. Having often worked with cameras that had no meters, I do often wonder about the complaints I sometimes hear about the M8's metering. No camera can read the photographer's mind (despite marketing hype to the contrary) and one always has to consider the exposure a camera suggests in relation to what kind of "digital negative" one wants to make. I don't let any camera set exposure for me, broadly speaking, even in AE mode, etc. The M8's meter is quite good in that it reliably tells me what I would expect to know from that kind of meter. It provides fairly reliable data but I never would allow it to be the final arbiter of exposure.

 

Cheers,

 

Sean

 

People fail to realize the meter is set for grey and only grey and anything going on be it black or white tones will average for grey. All camera meters and even handheld are set for grey. It's up to one either the camera average it and make a great guess at it or us knuckleheads to set it they way we want. There is no magic here and same for WB. It's all guesing on what is in the sensor path. Now most camera's do a great job at guessing what is going on, but certainly not foolproof. Also the meter on the M8 is heavyly center weighted. You can point up and boom underexpose the midtones and shadows or point down and overexpose the sky in landscape shots for example. Need to find the correct balance for the scene and only the operator can dtermine if it is correct. A mode does a great job but it can be fooled at any time.

Link to post
Share on other sites

{snipped} Based on my above observation I have accepted a little bit of red in the review, but clearly this is not something I can rely on and I have no way of knowing whether the intensity of the light there will be above or below the threshold that is there. Well, I suppose experience will show better what the limits are, but my current impression is that this is more of a problem in the M8 than in my former D2x. But no matter, problems are for solving.

 

Julius, no, you're thinking about this in a less than useful way...:)

 

The M8's metering is more agressive than the Nikons or the Canons (especially). You will get different exposure readings for the same light source, and depending on how you have the Nikon or Canon's meter set, you may actually be drawing on a "database" of exposure values where those cameras try to guess what you're shooting :)

 

The M8 doesn't work that way. It's a purely center-weighted spot-ish kind of thing (IIRC), and it's usually much closer to the mark.

 

So when I meter a scene with the M8, I know it's trying to make the thing in the middle a gray. Depending on what I want detail in, I'll either expose for the highlights (meaning I'll push them towards grey and stop down) or the shadows (meaning I'll push them towards grey and open up).

 

BTW--this is why I still say people should go buy a good, inexpensive, external incident meter (which is set up for overall exposure and NOT for grey reflectance) till they're comfortable with overall exposure. Reflected light--even with a great metering system--isn't exactly the most intuitive way to make pictures ;)

Link to post
Share on other sites

Ok I get it, I am comparing apples and oranges here, the Nikon supposedly intelligent matrix metering with it´s database and the simple centered Leica method. Very good advice here, of course when I think about it, the camera is trying to expose correctly for 18% gray. I have to try to remember this. But thanks for the reminder, never get enough of those.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...