Using Slimraw to manage dng files

[scaryink]

Advertisement (gone after registration)

Have anyone tried this product to manage the size of your data files?  I have demoed the app and it appears to work very well in reducing raw file sizes without compression.  The files open very quickly in photoshop and all metadata is retained.  I really can't see any downside.  I was looking at the app primarily for my cinema dng files but realized it works just as well on still imagery. 

 

Please note I have no connection with the company and am only a potential customer.

[jaapv]

Reducing size without compression? How would that work? As most of us use lossless compressed DNG it is hard to imagine that further size reduction can be anything but lossy.

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[cpc]

This is an unexpected place to see slimRAW mentioned.   (It is developed by me.)

scaryink, you are correct that it can be used for stills (see down here in the user guide, a short paragraph on stills: http://www.slimraw.com/userguide.html ), but since this is the Monochrom section, I am not sure the Monochrom is actually supported. I need to check this. I believe I've tested the M240, and it is fine.

 

jaapv, the specification allows wiggle room, you can squeeze more by doing lossless recompression cause in-camera compression is never optimal. Whether recompression gains are worth bothering is another story.

 

The original motivation for stills support in slimRAW was getting DNG stills and timelapses into Premiere Pro cause Premiere is finicky about the kind of DNGs it accepts, but it has also been used for some disk space savings by Pentax users in particular.

[pop]

..... appears to work very well in reducing raw file sizes without compression. ....

That's a bit misleading as the blurb and all descriptions I've seen so far clearly state that the software does indeed compress the files. However, it does so without losing any information.

[jaapv]

This is an unexpected place to see slimRAW mentioned.   (It is developed by me.)

scaryink, you are correct that it can be used for stills (see down here in the user guide, a short paragraph on stills: http://www.slimraw.com/userguide.html ), but since this is the Monochrom section, I am not sure the Monochrom is actually supported. I need to check this. I believe I've tested the M240, and it is fine.

 

jaapv, the specification allows wiggle room, you can squeeze more by doing lossless recompression cause in-camera compression is never optimal. Whether recompression gains are worth bothering is another story.

 

The original motivation for stills support in slimRAW was getting DNG stills and timelapses into Premiere Pro cause Premiere is finicky about the kind of DNGs it accepts, but it has also been used for some disk space savings by Pentax users in particular.

Ok - granted there might be some gain over in-camera  compressed DNG - how much would that  approximately  be ? Wiggle room does not sound like something worth going the extra mile for.

[cpc]

Ok - granted there might be some gain over in-camera  compressed DNG - how much would that  approximately  be ? Wiggle room does not sound like something worth going the extra mile for.

Around 1MB per image in a quick M240 test I just did. This depends on image specifics though since all lossless compression is variable bit rate by nature.

 

Also, nope -- no Monochrom support in the current version. Guess I'll add this in the next update for completeness' sake.

[scaryink]

Advertisement (gone after registration)

The app appears to make significant reductions in file size for lossless compression on cinema dng files - but at 1mb per still image for the monochrom, I don't think it would be worth it as a stand alone solution. 

[cpc]

The app appears to make significant reductions in file size for lossless compression on cinema dng files - but at 1mb per still image for the monochrom, I don't think it would be worth it as a stand alone solution. 

 

Yeah, it mostly depends on whether the input is already compressed. If you feed slimRAW uncompressed images, you will see big size reduction (typically 2:1 or better). But if you feed it compressed images, there is only so much that can be squeezed with lossless recompression. This is relative too -- for example, with the Pentax 645z, which makes huge and not particularly well compressed lossless files in the first place, you can further save some ~10MB by recompression.

[sandymc]

The compression that DNGs use is simple Huffman arithmetic compression - it will compress an image file to maybe 70% of original size. More advanced compression methods will compress to say 50%, depending on the image. E.g., on the new Fuji X-Pro2, a compressed file is usually about 25MB vs 50MB for an uncompressed file.

[cpc]

The compression that DNGs use is simple Huffman arithmetic compression - it will compress an image file to maybe 70% of original size. More advanced compression methods will compress to say 50%, depending on the image. E.g., on the new Fuji X-Pro2, a compressed file is usually about 25MB vs 50MB for an uncompressed file.

Not that simple. If you do plain Huffman you will get almost no compression with image data of this kind. Huffman works on top of an entropy reducing lossless step tailored towards bayer data. Typical compression is around 2:1 (around 50%), and depending on image complexity this can go up or down. In-camera compression is usually suboptimal (way suboptimal, bordering lame, in some cases).

Edited June 22, 2016 by cpc

[sandymc]

Not that simple. If you do plain Huffman you will get almost no compression with image data of this kind. Huffman works on top of an entropy reducing lossless step tailored towards bayer data. Typical compression is around 2:1 (around 50%), and depending on image complexity this can go up or down. In-camera compression is usually suboptimal (way suboptimal, bordering lame, in some cases).

 

Not really. Lossless JPEG (not JPEG-LS, which is a different specification) is decidedly Huffman based, coupled to a simple nearest-neighbor predictor: https://en.wikipedia.org/wiki/Lossless_JPEG

 

It doesn't know anything about Bayer data; it was designed for RGB data. But it's simple minded enough that Bayer or not doesn't matter much.

[cpc]

Not really. Lossless JPEG (not JPEG-LS, which is a different specification) is decidedly Huffman based, coupled to a simple nearest-neighbor predictor: https://en.wikipedia.org/wiki/Lossless_JPEG

 

It doesn't know anything about Bayer data; it was designed for RGB data. But it's simple minded enough that Bayer or not doesn't matter much.

 

So how does this simplicity fits in the huge differences you'll get by different implementations? 

[sandymc]

So how does this simplicity fits in the huge differences you'll get by different implementations? 

 

 

I haven't seen significant differences in size between lossless DNG's created by different programs from the same original. Can you post some examples?

[cpc]

I haven't seen significant differences in size between lossless DNG's created by different programs from the same original. Can you post some examples?

 

Sure, here are a couple of sets: https://drive.google.com/file/d/0B994SddHxZXfU0xRWU5RSlVKdHM/view?usp=sharing

Uncompressed images, then 3 different compressors for each. As you can see the ratio of the the best size divided by the worst size is significant, and it can go larger or smaller, depending ot image characteristics.