New Workshop - Unscripted!

News & Insight

Fascinating Look At What’s Really Going On Inside A JPEG | Color, Compression, Uses

By Kishore Sawh on May 24th 2015

JPEG-BMP-file-formats-RAW-compression-sotrage-photography-slrlounge-2

In the ever relentless race of ever advancing camera tech, a camera crowned king today is next month’s bargain deal. From the moment a top model is revealed, it tries to stay relevant while other upcoming announced cameras bite at its heel. Generally, time at the top is short, as the coup d’état comes quickly. The same cannot be said, however, for file formats.

JPEG has been under siege from competitor file types looking to usurp it, but it hasn’t happened in the near 25 years since its inception. It remains the ubiquitous file format in photography and general digital media, even as lossy as the typical 8-bit compression tends to be. In some manner of speaking, it’s likely that it has remained on top due to the vastness of its adoption. Like trying to turn the US to metric, trying to change from JPEG is daunting even though we know there are better options out there.

JPEG-BMP-file-formats-RAW-compression-sotrage-photography-slrlounge-1

But how much do you really understand about the format? Do you understand how it works, and what’s going on inside its algorithms? Probably not, if I may be so bold to say, and also because you just can’t be bothered. However, if you are the type who sees the world in matrices of ones and zeroes, this information would be of some interest for you, and to some degree, may even help you decide when you should and shouldn’t use it.

The YouTube channel Computerphile has released a few videos all dedicated to the JPEG, and really dive into the science and maths of how they work. In the process, there is discussion on the concept and applications of color spaces and a good look into the JPEG compression – something called Discrete Cosine Transform (DCT) which is essentially the mathematical technique deals with how information/pictures are turned into specific waves in order to save space.

JPEG-BMP-file-formats-RAW-compression-sotrage-photography-slrlounge-5

One thing is also made very clear, that other files like TFF and BMP, which are lossless type files, work almost like zip files, where they are compressed in such a fashion where once expanded they are the original files. JPEG, on the other hand, you aren’t guaranteed the same image on the output as received on the input, but close enough for most people. Good enough for you? You watch and make the call. It’s utterly fascinating.

[REWIND: JPEG Formats: Do You Know Why You Choose The Ones You Do?]

Source: PetaPixel

About

Kishore is, among other things, the Editor-In-Chief at SLR Lounge. A photographer and writer based in Miami, he can often be found at dog parks, and airports in London and Toronto. He is also a tremendous fan of flossing and the happiest guy around when the company’s good.

4 Comments

Please or register to post a comment.

  1. Dave Haynie

    JPEG is pretty conceptually simple, though the details are kind of complex.

    First of all is color decimation, which is familiar to anyone working in video. JPEG and most video formats aren’t stored in RGB, but rather, a more natural-to-imaging format dubbed YCrCb, where “Y” is a chrominance value and Cr and Cb are both color samples based on red and blue, respectively. So your RGB image is transformed to YCrCb, which is a reversible color-space transform, like of like going to Lab color in Photoshop.

    Next is the first lossy stage: decmination. Imaging researchers and cognitive psychologists have long known that we’re far more senstitive to luminance than color. So one of the first thing done in JPEG (or MPEG) is tossing out of some color samples. An undecimated image or video stream is called a 4:4:4 stream, meaning a full complement of luminance and chroma samples. We can toss out half of the color samples and get a 4:2:2 result, meaning that every other color sample is tossed out along the horizontal. Or, toss out half again, which gets you 4:2:0 (samples tossed out horizontally and vertically) or 4:1:1 (one color sample for every four pixels on the horizontal). NTSC DV video uses 4:1:1 decimation, DVD, Blu-ray, ATSC, and DVB television use 4:2:0 decimation. The level of decimation for a JPEG is entirely up the algorithm writer.

    Next comes the famous DCT (discrete cosine transform), which is actually a reversible transform. It’s related to the Fourier transform, and essentially, it blocks out the whole image into cells, usually 8×8 pixels. Then it applied the DCT to each cell, which translates spatial data into frequency data. Again, this is a reversible transform, nothing it lost. Yet.

    Next comes filtering. Since we now have little 8×8 blocks full of frequency information, you’ll find that blocks with very little happening in them tend to be full of lots of zeros. Depending on the settings, your JPEG algorithm will go through every cell and toss out some of the highest frequency information… in signal processing this is a low-pass filter, in photography, this is a blur function. Once this filter has run, the remaining data is Hufmann encoded — that’s a lossless compression used in file compression before being replaced by ZIP. It’s that previous step, and the decimation, that’s the lossy part of JPEG.

    So when brought back, the JPEG decoder de-compresss, runs the inverse DCT transform, and interpolates the decimated YPrPb data to deliver the resulting uncompressed, but changed, RGB output. Variations of this are used for I-Frames in MPEG and related (AVC, HEVC) algorithms.

    | |
  2. Thomas Horton

    My brain is cramping. :)

    | |
  3. Ralph Hightower

    Very interesting. I haven’t used calculus since I was in college; yes, I program computers, but calculus is not needed for the applications that I write.
    He did provide one veiled reference that opening and closing a JPEG won’t degrade the picture. I try to dispel that myth every time I hear it. If one saves a JPEG, then yes, losses will occur. But just opening and closing a JPEG file should no degrade the image unless that was a poorly written application that saves the file even if there were no modifications done.

    | |
  4. Graham Curran

    Interesting info. Thanks.

    | |