File Formats, Pixel Formats

2006-04-04T19:28:00.000-06:00

I thought it would be a good idea to kick off this blog with a lesson on the different formats an image can take. There are two distinct things to keep in mind here. The first is the file format that the image is saved in. GDI+ comes with several handy image encoders and decoders.

Built-in GDI+ Image Encoders:

Codec Name: Built-in BMP Codec
File Extension(s): *.BMP;*.DIB;*.RLE
Mime Type: image/bmp

Codec Name: Built-in JPEG Codec
File Extension(s): *.JPG;*.JPEG;*.JPE;*.JFIF
Mime Type: image/jpeg

Codec Name: Built-in GIF Codec
File Extension(s): *.GIF
Mime Type: image/gif

Codec Name: Built-in TIFF Codec
File Extension(s): *.TIF;*.TIFF
Mime Type: image/tiff

Codec Name: Built-in PNG Codec
File Extension(s): *.PNG
Mime Type: image/png

Built-in GDI+ Image Decoders:

Codec Name: Built-in EMF Codec
File Extension(s): *.EMF
Mime Type: image/x-emf

Codec Name: Built-in WMF Codec
File Extension(s): *.WMF
Mime Type: image/x-wmf

Codec Name: Built-in ICO Codec
File Extension(s): *.ICO
Mime Type: image/x-icon

The framework has Decoders for every Encoder in addition to the ones listed above.

As you can see, we can open BMP, JPG, GIF, TIF, PNG, EMF, WMF, and ICO file formats. What you need to realize is that all formats are converted to a Bitmap like format once opened. This means that while residing in memory, there are no compression algorithms in play, and everything is a raster image (EMF and WMF are converted from their metafile formats to the Bitmap's raster format when they are opened based on the Width and Height of the Bitmap object). A Bitmap is used as the memory representation of the image because it can display every color that your monitor can (we'll get into Color Depth and Pixel Format later), it supports transparency (the Alpha channel), and it can hold multiple frames (although you cannot work with them due to lack of support from GDI+, you are allowed to at least view what is already there). By saying that they are a Bitmap while in memory is not to say that they magically gain image quality (compression techniques of GIF, JPG, and TIF can reduce quality/amount of colors). The image keeps its exact same appearance, only now it takes up the amount of space required by a Bitmap (which equates to more space if you’re coming from a compressed format).

This conversion is a very good thing for us individuals who need to edit or otherwise manipulate pixels within an image. Image if you had to keep in mind the details of how a particular pixel you want to change was compressed and if your change will affect other pixels around it, if it will render the compression useless, if the new color is in the color palette, etc. Display is also much more efficient because graphics cards display images as is (Bitmap style).

Once an image is ready to be saved, file format comes back into play. Seeing as how EMF and WMF are metafile formats, we can't really save to them since all we have is a raster image (note that there are no EMF or WMF encoders. Metafiles are handled by the Metafile class and are outside the scope of this blog besides what is mentioned about opening them as a raster image). The raw, uncompressed pixel data is sent to the file type's respective encoder; the encoder does its magic and saves the image in that particular format. Some encoders accept optional parameters to control the encoding process. The JPG encoder accepts a Quality parameter (0-100, represents a percent of quality) and the TIF encoder accepts a parameter that determines which TIF Compression algorithm is used (None, CCITT3, CCITT4, LZW, or RLE). There is also the ColorDepth parameter which specifies bits per pixel (PixelFormat). This applies to encoders that have a non-compressing mode (ColorDepth doesn't make sense when talking about an image with a ColorPalette or that is compressed). GDI+ supports these parameters by providing a nice overload of Bitmap.Save(FileName, ImageCodecInfo, EncoderParameters).

Now that we have covered the transfer to and from a storage medium, let's talk about the image’s format while it's actually open in memory. This brings us to pixel format. An image is made up of rows and columns of pixels. One pixel can consist of three or more color channels. There is a Red, Green, and Blue channel in GDI+'s model. An Alpha channel (how "see-through" the pixel is) may also be present in some formats. An image's pixel format simply describes how many bits make up each channel. The different formats that are supported by GDI+ can be found in the PixelFormat enumeration.

Alpha

Canonical

Gdi

DontCare

Undefined

Extended

PAlpha

Indexed

Format1bppIndexed

Format4bppIndexed

Format8bppIndexed

Format16bppGrayScale

Format16bppRgb555

Format16bppRgb565

Format16bppArgb1555

Format24bppRgb

Format32bppRgb

Format32bppArgb

Format32bppPArgb

Format48bppRgb

Format64bppArgb

Format64bppPArgb

Let’s try to make sense of this badly formed enumeration shall we? Only pay attention to formats beginning with the prefix “Format”, all the others mean basically nothing. The number preceding ‘bpp’ refers to how many bits are taken up by one pixel. ‘A’ stands for “Alpha”, ‘PA’ stands for “Pre-multiplied Alpha”, ‘R’/‘r’ stands for “Red”, ‘g’ stands for “Green”, and ‘b’ stands for “Blue”, these stand for the color channels of the pixel. If there is a string of numbers after the channel combinations, they should be read so that they correspond with the color channel (Argb1555 means ‘1’ bit for ‘A’, ‘5’ bits for ‘r’, ‘5’ bits for ‘g’, and ‘5’ bits for ‘b’). If these numbers don’t add up to the bpp, it means the remaining bit(s) aren’t used. The ‘Indexed’ suffix means that a ColorPalette is used. The ‘GrayScale’ suffix that there is only one color channel and it should be interpreted as a shade of gray. It’s also funny to note that anything above 32bpp doesn’t work correctly. That’s right, they don’t work (which is fine seeing that today’s monitors can only display 8 bits per color channel anyway).

Another important point, you can’t draw on (create a Graphics object from) any Bitmap that has an Indexed, or GrayScale suffix, or 16bppArgb1555. Although you can’t draw on them, you can still open them up and stick them in a Bitmap object (remember what I said earlier about the difficulties of changing a compressed Bitmap or one with a ColorPalette, GrayScale is a different story, I can’t think of a reason why it’s not supported). Knowing this, we can now break down the enumeration into more intelligent pieces.

Formats you can create a Bitmap with AND create a Graphics from the resulting Bitmap object AND correctly draw on with:

Format16bppRgb555

Format16bppRgb565

Format24bppRgb

Format32bppRgb

Format32bppArgb

Format32bppPArgb

Formats you can create a Bitmap with AND create a Graphics from the resulting Bitmap object BUT can’t correctly draw on with:

Format48bppRgb

Format64bppArgb

Format64bppPArgb

Formats you can create a Bitmap with BUT can’t create a Graphics from the resulting Bitmap object:

Format1bppIndexed

Format4bppIndexed

Format8bppIndexed

Format16bppGrayScale

Format16bppArgb1555

There is one final thing to mention about drawing on specific formats, those formats being 16bppRgb555 and 16bppRgb565. Colors in GDI+ are represented with 32 bits (Argb), 24 bits without Alpha (Rgb). 16bpp and 15bpp are clearly less than 24bpp. So how do non-supported colors get drawn on these two formats? Well the color is dithered. You can read up on dithering and how it relates to images here. (just note that when that article talks about a color palette, it means every displayable color, it’s not used in the same sense as in this blog (we use it to mean a limited set of colors that bear no relation to the actual values in memory, they simply map to them)). Basically, it creates the illusion of a color that cannot be displayed by interspersing two valid colors in a way that allows the human eye to interpret it as the un-displayable color (obviously you will need a group of the pixels to perform this operation, it can’t be done on single, invalid pixel values).

You can always convert up in formats and not lose data. When converting down you will not always lose data though. In fact, when going from 32bppRgb to 24bppRgb you will lose nothing at all since the extra byte in 32bppRgb is unused. When converting from a format that supports Alpha to a format that doesn’t support it, you will lose all Alpha data. 16bpp555 and 16bpp565 are unique, converting from one to the other will always lose color data (or have the potential to lose color data, they do share approx. 125 common colors). To understand why, 11111-11111-11111 in 16bpp555 is not the same color as 11111-011111-11111 in 16bpp565. True, they are the same numbers, but they do represent different colors (probably due to the fact that Microsoft wanted 16bpp565 to be able to display max red, max green, and max blue but there are only 5 bits for red and blue but 6 bits for green). Try to picture it this way, the channel’s max will always be the true max for that color, and the channel’s min (0) will always be the complete absence of that color. All other values can be computed by interpolating (think gradient) between all “true” colors and only picking enough colors to fill up all available space in the specific format (you would want these colors to be equally spaced too, wont you?). So if you convert back and forth between 16bpp555 and 16bpp565, you will slowly make it down to only about 125 colors (this does not happen immediately because of dithering).

Bitmaps are laid out in memory just about as you would expect. To get at the memory of a Bitmap you will need to use the Bitmap.LockBits method to return a BitmapData object. The BitmapData.Scan0 property points to the first channel of the first pixel. In memory, the channel order is Bgr[A] A always comes last if it is supported by the format. Don’t ask me why it’s backwards, I couldn’t explain it. This continues until you reach the end of a row of pixels. Rows are aligned at 4 byte boundaries; this means that unused bits are inserted at the end of each row if this alignment does not occur naturally. That means that you can not find the address of the first channel in the second row of pixels by the formula StartAddress + Bitmap.Width * BytesPerPixel. You may be off by one or more bytes. Luckily, the BitmapData.Stride property reports the real length of a single row of pixels memory wise. Formats that do not encode channel in “byte lengths” (such as 16bpp555, 16bpp565, etc.) are harder to access. You will need to use bit shifting operations to get at the value since the smallest memory value you can read at a time is one byte (eight bits). In memory, the Bitmap may me laid top-to-bottom or bottom-to-top. In practice, I have never come across an image that was actually in the bottom-to-top configuration but just know that it’s a possibility.

One final note, the Bitmap.GetPixel and Bitmap.SetPixel methods do all the memory magic behind the scenes for you and work correctly for every image format. There is no problem with images that use a color palette since while in memory, it doesn’t use the palette and can display all of the colors that the non-indexed format can (even though the format is still 8bppIndexed for example). The bit shifts necessary for getting and setting pixels in the 16bpp555 and 16bpp565 formats is done correctly.

And that’s it! Hopefully this post helped clarify any doubts you had in your mind about how GDI+ represents images. It’s a lot of information to keep in mind but most of it is intuitive, and all of it is helpful when dealing with the internals of an image.

As a bonus, here is what happens when you try and work with some of the unsupported formats:

Original, After Saving as 48bppRgb Bitmap, After Recovery Attempt

After trying to draw a Black line with Width 1 on a 64bppArgb Bitmap

Caution, bmp in Road

File Formats, Pixel Formats