2013. október 7., hétfő

Big JPEG Compression and Statistic experiment

A short while ago I had to think about how to make images smaller, without loosing (any more) information. For PNGs it is easy: I am a long time user of IrfanView, which includes the PNGOUT plugin to optimize compression of the image information.
Unfortunately two color images (used frequently in document scanning, because they use 1 bit per pixel, 1bpp) are usually stored in TIFF, Group 4 compressed images and in the past I have found these to be smaller than the same image saved in 1bpp PNG from IrfanView using PNGOUT. I have found no tools to poke around with this compression, also, IrfanView tends to save larger TIFF G4 images, than what come out of scanning.

This made me tudn my attention to Grayscale (8bpp) images,  which are often saves in JPEG, even though that is a lossy compression (meaning the decompressed image will not be equivalent to what came out of the scanner, just will look like that).
It is very easy to make a JPEG smaller by lowering its quality during compression, but it will just increase the difference from the original image thus decrease the similarity to that. We want to keep as much information of the original image as we can, so I was only looking into improving the compression while keeping the encoded information intact.

This made me do a large scale experiment on all my fifty two thousand JPEG images: I compressed them with all the different methods I could find, and recorded the results. In a series of posts I will review these result, because I have found several interesting angles to them:
  1. How to check the integrity of all your photos in an automated way: easily done with a following these instructions! Basically, on linux install jpeginfo, then, in the designated folder perform: (let me know if you need windows instructions!)

    find -iname "*.jpg" -print0 | xargs -0 jpeginfo -c | grep -e WARNING -e ERROR
    
  2. How much space can you save by re-compressing your photos, _without_ losing a single pixel of information (nor the EXIF data)?
  3. Is it really worth all the extra hassle to create ultra-progressive JPEGs?
  4. What is an arithmetic compressed JPEG, how much size it gains and which applications can read it?
  5. Which cameras / camera makes have the best and the worst JPEG compression engines?
  6. How have the megapixels evolved along the years?
  7. How do re-compression gains change compared to image size (megapixels)?
  8. How to automate all these, and which problems need to be solved for this (e.g. how to create progressive, arithmetic coded JPEGs)? actual scripts and binaries where needed!
  9. Shall I look at my 30GB+ MJPEG movies as well? :)
For starters, let's see what cameras have produced my 51719 JPEGs: (yes, several images were taken with mobile phones)

Apple 562
BlackBerry 135
Canon 5928
Casio 214
Fuji 17552
HP 112
Kodak 234
Minolta 244
Motorola 2
Nikon 9783
Nokia 1430
Olympus 13813
Panasonic 698
Pentax 5
Samsung 236
Sony 276
#N/A 495

And a sneak peak into the total possible filesize / storage gains, and for you to have a quick answer:
  1. Originals: 135.6GB
  2. Re-compressing only the Huffman coding: 130.21GB, 3.78% total gain
  3. Re-compressing the Huffman coding, and storing a progressive JPEG: 123.67GB, 8.79% total gain, 4,82% gain over just Huffman optimization!
  4. Re-compressing the Huffman coding, and optimizing progressive JPEG storage (a.k.a. ultra-progressive JPEG):122.53GB, 9.64% total gain, only 0.85% gain over progressive JPEG
  5. Re-compressing using arithmetic compression instead of Huffman: 116.88GB, 13.81% total gain, 4.17% gain over the best possible Huffman compression!
  6. Re-compressing using arithmetic compression and storing a progressive JPEG: 114.72GB, 15.4% total gain, 1,59% gain over non-progressive arithmetic JPEG
  7. Re-compressing using arithmetic compression and and optimizing progressive JPEG storage (a.k.a. ultra-progressive arithmetic JPEG): 113.94GB, 15.97% total gain, only 0.58% gain over progressive arithmetic JPEG
Breakdown for each camera brand in the next post!

Questions welcome, as always! :)

2013. január 13., vasárnap

Better use of Panasonic Viera remote with XBMC

I am using XBMC on a Raspberry Pi (raspbmc actually) with my Panasonic Viera plasma screen.
It really is fascinating to see that it can be controlled with the TV remote out-of-the-box, right after install. It uses a communication between the computer and the display, on the HDMI cable called CEC.
However I missed some functions while found a few buttons on the remote which did not do anything useful to me. So here is the result of my investigation on which buttons can be programmed and how I changed the defaults to a more sensible layout.

I found, that not all the button presses from the remote are actually reported over the HDMI cable to the computer: (I used the custom remote config printig messages described here)

number on the right (label of the button) - what it sends to XBMC
8 (OK) - select
9 (OPTION) - title
10 (no labels, just colors) - red, green, yellow, blue
11 (text) - teletext
12 (STTL) - subtitle
13 (1-9, 0) - one two, three, four, five, six, seven, eight, nine, zero
19 (X) - clear
20 (GUIDE) - guide
21 (just arrows) - up, down, left, right
22 (arrow turning from right to left) - back
26 (up and down arrows) - pageplus, pageminus

These buttons do not send any signals on CEC: 1, 2, 3, 4, 5, 6, 7, 14, 15, 16, 17, 18, 23, 24, 25 and unfortunatelly the whole group 27 of buttons, even though they seem best fit for media control. :(
And this is how I added all the usually needed: on notmal XBMC systems you modify this file:
/opt/xbmc-bcm/xbmc-bin/share/xbmc/system/keymaps/remote.xml
on the raspbmc I modified this (raspbmc can have multiple versions of XBMC at the same time and switch between them):
/home/pi/.xbmc-current/xbmc-bin/share/xbmc/system/keymaps/remote.xml
In there, inside global I added:



That makes the red button to stop playback, the green to play and yellow to pause. Blue will bring up the window to download subtitle for the item being played. (The subtitle add-on has to be installed and enabled for that.) I also changed zero to highlight an item in the file listings.
In the Home section I added:


For Guide button to show the System Info window and the OPTION button to go directly into Settings.
In the FullscreenVideo I have added:


This makes the program changing up and down buttons to adjust the subtitle delay directly.

Should you want to add more or other functions to the buttons or bring up other windows you can find the Action IDs here and the Window IDs here!

Happy hacking with the rest of the buttons, screens and functions! :)

Rendszeres olvasók