j2kaudio
Overview
j2kaudio is a library designed to leverage the JPEG 2000 2-dimensional wavelet image compression standard to compress audio data. There are several distinct advantages to this approach over many existing lossless and lossy compression/encoding methods:
- the original waveform is more accurately recreated for lossy compression
- very high lossy compression ratios are possible
- 0 < encoding rate < 1 specifies lossy compression
- encoding rate == 1 specifies lossless compression
- j2kaudio data could be interleaved in MJPEG2000 video for a 1-codec A/V solution
- 8-bit integer, 16-bit integer, and 32-bit floating point audio samples are supported
j2kaudio uses the JasPer implementation, version 1.900.1, of JPEG 2000. libjasper.tar.gz is a minimal JasPer 1.900.1 library with a simplified Makefile and everything but JPEG 2000 removed.
j2kaudio.tar.gz is the source code, version 200707311200.
File Format
j2a files store the JPEG 2000 compressed audio data. The audio is divided up into many smaller frames; each frame is an independant jp2 image file. Here is an example layout of a 3-frame j2a audio file:
float fps
unsigned long bytes
jp2 image data frame 0
unsigned long bytes
jp2 image data frame 1
unsigned long bytes
jp2 image data frame 2
EOF
Data Organization
Each j2a frame is an individual jp2 JPEG 2000 image that is number-of-channels pixels high and samples pixels wide. The image height for stereo audio data is 2 pixels. The image width for 44100Hz audio at 1 FPS would be 44100 pixels. The image width for 44100Hz audio at 10 FPS would be 4410 pixels. Each jp2 image is 1 16-bit grayscale component, for CD-quality stereo audio data. 8-bit audio is stored as 1 8-bit grayscale component and 32-bit float audio is stored as 3 grayscale components: 1-bit sign, 8-bit exponent, and 23-bit mantissa. An example image/frame layout that shows the audio channels, scans, and samples is shown below.
Scan
0 1 2 3 ...
Channel 0 s0 s1 s2 s3 s...
1 s0 s1 s2 s3 s...
Here are some notes from the code:
j2kaudio extension and format: .j2a
sample rate: 44100 default for testing
number of audio channels: image height: 2 default for testing
suggested channel layout for multimedia apps:
row 0: left (mono)
row 1: right (stereo)
row 2: subwoofer (2.1)
row 3: center (3.1)
row 4: surround left (5.1)
row 5: surround right (5.1)
row 6: back (6.1) or rear left (7.1)
row 7: rear right (7.1)
frames per second: 1 default for testing
sample rate / frames per second: image width: default 44100 / 1 = 44100
bits per sample: 16 default 1-component grayscale images for 16-bit audio
The image frame layout makes adding additional audio channels a trivial task. Additional audio channels, from mono to 5.1 to 24 tracks, just make the images taller.
Compression
The pdf documentation included with the JasPer source code is an excellent reference on JPEG 2000 image compression. An encoding rate of 1 specifies lossless compression, which can recreate the original wave form data for each track exactly. An encoding rate of less than 1 is lossy compression, which means the original wave form data cannot be recreated exactly; information has been lost just like MP3. One interesting aspect of 2-dimensional audio compression is that similarities between channels will enable higher compression. This aspect is similar to MP3 joint-stereo, which also uses similarities between channels to increase compression, but it applies to any number of channels.
Example Program
The program j2kaudio is an example program that converts files from wav to j2a and j2a to wav. The j2kaudio example program loads, compresses or decompresses, and writes one frame of the input file to the output file in a loop, which is ideal for streaming and low memory footprint. The tiny non-streaming open-source wav file I/O code I use just happens to read/write to one giant buffer before doing any disk I/O, which I will fix soon. The example program also truncates the input wav file to the nearest whole frame, so that all frames are the same size. In my test of tk10.wav, this behavior only truncated 0.053s or about 9kB of uncompressed CD-quality data. Future work may allow the last frame to be a special case for a fraction of a second, but I have not worked on that yet.
encoding usage: j2kaudio encodingrate wavfilein.wav j2afileout.j2a (lossless or lossy)
encoding usage: j2kaudio wavfilein.wav j2afileout.j2a (lossless)
encoding usage: j2kaudio wavfilein.wav (lossless)
decoding usage: j2kaudio j2afilein.j2a wavfileout.wav
decoding usage: j2kaudio j2afilein.j2a
Building
- cd ~ (go to a working folder like your home directory or desktop)
- wget http://geocities.com/hildstrom/projects/j2kaudio/libjasper.tar.gz
- wget http://geocities.com/hildstrom/projects/j2kaudio/j2kaudio.tar.gz
- tar xzf libjasper.tar.gz
- cd libjasper
- make
- cd ..
- tar xzf j2kaudio.tar.gz
- cd j2kaudio/src
- make
Lossless Performance
Update: I discovered that passing rate=1 to the JasPer JPEG 2000 encoder does not result in lossless compression like I had thought; some information was lost in my initial testing! j2kaudio now seems to have the worst performance of any lossless compressor tested. Its compression performance is similar to shorten, but it is 10 times slower. =:) Just go straight to the tk10.wav results; I did not bother to retest the other wav files. Lossy performance was not affected by this bug. I discovered this undocumented behavior while working on 32-bit support and checking I/O sample by sample. The rate= option must not be passed to the JasPer JPEG 2000 encoder from the j2kaudio code at all for lossless compression. The lossless compression, obviously, sounds identical to the original. For the 16-bit lossless compression comparison I tested against FLAC, LPAC, Shorten, Monkey's Audio, OptimFrog, and LA. Three 16-bit test files were chosen: sinetest.wav, sweeptest.wav, and tk10.wav. For the 32-bit lossless comparison I tested against wavpack, tar gzip, and tar bzip2. Three 32-bit test files were chosen: 32b.wav and tk10-32b.wav, and 32bWN.wav. The test computer is a 3.0GHz P4, 1GB RAM, Fedora Core 6 Linux, and 32-bit kernel. The red cell text indicates an error in initial testing, green cell text indicates the best performance, and orange indicates an updated result. The tk10.wav file was ripped from an audio CD using CDex, which produced identical output to cdda2wav.
sinetest.wav 44.1kHz/16b/2ch
10sec 60Hz sine wave
1764044B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s)
|
|---|
| j2kaudio | 200707231230 | 73489 | 0.4 | 4.1 | 0.2
|
|---|
| flac | 1.1.2 | 344877 | 1.1 | 19.5 | 0.1
|
|---|
| lpac | 1.40 3.08 | 266520 | 0.5 | 15.1 | 0.2
|
|---|
| shorten | 3.6.1 | 427629 | 0.8 | 24.2 | 0.1
|
|---|
| monkey's audio (mac) | 3.99 | 322948 | 3.1 | 18.3 | 3.3
|
|---|
| optimfrog (ofr) | 4.52 | 204369 | 0.7 | 11.5 | 0.4
|
|---|
| la | 0.4 | 260779 | 2.4 | 14.8 | 2.1
|
|---|
sweeptest.wav 44.1kHz/16b/2ch
60sec 20Hz-20kHz sweep
10999852B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s)
|
|---|
| j2kaudio | 200707231230 | 3642175 | 6.1 | 33.1 | 3.1
|
|---|
| flac | 1.1.2 | 4738626 | 6.9 | 43.0 | 0.3
|
|---|
| lpac | 1.40 3.08 | 4184373 | 4.5 | 38.0 | 1.8
|
|---|
| shorten | 3.6.1 | 6805833 | 0.5 | 61.9 | 0.3
|
|---|
| monkey's audio (mac) | 3.99 | 4725656 | 21.7 | 43.0 | 22.6
|
|---|
| optimfrog (ofr) | 4.52 | 3947514 | 4.0 | 35.8 | 2.8
|
|---|
| la | 0.4 | 4813421 | 16.6 | 43.8 | 14.5
|
|---|
tk10.wav 44.1kHz/16b/2ch
230sec Twista Kamikaze Track 10 So Sexy
40757852B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s)
|
|---|
| j2kaudio rate=1 | 200707231230 | 23024802 | 32.4 | 56.5 | 17.2
|
|---|
| j2kaudio rate= omitted | 200707311200 | 31216712 | 23.0 | 76.6 | 21.3
|
|---|
| flac | 1.1.2 | 28600315 | 26.1 | 70.2 | 1.4
|
|---|
| lpac | 1.40 3.08 | 28110613 | 16.9 | 69.0 | 6.8
|
|---|
| shorten | 3.6.1 | 30651397 | 2.2 | 75.2 | 1.3
|
|---|
| monkey's audio (mac) | 3.99 | 27257032 | 81.4 | 66.9 | 83.7
|
|---|
| optimfrog (ofr) | 4.52 | 27016926 | 13.9 | 66.3 | 10.2
|
|---|
| la | 0.4 | 26710351 | 62.0 | 65.5 | 53.8
|
|---|
| tar czf (gzip) | 1.15.1 | 39589055 | 4.33 | 97.1 | 0.76
|
|---|
| tar cjf (bz2) | 1.15.1 | 39409349 | 19.53 | 96.7 | 10.50
|
|---|
32b.wav 44.1kHz/32b/2ch
60sec L ch 20-20kHz sine wave
R ch 20k-20Hz square wave
21168088B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s)
|
|---|
| j2kaudio | 200708011100 | 17792661 | 21.33 | 84.05 | 18.3
|
|---|
| wavpack | 4.41.0 | 16256172 | 2.23 | 76.79 | 1.16
|
|---|
| tar czf (gzip) | 1.15.1 | 11466309 | 5.69 | 54.17 | 0.4
|
|---|
| tar cjf (bz2) | 1.15.1 | 10257607 | 7.47 | 48.46 | 4.28
|
|---|
tk10-32b.wav 44.1kHz/32b/2ch
230sec L ch 32-bit tk10.wav mixed to mono
R ch 32-bit tk10.wav mixed to mono reversed
81515704B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s)
|
|---|
| j2kaudio | 200708011100 | 54756045 | 78.59 | 67.17 | 68.26
|
|---|
| wavpack | 4.41.0 | 31659262 | 7.54 | 38.84 | 4.12
|
|---|
| tar czf (gzip) | 1.15.1 | 57086563 | 33.80 | 70.03 | 2.35
|
|---|
| tar cjf (bz2) | 1.15.1 | 45290223 | 30.19 | 55.56 | 16.48
|
|---|
32bWN.wav 44.1kHz/32b/10ch
60sec 10 channels of white noise
105840152B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s)
|
|---|
| j2kaudio | 200708011100 | 96382920 | 101.14 | 91.06 | 94.98
|
|---|
| wavpack | 4.41.0 | 87122474 | 12.04 | 82.32 | 6.80
|
|---|
| tar czf (gzip) | 1.15.1 | 97676453 | 13.96 | 92.29 | 1.92
|
|---|
| tar cjf (bz2) | 1.15.1 | 98686951 | 50.47 | 93.24 | 27.01
|
|---|
Lossy Performance
For the lossy compression comparison I tested against LAME, Ogg Vorbis, and AAC (faac/faad). The lossy quality of j2kaudio does not sound as good as mp3, ogg, or aac of similar bitrate. The main audible compression artifact is high-frequency noise introduced while trying to compress high-frequency audio notes like the symbol, high-hat, triangle, or hiss. The traditional lossy audio encoders lose this high-frequency data in a smooth and mostly-unobtrusive way, although it is still audible. Even though j2kaudio produces some high-frequency noise, it recreates the original wave with less error than the other lossy encoding methods, which I calculated in 2003. A lowpass filter should be able to reduce the highest frequencies from 20kHz down to 15kHz or so, which would ease the compression noise, but this approach has not been implemented yet.
sinetest.wav 44.1kHz/16b/2ch
10sec 60Hz sine wave
1764044B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s) | audio quality rank
|
|---|
| j2kaudio | 200707231230 | 73489 | 0.4 | 4.1 | 0.2 | 4
|
|---|
| mp3 (lame) | 3.97 | 161331 | 0.7 | 9.1 | 0.1 | 3
|
|---|
| ogg (oggenc) | 1.0.2 | 163720 | 1.6 | 9.3 | 0.1 | 2
|
|---|
| aac (faac) | 1.25 | 162875 | 0.7 | 9.2 | 0.1 | 1
|
|---|
sweeptest.wav 44.1kHz/16b/2ch
60sec 20Hz-20kHz sweep
10999852B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s) | audio quality rank
|
|---|
| j2kaudio | 200707231230 | 918209 | 5.8 | 8.3 | 1.6 | 4
|
|---|
| mp3 (lame) | 3.97 | 998921 | 4.6 | 9.1 | 0.9 | 3
|
|---|
| ogg (oggenc) | 1.0.2 | 1015369 | 11.8 | 9.2 | 0.5 | 2
|
|---|
| aac (faac) | 1.25 | 1009872 | 4.7 | 9.2 | 0.5 | 1
|
|---|
tk10.wav 44.1kHz/16b/2ch
230sec Twista Kamikaze Track 10 So Sexy
40757852B
| version | compressed size (B) | encoding time (s) | compressed size (%) | decoding time (s) | audio quality rank
|
|---|
| j2kaudio | 200707231230 | 3461847 | 28.6 | 8.5 | 6.0 | 4
|
|---|
| mp3 (lame) | 3.97 | 3698101 | 20.0 | 9.1 | 3.1 | 3
|
|---|
| ogg (oggenc) | 1.0.2 | 3757244 | 52.6 | 9.2 | 2.0 | 2
|
|---|
| aac (faac) | 1.25 | 3734175 | 14.2 | 9.2 | 2.4 | 1
|
|---|
