Libmad on Android with the NDK
So i was porting all the decoders i had build for the onset detection tutorial to C++, using libmad as the mp3 decoder of choice. After getting that to work on the desktop properly i had to make it work on Android too. Now, there’s no build of libmad in the NDK for obvious reasons, so i had to build that myself. As the autotools configure script of libmad is not useable with the NDK toolchain i used the config.h file from http://gitorious.org/rowboat/external-libmad/blobs/raw/master/android/config.h, which has all the settings needed for building libmad on Android. Compiling libmad is then a simple matter of creating a proper Android.mk and Application.mk file. The Android.mk file looks like this:
LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := audio-tools LOCAL_ARM_MODE := arm LOCAL_SRC_FILES := NativeWaveDecoder.cpp NativeMP3Decoder.cpp mad/bit.c mad/decoder.c mad/fixed.c mad/frame.c mad/huffman.c mad/layer12.c mad/layer3.c mad/stream.c mad/synth.c mad/timer.c mad/version.c LOCAL_CFLAGS := -DHAVE_CONFIG_H -DFPM_ARM -ffast-math -O3 include $(BUILD_SHARED_LIBRARY)
Now there’s a couple of things that initially bogged down the performance of this. I tested it with the song “Schism” by tool which is a 6:47min long song, encoded at 192kbps. The file weights in at 9.31mb, pretty big for an mp3 imo. NativeMP3Decoder is just a libmad based implementation of the MP3Decoder in the onset detection tutorial framework. So it has a simple NativeMP3Decoder.readSamples method which will fill a float array with as many samples as there are elements in the float array. If the input file is in stereo the channels get mixed down to mono by averaging. The NativeMP3Decoder.readSamples()method internally calls a native method with a similar signature. Instead of a float array i pass in a direct ByteBuffer that has enough storage to hold all the samples requested. The native wrapper then writes the samples to this direct buffer which in turn then gets copied to the float array passed into theNativeMP3Decoder.readSamples() method. It looks something like this:
public int readSamples(float[] samples)
{
if( buffer == null || buffer.capacity() != samples.length )
{
ByteBuffer byteBuffer = ByteBuffer.allocateDirect( samples.length * Float.SIZE / 8 );
byteBuffer.order(ByteOrder.nativeOrder());
buffer = byteBuffer.asFloatBuffer();
}
int readSamples = readSamples( handle, buffer, samples.length );
if( readSamples == 0 )
{
closeFile( handle );
return 0;
}
buffer.position(0);
buffer.get( samples );
return samples.length;
}
The call to buffer.get( samples ) kills it all. Without any optimizations (thumb code, -O0, -DFPM_DEFAULT == standard fixed point math in libmad, no arm assembler optimized fp math) decoding the complete files takes 184 seconds on my Milestone. Holy shit, batman! If i eliminate thebuffer.get( samples ) call that gets down to 44 seconds! Incredible. Now i still thought that is way to slow so i started adding optimizations. The first thing i did was compiling to straight arm instead of thumb code. You can tell the NDK toolchain to do so by placing this in the Android.mk file:
LOCAL_ARM_MODE := arm
With this enabled decoding takes 36 seconds. The next thing i did was agressive optimization via -O3 as a CFLAG. That shaved off only 2 more seconds, so nothing to write home about. The last optimization is libmad specific. The config.h file i linked to above does not define the fixed point math mode libmad should use. Now, when you have a look at fixed.h of libmad you can see quiet some options for fixed point math there. There’s also a dedicated option for arm processors that uses some nice little arm assembler code to do the heavy lifting. You can enable this by passing -DFPM_ARM as a CFLAG. Now that did wonders! i’m now down to 20 seconds for decoding 407 seconds of mp3 encoded audio. That’s roughly 20x real-time which is totally ok with me. The song i chose is at the extreme end of the song length spectrum i will have to handle in my next audio game project. A song a user uses will be processed once and waiting for that 20 seconds is ok in my book.
I’m afraid i won’t release the source of the ported audio framework as it’s a bit of a mess and would need some work to clean up. What i can give you is the plain source for the native side of the NativeMP3Decoder class if you guarantee me not to laugh. My C days are long over so there’s probably a shitload of don’ts in there. The “handle” system is also kind of creative but good enough for my needs. I learned how to use the low level libmad api by looking at the code here. I actually like doing it this way more than with the shittycallback high-level API. Your mileage may vary. So here it is, be afraid:
#include "NativeMP3Decoder.h"
#include "mad/mad.h"
#include <stdio.h>
#include <string.h>
#define SHRT_MAX (32767)
#define INPUT_BUFFER_SIZE (5*8192)
#define OUTPUT_BUFFER_SIZE 8192 /* Must be an integer multiple of 4. */
/**
* Struct holding the pointer to a wave file.
*/
struct MP3FileHandle
{
int size;
FILE* file;
mad_stream stream;
mad_frame frame;
mad_synth synth;
mad_timer_t timer;
int leftSamples;
int offset;
unsigned char inputBuffer[INPUT_BUFFER_SIZE];
};
/** static WaveFileHandle array **/
static MP3FileHandle* handles[100];
/**
* Seeks a free handle in the handles array and returns its index or -1 if no handle could be found
*/
static int findFreeHandle( )
{
for( int i = 0; i < 100; i++ )
{
if( handles[i] == 0 )
return i;
}
return -1;
}
static inline void closeHandle( MP3FileHandle* handle )
{
fclose( handle->file );
mad_synth_finish(&handle->synth);
mad_frame_finish(&handle->frame);
mad_stream_finish(&handle->stream);
delete handle;
}
static inline signed short fixedToShort(mad_fixed_t Fixed)
{
if(Fixed>=MAD_F_ONE)
return(SHRT_MAX);
if(Fixed<=-MAD_F_ONE)
return(-SHRT_MAX);
Fixed=Fixed>>(MAD_F_FRACBITS-15);
return((signed short)Fixed);
}
JNIEXPORT jint JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_openFile(JNIEnv *env, jobject obj, jstring file)
{
int index = findFreeHandle( );
if( index == -1 )
return -1;
const char* fileString = env->GetStringUTFChars(file, NULL);
FILE* fileHandle = fopen( fileString, "rb" );
env->ReleaseStringUTFChars(file, fileString);
if( fileHandle == 0 )
return -1;
MP3FileHandle* mp3Handle = new MP3FileHandle( );
mp3Handle->file = fileHandle;
fseek( fileHandle, 0, SEEK_END);
mp3Handle->size = ftell( fileHandle );
rewind( fileHandle );
mad_stream_init(&mp3Handle->stream);
mad_frame_init(&mp3Handle->frame);
mad_synth_init(&mp3Handle->synth);
mad_timer_reset(&mp3Handle->timer);
handles[index] = mp3Handle;
return index;
}
static inline int readNextFrame( MP3FileHandle* mp3 )
{
do
{
if( mp3->stream.buffer == 0 || mp3->stream.error == MAD_ERROR_BUFLEN )
{
int inputBufferSize = 0;
if( mp3->stream.next_frame != 0 )
{
int leftOver = mp3->stream.bufend - mp3->stream.next_frame;
for( int i = 0; i < leftOver; i++ )
mp3->inputBuffer[i] = mp3->stream.next_frame[i];
int readBytes = fread( mp3->inputBuffer + leftOver, 1, INPUT_BUFFER_SIZE - leftOver, mp3->file );
if( readBytes == 0 )
return 0;
inputBufferSize = leftOver + readBytes;
}
else
{
int readBytes = fread( mp3->inputBuffer, 1, INPUT_BUFFER_SIZE, mp3->file );
if( readBytes == 0 )
return 0;
inputBufferSize = readBytes;
}
mad_stream_buffer( &mp3->stream, mp3->inputBuffer, inputBufferSize );
mp3->stream.error = MAD_ERROR_NONE;
}
if( mad_frame_decode( &mp3->frame, &mp3->stream ) )
{
if( mp3->stream.error == MAD_ERROR_BUFLEN ||(MAD_RECOVERABLE(mp3->stream.error)))
continue;
else
return 0;
}
else
break;
} while( true );
mad_timer_add( &mp3->timer, mp3->frame.header.duration );
mad_synth_frame( &mp3->synth, &mp3->frame );
mp3->leftSamples = mp3->synth.pcm.length;
mp3->offset = 0;
return -1;
}
JNIEXPORT jint JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_readSamples__ILjava_nio_FloatBuffer_2I(JNIEnv *env, jobject obj, jint handle, jobject buffer, jint size)
{
MP3FileHandle* mp3 = handles[handle];
float* target = (float*)env->GetDirectBufferAddress(buffer);
int idx = 0;
while( idx != size )
{
if( mp3->leftSamples > 0 )
{
for( ; idx < size && mp3->offset < mp3->synth.pcm.length; mp3->leftSamples--, mp3->offset++ )
{
int value = fixedToShort(mp3->synth.pcm.samples[0][mp3->offset]);
if( MAD_NCHANNELS(&mp3->frame.header) == 2 )
{
value += fixedToShort(mp3->synth.pcm.samples[1][mp3->offset]);
value /= 2;
}
target[idx++] = value / (float)SHRT_MAX;
}
}
else
{
int result = readNextFrame( mp3 );
if( result == 0 )
return 0;
}
}
if( idx > size )
return 0;
return size;
}
JNIEXPORT jint JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_readSamples__ILjava_nio_ShortBuffer_2I(JNIEnv *env, jobject obj, jint handle, jobject buffer, jint size)
{
MP3FileHandle* mp3 = handles[handle];
short* target = (short*)env->GetDirectBufferAddress(buffer);
int idx = 0;
while( idx != size )
{
if( mp3->leftSamples > 0 )
{
for( ; idx < size && mp3->offset < mp3->synth.pcm.length; mp3->leftSamples--, mp3->offset++ )
{
int value = fixedToShort(mp3->synth.pcm.samples[0][mp3->offset]);
if( MAD_NCHANNELS(&mp3->frame.header) == 2 )
{
value += fixedToShort(mp3->synth.pcm.samples[1][mp3->offset]);
value /= 2;
}
target[idx++] = value;
}
}
else
{
int result = readNextFrame( mp3 );
if( result == 0 )
return 0;
}
}
if( idx > size )
return 0;
return size;
}
JNIEXPORT void JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_closeFile(JNIEnv *env, jobject obj, jint handle)
{
if( handles[handle] != 0 )
{
closeHandle( handles[handle] );
handles[handle] = 0;
}
}
To compile that for Android all you have to do is download libmad and put the source files into your Android project’s jni folder along with the code above. Then use the Android.mk from above et voila you got yourself a native mp3 decoder for Android. You can use it in combination with the AndroidAudioDevice class of the last post. If you feel adventureous you could even extend it to return stereo data.