hello, First of all, my english is bad. I am looking for descriptions and off source code to analyse 8 bits 22050 hz WAV mono sound to move lips for a (bitmap)animation and my servo''s from my robot''s. A also looking for (free)bitmap sprites or a 3D character with animated mouth or lips. Thanks, Henk from Holland.

Hello!

I can''t help you find any code. However, I''m confident that it can''t be too hard to make a mouth go up and down, or at least be open or shut, by analysing a WAV file. What is hard is matching the different mouth shapes, I think that is a manual labour job I''m afraid. Anyone know about this?

I''m not a programmer, but maybe the AI guys? I unno ... prolly not anyone on this board anyway.

Thanks for the reply.
I am working now on a application that analysing the wave shape.
This is (byte) 128 silence, 0 lower wave, 255 high wave.
De sum of the energy is add low and high together. Low 45 and high (235-128) You cannot add 45 and 235 this is bigger than max 8 bit sample of 255.De energy is 235-128=107+45=152. this value must by open the mouth(lips). Now i am working on the frequentie of te wave, How many wave''s in a seccond.
Thread,

just a idea how to diffrentiate between vowels (aeiouy).
Maybe you could find out how ''square'', ''triangular'' and ''sinusiodal'' the sound is through a Fourier Transform, and check for the frequency spread.
if the freq is spread like 1,2,4,6,8 (times the basic freq) it is square, if it is spread as 1,3,5,7,9 it is saw ... and so on, no muliplies would be a sinusoidal sound...

Maybe its overkill...

Thanks for the idea. I am looking for a C/C++ FFT function.
I have some examples in (math)text format but that is ferry difficult for me. Maybe in C source its better to learn about it.
I have also look at filtering the sound source.
Now i have a function that give my the energy of the wave.
Also with a variable threshold build in.
Idea''s about this function are welkom.
The source://-----------------------------------------------------------------------------
// Name: ShowEnergy()
// Desc: Display the energy level.
//-----------------------------------------------------------------------------
void ShowEnergy(HWND hDlg,DWORD nIndex)
{
if(Eng < Freq) // Freq = 22050 sample /50 hz = 441
{
nWav = TempBuffer[nIndex]; // soundbuffer[n]

if(nWav >= 128) // high ampltude wave
{
nWav = nWav-128; // yes example 245-128=117
}
else
{
nWav = 128-nWav; // low amplitude wave inverse
}
// check threshold > behind 0 to 255 (8 bits sound format)
if(nWav > (int)dwThresHold) tWav+=nWav;

Eng++;
}
else
{
Eng=1;
if(tWav > Freq)
{
tWav = tWav/Freq;
SendMessage( GetDlgItem(hDlg,IDC_PROGRESS1),PBM_SETPOS,tWav , 0 );
//Sleep(100);
SetDlgItemInt(hDlg,IDC_ENERGY,tWav,FALSE);
}
else
{
SendMessage( GetDlgItem(hDlg,IDC_PROGRESS1),PBM_SETPOS,0 , 0 );
SetDlgItemInt(hDlg,IDC_ENERGY,0,FALSE);
}
}

}// end ShowEnergy

At some point, when you want to get *really* detailed, you might want to look into the IPA (International Phonetic Alphabet), it''s typically a tool used by singers, but what it''s for is dividing up all of the different sounds that different European languages make, and classifying them, and instructing on how to shape the mouth properly to make them.

Singers use it to learn how to properly sing in foreign languages that they do not know but need to have a correct accent within.

Some books can give really good ways of showing exactly what it is our mouths do on plosives versus hard consonants, versus dipthongs and tripthongs and the like, could be very useful for a realistic lip-synch interpreter.

The internet is awash with information on phonemes:

www.gamasutra.com/features/20000406/lander_01.htm

www.cis.ohio-state.edu/research/graphics/research/FacialAnimation/

www.cs.wright.edu/~kpraveen/FA/overview.htm

Thanks for the answers to all.
My application begins to work, at the moment not with phonemes but maybe in the future. Now I am only using the power and the frequency of the waveforms. And yes, my character move his mouth.
I now I have a lot off work to fine-tune the application.
But this is really fun.
And then I go to phoneme (in Dutch) animation (really hard I think).
I am gonna visit the site’s from your answers and go on with it.

regard’s Thread.