How to Stream Audio Captured via speex c#

Oct 29, 2010 at 10:19 AM

Hi,

  how we can stream audio captured using speex.Currently my code save the sound using
            fileStream = new FileStream("test.spx", FileMode.OpenOrCreate);
            binaryWriter = new BinaryWriter(fileStream);

and play from this file .

What i want is to stream audio after encoding it and on the other end decode it before playing

How it is possible??.. .All my code is in C# and i use socket.Send(byte[] )  for sending data..

Thanks

JIbin

jibin.mn@gmail.com

Coordinator
Oct 30, 2010 at 9:04 AM

Hi,

Of course is it possible to stream voice data encoded with NSpeex.

The only (but big) difference to playing from a file is that you have to wait on the socket until a complete package is received. So for that you would some sort of container which wraps a package (eg Ogg but there are numerous others). However, the minimum would be to send the length of the package before sending the data so the receiver knows how many bytes to read from the socket.

Since you have to transfer the data in real-time it would lead to problems when the network conditions change. For this you would need a jitter buffer (have a look at the example code). Also if you are using udp as transport protocol (I would suggest that) you have to re-order the packages.

Christoph

Am 2010 10 29 11:19 schrieb "jibin4119" <notifications@codeplex.com>:
> From: jibin4119
>
> Hi, how we can stream audio captured using speex.Currently my code save the sound using
> fileStream = new FileStream("test.spx", FileMode.OpenOrCreate);
> binaryWriter = new BinaryWriter(fileStream);and play from this file .What i want is to stream audio after encoding it and on the other end decode it before playingHow it is possible??.. .All my code is in C# and i use socket.Send(byte[] ) for sending data..ThanksJIbinjibin.mn@gmail.com
>
>
Nov 1, 2010 at 9:45 AM

HI,

  Thanks for the reply.....

Now i am able to capture sound and at the same time i send it over network like this

 

private void waveIn_DataAvailable(object sender, WaveInEventArgs e)
        {          
            byte[] buffer = e.Buffer;

            byte[] dataToWrite = ALawEncoder.ALawEncode(buffer);

            if (socket_Audio != null)
                socket_Audio.SendTo(dataToWrite, new IPEndPoint(IPAddress.Parse(RemoteIpv6), SoundPort));
       
        }

 

Now the  problem that i am facing is how to get back this audio at receiving end and and send it to the speaker of the system

I am just starting a new thread for rxving data from network

  myAudioThread = new Thread(new ThreadStart(AudioListener));
  myAudioThread.Start();

And in the AudioListener() i use the following code

 #region for Audio socket Its rxving audio always
          
            try
            {

                socket_Audio = new Socket(AddressFamily.InterNetworkV6, SocketType.Dgram, ProtocolType.Udp);

                socket_Audio.Bind(new IPEndPoint(IPAddress.IPv6Any, SoundPort));
                IPEndPoint remoteEP = new IPEndPoint(IPAddress.IPv6Any, SoundPort);
                    //Receive data.
                   
                byte[] byteData;
                while (true)
                {

                    byteData = new byte[2048];
                    //Receive data.
                    socket_Audio.Receive(byteData);
                    
                    //G711 compresses the data by 50%, so we allocate a buffer of double
                    //the size to store the decompressed data.
                    byte[] byteDecodedData = new byte[byteData.Length * 2];

                    //Decompress data using the proper vocoder.
                    
                    ALawDecoder.ALawDecode(byteData, out byteDecodedData);

                  

                }

Now the decoded audio is in the byte array byteDecodedData.How i send it to speaker of that system.

I found only waveOut.Play()  and

 public override int Read(byte[] buffer, int offset, int count)
 {

   //code to process data

 }

  in the examples

Pls guide me to the next step....Pls let me know if i am in wrong path.

Thanks

JIbin

Coordinator
Nov 3, 2010 at 9:03 AM

Yes, that's where the work starts ;)

ok, the waveout is an object taken from the NAudio library (can be found under: http://naudio.codeplex.com/). NAudio facilitates the playback of audio data coming from any source. Thus, it supports a stream interface.

Now, this stream has to be filled from the data coming from the network. But as mentioned in my previous post you have to use a buffer between the data from the network stream and the data read into the playback system. The thread reading the data from the network puts it into the buffer. On the other side there is the playback thread. Created automatically by the playback system which constantly asks for data to be played (since this is timed it asks every couple of milliseconds). In the playback thread you have to query the buffer for the next still encoded audio data and when found decode the data and return it. Have a look at the decoding with a jitterbuffer example (found under: http://nspeex.codeplex.com/documentation) plus the sample code in MainViewModel.cs of NSpeexTest Project. You can actually use most of this code buy replacing the WCF stuff with your socket code.

Nov 3, 2010 at 1:33 PM
Edited Nov 3, 2010 at 2:04 PM

Hi,
 Thanks for the information to go on with Audio conferencing using Naudio.One Good Thing is that now i am able to hear the audio in Remote-PC as live ..But Some problems still i am facing are the speech is not clear in remote machine (when i make some ticks in mic it is return as ticks  in remote machine,but when i say 'Hello' in remote machine its not 'Hello' Some disturbances only)
I thought i am not correctly implement the second part of your reply (i am not an experienced programmer..But i try to implement that with my knowledge )


"On the other side there is the playback thread. Created automatically by the playback system which constantly asks for data to be played (since this is timed it asks every couple of milliseconds). In the playback thread you have to query the buffer for the next still encoded audio data and when found decode the data and return it."

Now i am using g711Encoding&Decoding for simplicity is that a problem/not ?Hope you can solve my problem

My Code in Rxving Side ---Capture from network Stream & send to Speaker

public void start_AudioTransffer()
{
try
{

//start a thread to continuosly rxv data from network

myAudioThread = new Thread(new ThreadStart(AudioListener));
myAudioThread.Start();

}
catch (Exception e)
{
ObjLog.Write("Exception at Initiate()" + e.Message);
}
}



private void AudioListener()
{
//for storing audio captured from socket

MemoryStream AudioRxvdStream;

#region Its rxving audio From Network Continously
.I thought the problem is on this part
try
{
socket_Audio = null;

socket_Audio = new Socket(AddressFamily.InterNetworkV6, SocketType.Dgram, ProtocolType.Udp);

socket_Audio.Bind(new IPEndPoint(IPAddress.IPv6Any, SoundPort));

byte[] byteData;
while (true)
{
AudioRxvdStream = null;
AudioRxvdStream = new MemoryStream();
byteData = new byte[800];
socket_Audio.Receive(byteData);
//G711 compresses the data by 50%, so we allocate a buffer of double size to store the decompressed data.
byte[] byteDecodedData = new byte[byteData.Length * 2];
//Decompress data using the proper vocoder.
ALawDecoder.ALawDecode(byteData, out byteDecodedData);



//write data captured to memorystream
AudioRxvdStream.Write(byteDecodedData, 0, byteDecodedData.Length);


//call waveout clss for playing audio
if (AudioRxvdStream != null)
{
waveOut = null;
waveOut = new WaveOut();
waveOut.Init(new WaveOutStream(AudioRxvdStream));
waveOut.Play();
}
}
}
catch (Exception e)
{
ObjLog.Write("Exception- Audio-Listner() -- Rxv Part*************" + e.ToString());
}
 
}

And implementation of the WaveOutStream Class is
public class WaveOutStream : WaveStream
{
MemoryStream AudioRxvdStream;
private bool Streaming = true;
clsLog ObjLog = new clsLog();

public WaveOutStream(MemoryStream _AudioRxvdStream)
{
AudioRxvdStream = _AudioRxvdStream;
AudioRxvdStream.Position = 0;
AudioRxvdStream.SetLength(0);
}

public override int Read(byte[] buffer, int offset, int count)
{
try
{
if (AudioRxvdStream.Length == 0)
{
buffer[0] = 0;
return 1;
}
else
{
AudioRxvdStream.Position = 0;
int bytesRead = AudioRxvdStream.Read(buffer, offset, count);

return bytesRead;
}
}
catch (Exception ec)
{
ObjLog.Write("Exception In Reading From Stream*************" + ec.Message);
return 1;
}
}

public override WaveFormat WaveFormat
{
get { return new WaveFormat(8000, 16, 1); }
}

public override long Length
{
get { return AudioRxvdStream.Length; }
}

public override long Position
{
get
{
throw new NotImplementedException();
}
set
{
throw new NotImplementedException();
}
}
}


After a minute of execution i got this exception also
"NAudio.MmException: MemoryAllocationError calling waveOutOpen at NAudio.Wave.WaveOut.Init(IWaveProvider waveProvider)"

My Code in Sending Side (I thought this part is OK)---Capture from mic & send to Remote PC

public void start_AudioTransffer()
{
try
{
waveIn = null;

waveIn = new WaveIn { WaveFormat = new WaveFormat(8000, 16, 1) };

waveIn.DataAvailable += waveIn_DataAvailable;

waveIn.RecordingStopped += waveIn_RecordingStopped;

waveIn.StartRecording();

btn_voicecontrol.Text = "Mute Voice";
}
catch (Exception e)
{
ObjLog.Write("Exception at Initiate()" + e.Message);
}
}

private void waveIn_DataAvailable(object sender, WaveInEventArgs e)
{

try
{
byte[] buffer = e.Buffer;

byte[] dataToWrite = ALawEncoder.ALawEncode(buffer);

if (socket_Audio != null)
{
socket_Audio.SendTo(dataToWrite, new IPEndPoint(IPAddress.Parse(RemoteIpv6), SoundPort));

}
}
catch (Exception ex)
{
ObjLog.Write("Exception at Send_AudioTransffer()***********" + ex.Message);
}
}



Help me to correct the problem if something wrong in my code/logic

Thanks,
JIbin
jibin.mn@hotmail.com

 

Coordinator
Nov 3, 2010 at 6:50 PM

I don't know much about the G711 codec.
One point would to check that you have the correct sample rate, sample size on both sides. Also make sure that you read a complete packet from the socket so you need to wrap an encoded packet into a container (as mentioned in my first post).

Am 2010 11 3 13:34 schrieb "jibin4119" <notifications@codeplex.com>:
> From: jibin4119
>
> Hi,
> Thanks for the information to go on with Audio conferencing using Naudio.One Good Thing is that now i am able to hear the audio in Remote-PC as live ..But Some problems still i am facing are the speech is not clear in remote machine (when i make some ticks in mic it is return as ticks in remote machine,but when i say 'Hello' in remote machine its not 'Hello' Some disturbances only)
> I thought i am not correctly implement the second part of your reply (i am not an experienced programmer..But i try to implement that with my knowledge )
> "On the other side there is the playback thread. Created automatically by the playback system which constantly asks for data to be played (since this is timed it asks every couple of milliseconds). In the playback thread you have to query the buffer for the next still encoded audio data and when found decode the data and return it."
>
> Now i am using g711Encoding&Decoding for simplicity is that a problem/not ?Hope you can solve my problem
>
> My Code in Rxving Side ---Capture from network Stream & send to Speakerpublic void start_AudioTransffer(){ try {
> //start a thread to continuosly rxv data from network myAudioThread = new Thread(new ThreadStart(AudioListener)); myAudioThread.Start(); } catch (Exception e) { ObjLog.Write("Exception at Initiate()" + e.Message); }}private void AudioListener(){ //for storing audio captured from socket MemoryStream AudioRxvdStream; #region Its rxving audio From Network Continously .I thought the problem is on this part try { socket_Audio = null; socket_Audio = new Socket(AddressFamily.InterNetworkV6, SocketType.Dgram, ProtocolType.Udp); socket_Audio.Bind(new IPEndPoint(IPAddress.IPv6Any, SoundPort)); byte[] byteData; while (true) { AudioRxvdStream = null; AudioRxvdStream = new MemoryStream(); byteData = new byte[800]; socket_Audio.Receive(byteData); //G711 compresses the data by 50%, so we allocate a buffer of double size to store the decompressed data. byte[] byteDecodedData = new byte[byteData.Length * 2]; //Decompress data using the proper vocoder. ALawDecoder.ALawDecode(byteData, out byteDecodedData); //write data captured to memorystream AudioRxvdStream.Write(byteDecodedData, 0, byteDecodedData.Length); //call waveout clss for playing audio if (AudioRxvdStream != null) { waveOut = null; waveOut = new WaveOut(); waveOut.Init(new WaveOutStream(AudioRxvdStream)); waveOut.Play(); } } } catch (Exception e) { ObjLog.Write("Exception- Audio-Listner() -- Rxv Part*************" + e.ToString()); }
> }And implementation of the WaveOutStream Class is
> public class WaveOutStream : WaveStream{ MemoryStream AudioRxvdStream; private bool Streaming = true; clsLog ObjLog = new clsLog(); public WaveOutStream(MemoryStream _AudioRxvdStream) { AudioRxvdStream = _AudioRxvdStream; AudioRxvdStream.Position = 0; AudioRxvdStream.SetLength(0); } public override int Read(byte[] buffer, int offset, int count) { try { if (AudioRxvdStream.Length == 0) { buffer[0] = 0; return 1; } else { AudioRxvdStream.Position = 0; int bytesRead = AudioRxvdStream.Read(buffer, offset, count); return bytesRead; } } catch (Exception ec) { ObjLog.Write("Exception In Reading From Stream*************" + ec.Message); return 1; } } public override WaveFormat WaveFormat { get { return new WaveFormat(8000, 16, 1); } } public override long Length { get { return AudioRxvdStream.Length; } } public override long Position { get { throw new NotImplementedException(); } set { throw new NotImplementedException(); } }}
> My Code in Sending Side (I thought this part is OK)---Capture from mic & send to Remote PC
>
> public void start_AudioTransffer(){ try { waveIn = null; waveIn = new WaveIn { WaveFormat = new WaveFormat(8000, 16, 1) }; waveIn.DataAvailable += waveIn_DataAvailable; waveIn.RecordingStopped += waveIn_RecordingStopped; waveIn.StartRecording(); btn_voicecontrol.Text = "Mute Voice"; } catch (Exception e) { ObjLog.Write("Exception at Initiate()" + e.Message); }}private void waveIn_DataAvailable(object sender, WaveInEventArgs e){ try { byte[] buffer = e.Buffer; byte[] dataToWrite = ALawEncoder.ALawEncode(buffer); if (socket_Audio != null) { socket_Audio.SendTo(dataToWrite, new IPEndPoint(IPAddress.Parse(RemoteIpv6), SoundPort)); } } catch (Exception ex) { ObjLog.Write("Exception at Send_AudioTransffer()***********" + ex.Message); }}
> Help me to correct the problem if something wrong in my code/logic
>
> Thanks,
> JIbin
> jibin.mn@hotmail.com
>
>