Inertia's picture

[Solved] OpenAL Capture Extension

Edit: The source code below has been changed to work, so don't be confused if the questions don't make too much sense.

I've been wondering if someone managed to get OpenTK.Audio's capture functionality to work?

This code runs fine, does neither throw exceptions nor report any Alc.GetError, but apparently the array is all 0's after the call to Alc.CaptureSamples (which is afaik legal to call after Alc.CaptureStop).

It would be really appreciated if someone could point to where the gremlin is hiding. Afaik the audio hardware is ok (works with other programs), the problem also happens after a reboot (so it's not related to the device not being properly released from a previous run of the program) and the platform is Vista x64 with OpenAL Soft driver (the .dll itself does not report any version number).

using System;
using System.Threading;
using System.Runtime.InteropServices;
 
using OpenTK.Audio;
 
namespace OpenALCaptureTestApplication
{
    class Program
    {
 
        public static IntPtr RecordingDevice;
 
        public static void CheckAlcError( string location )
        {
            AlcError ALCerr = Alc.GetError( RecordingDevice );
            if ( ALCerr != AlcError.NoError )
                Console.WriteLine( "ALC RecordingDevice: An error occured at " + location + "  Value: " + ALCerr.ToString() );
            ALError ALerr = AL.GetError();
            if ( ALerr != ALError.NoError )
                Console.WriteLine( "AL Error occured at " + location + "  Value: " + ALerr.ToString() );
        }
 
        static void Main( string[] args )
        {
            const ALFormat Format = ALFormat.Mono16;
            const int Frequency = 22050;
            int sampleCount = 0;
 
            const int MyBufferSizeInBytes = 50000;
            byte[] MyBuffer = new byte[MyBufferSizeInBytes];
            GCHandle handle = GCHandle.Alloc( MyBuffer, GCHandleType.Pinned );
            IntPtr MyBufferPtr = handle.AddrOfPinnedObject();
 
            Console.WriteLine( "Found Devices: " );
            foreach ( string s in AudioContext.AvailableDevices )
                Console.WriteLine( s );
 
            AudioContext PlaybackDevice = new AudioContext();
            CheckAlcError( "opening the AudioContext" );
 
            IntPtr dev = Alc.GetContextsDevice( Alc.GetCurrentContext() );
            if ( Alc.IsExtensionPresent( dev, "ALC_EXT_CAPTURE" ) )
                Console.WriteLine( "Capture Extension found." );
 
            string RecorderName = Alc.GetString( dev, AlcGetString.CaptureDefaultDeviceSpecifier );
            Console.WriteLine( "Recording with: " + RecorderName );
            RecordingDevice = Alc.CaptureOpenDevice( RecorderName, Frequency, Format, MyBufferSizeInBytes );
            CheckAlcError( "opening the recording device" );
 
            Alc.CaptureStart( RecordingDevice );
            CheckAlcError( "starting the capture" );
 
            do
            {
                Alc.GetInteger( RecordingDevice, AlcGetInteger.CaptureSamples, 1, out sampleCount );
                Console.Write( sampleCount + " " );
                Thread.Sleep( 10 );
            }
            while ( sampleCount < MyBufferSizeInBytes / 2 );
 
            Alc.CaptureStop( RecordingDevice );
            CheckAlcError( "stopping the capture" );
 
            Alc.CaptureSamples( RecordingDevice, MyBufferPtr, MyBufferSizeInBytes / 2 );
            CheckAlcError( "consuming the samples" );
 
            int before = sampleCount;
            Alc.GetInteger( RecordingDevice, AlcGetInteger.CaptureSamples, 1, out sampleCount );
            Console.WriteLine( "\nDone recording. Consumed " + ( before - sampleCount ) + " samples. " + sampleCount + " samples left to consume." );
 
            // Playback
 
            uint src;
            AL.GenSource( out src );
            uint buf;
            AL.GenBuffer( out buf );
            AL.BufferData( buf, Format, MyBufferPtr, MyBufferSizeInBytes, Frequency );
 
            AL.BindBufferToSource( src, buf );
 
            AL.Source( src, ALSourceb.Looping, true );
            // AL.Source( src, ALSourcef.Gain, 32.0f );
            // AL.Listener( ALListenerf.Gain, 32.0f );
            AL.SourcePlay( src );
 
            Console.WriteLine( "Press Enter to exit." );
            Console.ReadLine();
 
            AL.SourceStop( src );
            AL.BindBufferToSource( src, 0 );
            AL.DeleteBuffer( ref buf );
            AL.DeleteSource( ref src );
 
            Alc.CaptureCloseDevice( RecordingDevice );
            PlaybackDevice.Dispose();
 
            handle.Free();
        }
    }
}

Some thing that might be worth mentioning is that the returned data from Alc.GetString for querying the default device and all devices does not match. The default device is not present in the list of all devices.

Another thing that might be the problem is that the signature of Alc.CaptureSamples( RecordingDevice, out MyBufferPtr, out sampleCount ); is incorrect. The last parameter is definitely an input (how many of the recorded samples do you want in MyBufferPtr) and the second parameter is not written to, it's also an input of a pointer, where the data should be written to.

Although the last issue is something easy to resolve I'd really like some feedback whether someone has used this before with success, and whether the design to create the recording device and start the recording is actually correct. I have a very low opinion of the onboard sound from my mainboard (when the onboard LAN and onboard sound are both active for a longer period of time sometimes random failures occur in either of them) so I'd really like some feedback whether someone else has used the AL Extension with success before. Thanks!


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
the Fiddler's picture

I haven't used the capture device before - for all I know, this part of OpenTK is completely untested.

I've been following the OpenAL list for a few years and there are several detailed examples on capture contexts. Here is one of the more complete ones.

A few points: the code queries Alc.GetInteger(RecordingDevice, ALC_CAPTURE_SAMPLES, 1, out SamplesAvailable) repeatedly until SamplesAvailable become larger than the desired "block" size. Once that happens, it calls Alc.CaptureSamples to read the block (the capture device is still running, but I don't think this is necessary).

The code is similar to yours in most important aspects, so it is likely the issue lies in the Alc.CaptureSamples signature: the unmanaged code effectively receives 0 as the sample count and returns without writing anything.

Inertia's picture

Thanks for the link, in their example code they also call alGetError besides alcGetError, something I should probably do aswell. They also do what would have been my next step, using source queueing for seamless playback of the recorded data, but for that recording must work first ;) I'll fix the CaptureSamples method and see if that helps.

Inertia's picture

CaptureSamples was part of the problem, I've updated the initial post to working code (no point having broken code around). The other half of the problem was that the recorded voice is very very faint. There appears no way to amplify the input while recording, however when playing back the buffer one may amplify the volume with either AL.Source( src, ALSourcef.Gain, ...) or AL.Listener( ALListenerf.Gain, ...). I have commented out those lines for amplification in the above code to avoid that it blows someones speakers, might be related to my recording device that the recording is faint, and other devices might record considerably louder.

The correct import for alcCaptureSamples is:

public static extern void CaptureSamples([In] IntPtr device, [In] IntPtr buffer, [In] int samples);

Fiddler, if you don't mind fixing that line and commit it with your next batch of changes it would be appreciated.

the Fiddler's picture

Committed to revision 1818, along with the relevant convenience overloads, e.g:

short[] buffer = new short[4096];
Alc.CaptureSamples(CaptureDevice, buffer, buffer.Length);
Inertia's picture

Mhmm I'm not sure whether abandoning type safety is allowed or a good idea, afaik both Alc.CaptureSamples and AL.BufferData expect a byte[]. Size of a sample depends on whether you're recording with 8 or 16 Bits and Mono or Stereo.

Mono 8Bit - 1 byte / sample
Stereo 8Bit - 2 byte / sample
Mono 16 Bit - 2 byte / sample
Stereo 16 Bit - 4 byte / sample

Since you must pin the buffer manually anyway, IntPtr as parameter did not seem a bad choice. I'm keeping the buffer pinned for the lifetime of the application - due to it's size any type of usage - there is no good reason why it should be moved by the GC.

Another observation I've made is that Alc.GetInteger( AlcGetInteger.CaptureSamples ) always returns a multiple of 1102 when recording with 22050 Hz, which implies that array holding audio data should not be a power of 2, but instead a multiple of 1102. ( 1102 * 20 = 1 second of samples)

P.S: Streaming playback using the AL.SourceQueue/Unqueue commands works perfectly fine with OpenTK.

the Fiddler's picture

Damn, I completely forgot that CaptureSamples is not blocking! The API is no good then... Filed as #914: [Audio] Overloads: Alc.CaptureSamples

Capture functionality should probably receive the same treatment as AudioContext in some future release.

Can you post a short sample for SourceQueue/Unqeueue? Last time I didn't manage to get those working without clicking (although the OpenAL SI may have been at fault there).

Inertia's picture

Mind I'm doing excessive error checking, which is not really needed.

this happens pretty close after capturing the samples from the device into MyBuffer...

bufferIsUsable[CurrentBuffer] = false;
AL.BufferData( buffers[CurrentBuffer], Format, MyBufferPtr, CapturedSamples * SampleToByte, Frequency );
CheckAlcError( "AL.BufferData" );
AL.SourceQueueBuffers( src, 1, ref buffers[CurrentBuffer] );
CheckAlcError( "AL.SourceQueueBuffer" );

this happens unrelated to Queueing...

int BuffersCompletedProcessing;
AL.GetSource( src, ALGetSourcei.BuffersProcessed, out BuffersCompletedProcessing );
CheckAlcError( "AL.GetSource(BuffersProcessed)" );
 
if ( BuffersCompletedProcessing > 0 )
{
  uint[] freedbuffers = new uint[BuffersCompletedProcessing];
  unsafe
  {
    fixed ( uint* ptr = &freedbuffers[0] )
    {
       AL.SourceUnqueueBuffers( src, BuffersCompletedProcessing, ptr );
     }
  }
  CheckAlcError( "AL.SourceUnqueueBuffers" );
 
  for ( int i = 0; i < freedbuffers.Length; i++ )
    for ( int j = 0; j < buffers.Length; j++ )
      if ( freedbuffers[i] == buffers[j] )
      {
        bufferIsUsable[j] = true;
        break;
      }
      else
      {
      if ( j == buffers.Length - 1 )
        MessageBox.Show( "Could not find match between unqueued buffer and allocated buffers." );
      }
}

The situation where the MessageBox is shown should never occur, as long as you don't feed the source quicker with buffers than it can play them back. Instead of the bool[] bufferIsUsable and the for-loops, a Stack or Queue collection could be used to track available buffers for updating with new recorded data. I will most likely do this when the MessageBox never pops up after some more testing on this.

Edit: I think the clicking in your application was related to the buffer size not being a multiple of 1102. IIRC you had it set to 4kb chunks being read from the file.

the Fiddler's picture

Thanks!

Inertia's picture

Since this is the only topic related to capturing audio with OpenAL I'll add observations here rather than creating a new topic or blog (isn't is weird that they chose to remove the "we" from weblog?)

The sources used to queue the buffers may starve and require restarting from time to time (See AL.GetSourceState( src ) and AL.GetSource( src, ALGetSourcei.BuffersQueued,...). Enqueueing more buffers on a stopped source is not a problem, but the source will not automatically restart.

This is always the case when an UDP packet is not delivered, which is the protocol that should be used for voice communication.

The number 1102 appears to be constant for 22KHz 16Bit Mono recording (these settings are chosen by my own perception of the quality of the recording, I'm unable to tell the difference between 22KHz and 44 with this microphone), however from 2 Creative Labs ExtremeGamer cards, one would reliably record chunks of 1102 bytes while the other would sometimes also return chunks which were not a multiple of 1102. Might be a driver issue, investigating.