Music and technology are blending together harmoniously. Just ask T-Pain. With the public pre-release of Flash 10.1, Adobe has made available technology that gives Actionscript 3 programmers access to Microphone/Line-in data! In theory, this is the same kind of access that the ingenious producers and audio techies exploit to "vocode" their way to the top of the charts. Granted the Flash player probably isn't quite in the realm of advanced digital signal processing (DSP) but that is not quite where my interest lies. As a humble guitarist/programmer, I would love to have a true Flash guitar tuner. Thanks to Flash 10.1, I can make one.
The folks at http://www.getmicrophone.com/ felt that opening the Flash player to microphone access was important enough to dedicate a whole site to this feature request. Now that it's possible, let's take a look at how we can grab the ByteArray data with the microphone's shiny new SampleDataEvent.SAMPLE_DATA.
mic = Microphone.getMicrophone(0);
mic.rate = 44;
if (mic.muted)
{
stopCapture();
}
else
{
startCapture();
}
private function startCapture():void
{
mic.removeEventListener(StatusEvent.STATUS, mic_StatusChanged);
mic.addEventListener(SampleDataEvent.SAMPLE_DATA, mic_SampleData);
if(contains(securityButton))
removeChild(securityButton);
addChild(spectrumViewer);
}
private function stopCapture():void
{
mic.addEventListener(StatusEvent.STATUS, mic_StatusChanged);
mic.removeEventListener(SampleDataEvent.SAMPLE_DATA, mic_SampleData);
if (contains(spectrumViewer))
removeChild(spectrumViewer);
addChild(securityButton);
}
.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, "Courier New", courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }
Once we have permission to access the microphone and the listener is added, the SAMPLE_DATA event will start firing. The SampleDataEvent contains a byte array of a time-domain spectrum with which we will read the float values and populate an Array.
private function mic_SampleData(e:SampleDataEvent):void
{
var data:Array = new Array();
for (var i:int = 0; i < e.data.length && e.data.bytesAvailable; i++)
{
var sample:Number = e.data.readFloat();
data.push(sample);
}
}
.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, "Courier New", courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }
In order make sense of this data, say to evaluate the accuracy of the pitch of a guitar string, we need to apply an efficient algorithm to determine the the frequency of the incoming microphone signal. http://cnx.org/content/m11714/latest/ provides a great explanation of different signal processing algorithms, but in short, we are looking at 2 major options.
- Apply an Auto Correlation algorithm that compares a window of the signal to itself shifted on a given interval.
- Apply a Fast Fourier Transform to convert the time-domain signal to a frequency domain signal.
My first stab at pitch detection will use the Fast Fourier Transform. I found a very fast AS3 Fast Fourier Transform using http://www.koders.com, which can be found here http://www.koders.com/actionscript/fid30195A744AE561A65738EB00F7647BD35182F169.aspx?s=FFT#L8. After applying this algorithm to the Array as follows, we can draw out our spectrum to get a visual of the incoming microphone signal.
private function mic_SampleData(e:SampleDataEvent):void
{
var data:Array = new Array();
for (var i:int = 0; i < e.data.length && e.data.bytesAvailable; i++)
{
var sample:Number = e.data.readFloat();
data.push(sample);
}
try
{
FFT.transform(data);
spectrumViewer.data = data;
}
catch (e:Error)
{
}
}
.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, "Courier New", courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }
In the SpectrumViewer.as class, we have a simple loop on the Array to draw our signal.
var stepWidth:Number = _explicitWidth / (_data.length / 2);
signal.graphics.clear();
signal.graphics.lineStyle(1, 0xFF0000, 1);
for (var i:int = 0; i < _data.length / 2; i++)
{
//signal
var newY:Number = yPos - Math.abs(_data[i]);
signal.graphics.lineTo(xPos, newY);
xPos += stepWidth;
}
.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, "Courier New", courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }
Now it's time to test while playing some music into the mic. Here's our frequency-domain signal in flash!
