Visualizing Audio

From ActiveArchives

Jump to: navigation, search

This example is best viewed in the latest version of Firefox. Example audio from LibriVox: Les Chants de Maldoror read by Raphaël Badawi. View the result here.

Contents

Download the audio, convert to OGG

Download the source MP3: 

wget http://upload.librivox.org/share/uploads/ez/leschantsdemaldoror_2_01_lautreamont.mp3
mv leschantsdemaldoror_2_01_lautreamont.mp3 maldoror201.mp3

Convert it to OGG/Vorbis: 

ffmpeg -i maldoror201.mp3 -ab 128k -acodec libvorbis maldoror201.ogg

Look at the waveform

To get a view of the audio levels, we can open the file in Audacity.

Maldoror201 audacity.png

Extract Audio Levels Data

Python + Gstreamer can be used to dump "levels" data from an audio file as a big list of numbers.

Source: Audio_levels.py 

 python audio_levels.py maldoror201.ogg

output: 

-48.8847458619
-37.1889925385
-26.2443369875
-23.916325213
-14.6759872226
...

To use this data in a web page, we'll convert it to a JSON-encoded list of floats.

Source: Wrapfloats.py 

python audio_levels.py maldoror201.ogg | python wrapfloats.py

outputs: 

[-48.8847458619, -37.1889925385, -26.2443369875, -23.916325213, ...

Now add a "save to file" at the end of the pipeline: 

python audio_levels.py maldoror201.ogg | python wrapfloats.py > maldoror201.json

Load the data in a webpage

Here, using JQuery, NB: console.log may require you to use the Firebug Firefox extension. 

<!DOCTYPE html>
<html lang="fr">
<head>
<meta charset="utf-8">
<title></title>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js" charset="utf-8"></script>
<script>
$(document).ready(function () {
    var dmin, dmax;
    $.getJSON("maldoror201.json", function (data) {
        // scan for min and max
        for (var i=0; i<data.length; i++) {
            if ((dmin === undefined) || (data[i] < dmin)) { dmin = data[i]; }
            if ((dmax === undefined) || (data[i] > dmax)) { dmax = data[i]; }
        }
        console.log("dmin", dmin, "dmax", dmax);
    });
});
</script>
</head>
<body>
</body>
</html>

Displays (in the browser console): 

dmin -48.8847458619 dmax -7.11560330356

Draw the data with a canvas 

<!DOCTYPE html>
<html lang="fr">
<head>
<meta charset="utf-8">
<title></title>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js" charset="utf-8"></script>
<script>
$(document).ready(function () {
    var dmin, dmax;
    $.getJSON("maldoror201.json", function (data) {
        // scan for min and max
        for (var i=0; i<data.length; i++) {
            if ((dmin === undefined) || (data[i] < dmin)) { dmin = data[i]; }
            if ((dmax === undefined) || (data[i] > dmax)) { dmax = data[i]; }
        }
        var cx = $("#c").get(0).getContext("2d");
        cx.strokeStyle="#000";
        var canvas_width = $("#c").width();
        var canvas_height = $("#c").height();
        for (var x=0; x<canvas_width; x++) {
            var p = (data[x] - dmin) / (dmax - dmin);
            var h = (canvas_height * p);
            cx.moveTo(x,0);
            cx.lineTo(x, Math.round(h));
            cx.stroke();
        }
    });
});
</script>
</head>
<body>
<canvas id="c" width="640" height="40"></canvas>
</body>
</html>

This results in something like:

Maldoror201 01.png 

        var midy = canvas_height / 2;
        for (var x=0; x<canvas_width; x++) {
            var p = (data[x] - dmin) / (dmax - dmin);
            var h = (canvas_height * p) / 2;
            cx.moveTo(x, midy - h);
            cx.lineTo(x, midy + h);
            cx.stroke();
        }

Maldoror201 01 c.png

This code simply maps the x position to the data position (ie the first pixel is the first data value, the second the second, and so on). This means that when there's more data than pixels, as is the case here, only the first part of the data is shown. To show the whole picture, we can better translate the x position to the best data position: 

        for (var x=0; x<canvas_width; x++) {
            var data_index = Math.floor((x/canvas_width)*data.length);
            var p = (data[data_index] - dmin) / (dmax - dmin);
 
        }

Which looks like:

Maldoror201 02.png

To check if this data is even remotely related to audio, it would be very handy to listen to the audio and look for a relation. We add the original recording using an audio tag, and have it autoplay. To really check the connection, we connect a callback on the audio "timeupdate" event to display a cursor showing the current position. 

<!DOCTYPE html>
<html lang="fr">
<head>
<meta charset="utf-8">
<title></title>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js" charset="utf-8"></script>
<script>
$(document).ready(function () {
    var dmin, dmax;
    $.getJSON("maldoror201.json", function (data) {
        // scan for min and max
        for (var i=0; i<data.length; i++) {
            if ((dmin === undefined) || (data[i] < dmin)) { dmin = data[i]; }
            if ((dmax === undefined) || (data[i] > dmax)) { dmax = data[i]; }
        }
        // init the canvas
        var cx = $("#c").get(0).getContext("2d");
        cx.strokeStyle="#000";
        var canvas_width = $("#c").width();
        var canvas_height = $("#c").height();
        var midy = canvas_height / 2;
        for (var x=0; x<canvas_width; x++) {
            var data_index = Math.floor((x / canvas_width) * data.length);
            var p = (data[data_index] - dmin) / (dmax - dmin);
            var h = (canvas_height * p) / 2;
            cx.moveTo(x, midy - h);
            cx.lineTo(x, midy + h);
            cx.stroke();
        }
        // setup the audio callback
        cx.beginPath();
        cx.strokeStyle="#F00";
        var audio = document.getElementById("a");
        $(audio).bind("timeupdate", function () {
            // console.log(audio.currentTime, audio.duration);
            if (audio.duration) {
                var x = (audio.currentTime / audio.duration) * canvas_width;
                cx.moveTo(x,0);
                cx.lineTo(x, 5);
                cx.stroke();
            }
        });
        // enable clicking on the canvas to set audio.currentTime
        $("#c").click(function (e) {
            var x = e.pageX - this.offsetLeft;
            if (audio.duration) {
                audio.currentTime = (x/canvas_width) * audio.duration;
            }
        });
    });
});
</script>
</head>
<body>
<canvas id="c" width="640" height="120"></canvas><br />
<audio id="a" src="maldoror201.ogg" controls autoplay></audio>
</body>
</html>

Maldoror201 03.png

See the result live

Retrieved from "http://www.activearchives.org/wiki/Visualizing_Audio"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox