In one of our last projects we were dealing with audio: capture audio in browser, store it on server and then return it by a request and replay in browser.

Though an audio capturing is by itself rather interesting and challenging task, it's addressed by HTML5, so for example take a look at this article. Here we share our findings about other problem, namely an audio conversion.

You might thought that if you have already captured an audio in browser then you will be able to play back it. Thus no additional audio conversion is required.

In practice we are limited by support of various audio formats in browsers. Browsers can capture audio in WAV format, but this format is rather heavy for storing and streaming back. Moreover, not all browsers support this format for playback. See wikipedia for details. There are only two audio formats that more or less widely supported by mainstream browsers: MP3 and AAC. So, you have either convert WAV to MP3, or to AAC.

The obvious choice is to select WAV to MP3 conversion, the benefit that there are many libraries and APIs for such conversion. But in this case you risk falling into a trap with MP3 licensing, especially if you deal with iteractive software products.

Eventually, you will come to the only possible solution (at least for moment of writting) - conversion WAV to AAC.

The native solution is to use NAudio library, which behind the scene uses Media Foundation Transforms. You'll shortly get a working example. Actually the core of solution will contain few lines only:

var source = Path.Combine(root, "audio.wav");
var target = Path.Combine(root, "audio.m4a");

using(var reader = new NAudio.Wave.WaveFileReader(source))
{
  MediaFoundationEncoder.EncodeToAac(reader, target);
}

Everything is great. You'll deploy your code on server (by the way server must be Windows Server 2008R2 or higher) and at this point you may find that your code fails. The problem is that Media Foundation API is not preinstalled as a part of Windows Server installation, and must be installed separately. If you own this server then everything is all right, but in case you use a public web hosting server then you won't have ability to install Media Foundation API and your application will never work properly. That's what happened to us...

After some research we came to conclusion that another possible solution is a wrapper around an open source video/audio converter - FFPEG. There were two issues with this solution:

• how to execute ffmpeg.exe on server asynchronously;
• how to limit maximum parallel requests to conversion service.

All these issues were successfully resolved in our prototype conversion service that you may see here, with source published on github. The solution is Web API based REST service with simple client that uploads audio files using AJAX requests to server and plays it back. As a bonus this solution allows us perform not only WAV to AAC conversions, but from many others format to AAC without additional efforts.

Let's take a close look at crucial details of this solution. The core is FFMpegWrapper class that allows to run ffmpeg.exe asynchronously:

/// <summary>
/// A ffmpeg.exe open source utility wrapper.
/// </summary>
public class FFMpegWrapper
{
  /// <summary>
  /// Creates a wrapper for ffmpeg utility.
  /// </summary>
  /// <param name="ffmpegexe">a real path to ffmpeg.exe</param>
  public FFMpegWrapper(string ffmpegexe)
  {
    if (!string.IsNullOrEmpty(ffmpegexe) && File.Exists(ffmpegexe))
    {
      this.ffmpegexe = ffmpegexe;
    }
  }

  /// <summary>
  /// Runs ffmpeg asynchronously.
  /// </summary>
  /// <param name="args">determines command line arguments for ffmpeg.exe</param>
  /// <returns>
  /// asynchronous result with ProcessResults instance that contains 
  /// stdout, stderr and process exit code.
  /// </returns>
  public Task<ProcessResults> Run(string args)
  {
    if (string.IsNullOrEmpty(ffmpegexe))
    {
      throw new InvalidOperationException("Cannot find FFMPEG.exe");
    }

    //create a process info object so we can run our app
    var info = new ProcessStartInfo 
    { 
      FileName = ffmpegexe,
      Arguments = args,
      CreateNoWindow = true
    };

    return ProcessEx.RunAsync(info);
  }

  private string ffmpegexe;
}

It became possible to run  a process asynchronously thanks to James Manning and his ProcessEx class.  

Another useful part is a semaphore declaration in Global.asax.cs:

   public class WebApiApplication : HttpApplication
  {
    protected void Application_Start()
    {
      GlobalConfiguration.Configure(WebApiConfig.Register);
    }

    /// <summary>
    /// Gets application level semaphore that controls number of running 
    /// in parallel FFMPEG utilities.
    /// </summary>
    public static SemaphoreSlim Semaphore
    {
      get { return semaphore; }
    }

    private static SemaphoreSlim semaphore;

    static WebApiApplication()
    {
      var value =
        ConfigurationManager.AppSettings["NumberOfConcurentFFMpegProcesses"];

      int intValue = 10;

      if (!string.IsNullOrEmpty(value))
      {
        try
        {
          intValue = System.Convert.ToInt32(value);
        }
        catch
        {
          // use the default value
        }
      }

      semaphore = new SemaphoreSlim(intValue, intValue);
    }
  }

And the last piece is the entry point, which was implemented as a REST controller:

/// <summary>
/// A controller to convert audio.
/// </summary>
public class AudioConverterController : ApiController
{
  /// <summary>
  /// Gets ffmpeg utility wrapper.
  /// </summary>
  public FFMpegWrapper FFMpeg
  {
    get
    {
      if (ffmpeg == null)
      {
        ffmpeg = new FFMpegWrapper(
          HttpContext.Current.Server.MapPath("~/lib/ffmpeg.exe"));
      }

      return ffmpeg;
    }
  }

  /// <summary>
  /// Converts an audio in WAV, OGG, MP3 or other formats 
  /// to AAC format (MP4 audio).
  /// </summary>
  /// <returns>A data URI as a  string.</returns>
  [HttpPost]
  public async Task<string> ConvertAudio([FromBody]string audio)
  {
    if (string.IsNullOrEmpty(audio))
    {
      throw new ArgumentException(
        "Invalid audio stream (probably the input audio is too big).");
    }

    var tmp = Path.GetTempFileName();
    var root = tmp + ".dir";

    Directory.CreateDirectory(root);
    File.Delete(tmp);

    try
    {
      var start = audio.IndexOf(':');
      var end = audio.IndexOf(';');
      var mimeType = audio.Substring(start + 1, end - start - 1);
      var ext = mimeType.Substring(mimeType.IndexOf('/') + 1);
      var source = Path.Combine(root, "audio." + ext);
      var target = Path.Combine(root, "audio.m4a");

      await WriteToFileAsync(audio, source);

      switch (ext)
      {
        case "mpeg":
        case "mp3":
        case "wav":
        case "wma":
        case "ogg":
        case "3gp":
        case "amr":
        case "aif":
        case "mid":
        case "au":
        {
          await WebApiApplication.Semaphore.WaitAsync();

          var result = await FFMpeg.Run(
            string.Format(
              "-i {0} -c:a libvo_aacenc -b:a 96k {1}",
              source,
              target));

          WebApiApplication.Semaphore.Release();

          if (result.Process.ExitCode != 0)
          {
            throw new InvalidDataException(
              "Cannot convert this audio file to audio/mp4.");
          }

          break;
        }
        default:
        {
          throw new InvalidDataException(
            "Mime type: '" + mimeType + "' is not supported.");
        }
      }

      var buffer = await ReadAllBytes(target);
      var response = "data:audio/mp4;base64," + System.Convert.ToBase64String(buffer);

      return response;
    }
    finally
    {
      Directory.Delete(root, true);
    }
  }

For those who'd like to read more about audio conversion, we may suggest to read this article.