Sine model Analysis/Synthesis - MusicBricks Tutorial¶

Introduction¶

This tutorial will guide you through some tools for performing spectral analysis and synthesis using the Essentia library (http://www.essentia.upf.edu).

This algorithm shows how to analyze the input signal, and resynthesize it again, allowing to apply new transformations directly on the spectral domain.

You should first install the Essentia library with Python bindings. Installation instructions are detailed here: http://essentia.upf.edu/documentation/installing.html .

Processing steps¶

In [1]:

# import essentia in streaming mode
import essentia
import essentia.streaming as es

After importing Essentia library, let's import other numerical and plotting tools

In [2]:

# import matplotlib for plotting
import matplotlib.pyplot as plt
import numpy as np

Define the parameters of the STFT workflow

In [3]:

# algorithm parameters
params = { 'frameSize': 2048, 'hopSize': 512, 'startFromZero': False, 'sampleRate': 44100, \
          'maxnSines': 100,'magnitudeThreshold': -74,'minSineDur': 0.02,'freqDevOffset': 10, \
          'freqDevSlope': 0.001}

Specify input and output audio filenames

In [4]:

inputFilename = 'singing-female.wav'
outputFilename = 'singing-female-sindemodel.wav'

In [5]:

# create an audio loader and import audio file
out = np.array(0)
loader = es.MonoLoader(filename = inputFilename, sampleRate = 44100)
pool = essentia.Pool()

Define algorithm chain for frame-by-frame process: FrameCutter -> Windowing -> FFT -> IFFT -> OverlapAdd -> AudioWriter

In [6]:

# algorithm instantation
fcut = es.FrameCutter(frameSize = params['frameSize'], hopSize = params['hopSize'], startFromZero =  False);
w = es.Windowing(type = "blackmanharris92");
fft = es.FFT(size = params['frameSize']);
smanal = es.SineModelAnal(sampleRate = params['sampleRate'], maxnSines = params['maxnSines'], magnitudeThreshold = params['magnitudeThreshold'], freqDevOffset = params['freqDevOffset'], freqDevSlope = params['freqDevSlope'])
smsyn = es.SineModelSynth(sampleRate = params['sampleRate'], fftSize = params['frameSize'], hopSize = params['hopSize'])
ifft = es.IFFT(size = params['frameSize']);
overl = es.OverlapAdd (frameSize = params['frameSize'], hopSize = params['hopSize'], gain = 1./params['frameSize'] );
awrite = es.MonoWriter (filename = outputFilename, sampleRate =  params['sampleRate']);

Now we set the algorithm network and store the processed audio samples in the output file

In [7]:

# analysis
loader.audio >> fcut.signal
fcut.frame >> w.frame
w.frame >> fft.frame
fft.fft >> smanal.fft
# synthesis
smanal.magnitudes >> smsyn.magnitudes
smanal.frequencies >> smsyn.frequencies
smanal.phases >> smsyn.phases
smsyn.fft >> ifft.fft
ifft.frame >> overl.frame
overl.signal >> awrite.audio
overl.signal >> (pool, 'audio')

Finally we run the process that will store an output file in a WAV file

In [8]:

essentia.run(loader)