Import¶

Import the package and some utils functions

Define a path object, it manages the folders directions of results, auxiliar data, audios, and birdsongs paths; and ploter object, to visualize syllables. The path object look for all the wav files in the audio file, located at root_path/Audios.

You must fill the root path according where you clone the repositry and your Operative System.

Objects Definition¶

Song¶

Choose what wav audio you want to import, no_file. The song object is define from the path file name and the number of file choosed, the path of the song is stored in the paths object

define the song by the file number

define the song by a time interval, interval of interest from a complete song

Plot¶

Visualize the song with the ploter object use ploter function Plot and enter the object you want to visualize

The syllable extraction has three methods:

• amplitud:

find where the normalized audio amplitud crosses an umbral, 0.05.

• freq:

find where the fundamental frequency changes drastically, where the changes is more than 500 Hz.

• maad:

use the maad segmentation tools (from scitik-maad) to find Region Of Interest (not implemented yet).

Syllable¶

This is the kernel of the model, the Syllable object. This object extract the syllable tempo-spectral features and define the neccesary variables to implement and solve the Motor Gesture. It returns a synthetic syllable as syllable object.

To solve the syllable, find its synthetic syllable from some parameters ($\alpha, \beta, \gamma$), use its method Solve. This mehotd dependes of the model parameters $p$, defined in each syllable object, which are $\alpha_i, \beta_i, $ and $\gamma$. Although it is possible to define cutom parameters set, the syllable object has an initial set predefined (it is in the factible region).

To display the parametres set use the command Display(object.p). To change one of them values use syllable_object.p["a0"].set(value=value_a0).

Definition¶

Defining with the Song object. Here is used the bird syllables divider, it requires the syllable number of interest

define with the Syllable object, ir defines a syllable from an audio bird. You can optionally give a time range for clip the bird audio

Solution¶

Synthetic Plots¶

Syllable variables are visualize with ploter object, here also is measured the time exectuion of the Motor Gesture definition and solution.

One of the biggest advantages of model implementation is the easily parameters exploration, as an example let's use the previous syllable defined but varying the input saic pressure in three levels: low, medium and high.

To plot each object, song or syllable, use the command ploter.Plot(obj).

There is also the posibility to define a syllable object from a time interval and the bird object, nevertheless, you have to define some attribues to the object in order to ploter works fine. Avoid to enter wrong time limits, the object will not be defined.

To improve the syllable frequency resolution, modify the Short Time Fourier Transform window length but remeber that better frequency resolution implies loss time resolution. This feature is useful when the syllable spectrum is "complex" (trilled syllables).

Varying Parameters¶

Low¶

$ a_0 = 0.01 $

Medium¶

$ a_0 = 0.11 $

High¶

$ a_0 = 1.25 $

Chunck¶

Although the sylalble division is a good approximation to solve the problem, a better methodology is take chuncks of a syllable, divide the syllable in fractions. Since the object syllable is already defined, is worth to use it again to define the chunck object.

The biggest difference with the syllable object is the Fourier Transform window length and envelope parameters. Since these chuncks are smaller than the syllabes the space parameters curves is also smaller.

Definition¶

Choose what fraction of the syllable are you interested, no_chunck, and define it as a syllable object

Solution¶

Show the parameters used and solve generate the synthetic chunck

Plot Results¶

Visualize the chunck tempo-spectral features and the scored variables, scored defined to compare real and synthetic syllables

Plot All Objects¶

Visualize again the song but with the chunck and syllables objects also plotted. The syllable and chunck must be previously defined

Optimization Problem¶

General Problem¶

\begin{equation}\label{opt_general} \begin{aligned} \underset{ \gamma \in \mathbb{R},\; \alpha,\beta\in \mathbb{R}^n}{\text{min}} &\qquad ||\hat{SCI}_{real} - \hat{SCI}_{synt} ( \gamma,\alpha,\beta)||_2 + || (\hat{FF}_{real} - \hat{FF}_{synt}(\gamma,\alpha,\beta)||_2 \\ & \qquad \qquad - corr(FC_{real},FC_{synt}(\gamma, \alpha, \beta)) \\ \text { subject to } & \qquad \gamma \in \Omega_\gamma, \quad \beta \in \Omega_\beta , \quad \alpha \in \Omega_\alpha \end{aligned} \end{equation}

with $\Omega_\gamma , \Omega_\alpha$, and $\Omega_\beta$ the known feasible regions for each variable. In order to get an objective function adimensionless the following two variables are define

$$ \hat{SCI} := \frac{SCI}{dim(SCI)} , \qquad \hat{FF} := \frac{1}{dim(FF)} \frac{FF}{1 \; KHz} $$

where $dim()$ is the dimension the corresponding vector.

Sub-Optimization Problems¶

The general problem is computationally expensive since depends on many variables. Although solve the problem of one shot is the ideal method, a better approach is split the general problem in three auxiliar problems

Optimal $\gamma$¶

\begin{equation}\label{eq_optimal_gamma} \begin{aligned} \underset{ \gamma \in \mathbb{R}}{\text{min}} &\qquad || \hat{SCI}_{real} - \hat{SCI}_{synt} ( \gamma)||_2 + || \hat{FF}_{real} - \hat{FF}_{synt}(\gamma)||_2\\ \text { subject to } & \qquad \; \gamma \in \Gamma_\gamma = [10000, 100000] \end{aligned} \end{equation}

Optimal $\alpha$ Coeficients¶

The coefficients of $\alpha$ are calculated with the spectrum coeffcients correlation between the real syllable and the synthetic one

\begin{equation}\label{optimal_a_min} \begin{aligned} \underset{a \in \mathbb{R}^3}{\text{min}} & \qquad - corr (real, synthetic(a)) \\ \text { subject to } & a\in\Omega_a \end{aligned} \end{equation}

Optimal $\beta$ Coeficients¶

The last step is to find the beta coefficients $b_i$

\begin{equation}\label{optimal_b} \begin{aligned} \underset{b \in \mathbb{R}^3}{\text{min}} &\qquad || FF_{real} - FF_{synt} (b)|| \\ \text { subject to } & \qquad \; b \in \Omega_b \end{aligned} \end{equation}

with $t\in [0,T]$ where $T$ is the duration of the sillable (chunck).

The air-sac pressure and labial tension are defined by six coefficients, 3 coefficients each one, this means their time curves are parabolic functions (the motor gestures are parabolas) but this can be modify but ommiting the third coefficients $a_2, b_2$ and working with lines curves as a motor gestures.

Optimization Solvers¶

Define the method and its parameters to solve the optimization problem

Define the object to optimize and its corresponding optimizer

You can check all the methods availables uncommenting the following line, check the method attribute

Initial Synthetic Syllable¶

Show model parameters and plots of the inital synthetic syllable

Solution¶

Optimal $\gamma$¶

Find the optimal time constant parameter ($\gamma^*$) by solving the first suboptimization problem for each syllable.

Altough the optimal parameters are stores in the optimizer, it is recommend to save this value in the object parameters set

Plot¶

Show parameter set, solve the object with this parameters, and visualize synthetic syllable

Optimal Parameters $\alpha_i$ and $\beta_i$¶

Dependeing of what gesture approximation are you interested (linear or quadratic curves for $\alpha$ or $\beta$), the optimizer object find the optimal parameteres using its OptimalVariable method

Optimal Parameters¶

One shot solving is also implemented but the execution is very slow, since the parameter space has a dimension of at least 5 parameters.

Finding optimal $\gamma$, $b_0$, and $b_1$ by the brute method

Plot Best Syllable¶

Solve and visualize the optimal synthetic syllable and its features

Write Audio¶

The final step is write the audio. To export the syllable in audio format use the syllable method WriteAudio

Times¶

Whole Song¶

This function attends to solve all the song, it calculates the optimal gamma and find the optimal parameters for each syllable