Prof. Dr. -Ing. Gerald Schuller
Jupyter Notebook: Renato Profeta

Quantization: Signal to Noise Ratio (SNR)¶

Assume we have a A/D converter with a quantizer with a certain number of bits (say N bits), what is the resulting Signal to Noise Ratio (SNR) of this quantizer?

The SNR is defined as the ratio of the expectation of the signal power to the expectation of the noise power.

In our case, the expectation of the noise power is the expectation of the quantization error power. We already have the expectation of the quantization error power as $\large \dfrac{\Delta^2}{12}$.
So what we still need for the SNR is the average or expectation of the signal power. How do we obtain this?
Basically we can take the same approach as we did for the expectation of the power of the quantization error (which is basically the second moment of the distribution of the quantization error). So what we need to know from our signal is its probability distribution. For the quantization error it was a uniform distribution between $-\dfrac{\Delta}{2}$ and $+\dfrac{\Delta}{2}$.
A very simple case would be a uniformly distributed signal with amplitude $\dfrac{A}{2}$, which has values between $-\dfrac{A}{2}$ up to $+\dfrac{A}{2}$.

So we could again use our formula for the average power, but now for our signal x:

$$\large E(x^2)=\int_ {-A/2} ^ {A/2} x^2 \cdot p(x) dx$$

So here we have the same type of signal, and the resulting expectation of the power (its second moment, assumed we have a zero mean signal) is obtained by using our previous formula, and replace $\Delta$ by A. The resulting power is: $\frac{A^2}{12}$.

Which signals have this property? One example is uniformly distributed random values (basically like our quantization error).

Observe: Speech or music has a non-uniform pdf, it is usually modeled by a Laplacian distribution or a gaussian mixture model, so it doesn't apply to this case!

An example for a uniform pdf: a triangular wave:

How do we obtain its pdf? One can imagine the vertical axis (the function value) covered by small intervals, and each interval is then passed in the same time-span. This means that the resulting pdf is also uniform!

A further example: A sawtooth wave:

Again we can make the same argument, each small interval of our function value is covered in the same time-span, hence we obtain a uniform distribution.

We now have seen a few examples which fulfil our assumption of a uniform distribution (realistic examples), and we know: their expectation of their power is $\dfrac{A^2}{12}$. So what does this then mean for the SNR? The SNR is just the ratio:

$$ \large SNR = \frac {\dfrac{A^2}{12}} {\dfrac{\Delta^2}{12}}= \frac{A^2} { \Delta^2} $$

If we assume our signal is full range, meaning the maximum values of our A/D converter is $-\dfrac{A}{2}$ and $+\dfrac{A}{2}$ (the signal goes to the maximum), we can compute the step size $\Delta$ if we know the number of bits of converter, and if we assume uniform quantization step sizes. Assume we have N bits in our converter. This means we have $2^N$ quantization intervals. We obtain $\Delta$ by dividing the full range by this number,

$$ \large \Delta = \frac{A}{2^N} $$

Plug this in the SNR equation, and we obtain:

$$ SNR= \frac{A^2} { \Delta^2}= \frac{A^2} {\left( \dfrac {A}{2^N} \right)^2} = {2^{2N}} $$

This is now quite a simple result! But usually, the SNR is given in dB (deciBel), so lets convert it into dB:

$$SNR_{dB} = 10 \cdot \log_{10} (2^{2N})=10 \cdot 2N \cdot \log_{10}(2) \approx $$ $$ \approx 10 \cdot 2N \cdot 0.301 dB =N \cdot 6.02 dB$$

This is now our famous rule of thumb, that each bit more gives you about 6 dB more SNR. But observe that the above formula only holds for uniformly distributed full range signals! (the signal is between -A/2 and +A/2, using all possible values of our converter).

What happens if the signal is not full range? What is the SNR if we have a signal with reduced range? Assume our signal has an amplitude of A/c, with a factor c>1.
We can then simply plug this into our equation:

$$ \large SNR= \frac{\left(\frac{A}{c}\right)^2} {\Delta^2}= \frac{\left(\frac{A}{c}\right)^2 }{ \left(\frac{A}{2^N}\right)^2} = \frac{2^{2N}}{c^2}$$

in dB:

$$ \large SNR_{dB}=10 . \log_{10}( \frac {2^{2N}} {c^2})=10\cdot2N.\log_{10}(2)-20.\log_{10}(c) \approx $$$$ \large \approx 10 \cdot 2N \cdot 0.301 dB -20 \cdot \log_{10}(c) = $$
$$ \large =N \cdot 6.02 dB -20 \cdot log_{10}(c) $$

The last term, the $20 \cdot \log_{10}(c) $, is the number of dB which we are below our full range. This means we reduce our SNR by this number of dB which we are below full range!

Example: We have a 16 bit quantiser, then the SNR for uniformly distributed full range signals would be:

$$SNR = 6.02 \cdot 16 dB = 96.32 dB$$

Now assume we have the same signal, but 20dB below full range (meaning only 1/10th of the full range). Then the resulting SNR would be only:

$$SNR = 96.32-20 = 76.32 dB$$

This is considerably less. This also shows why it is important not to make the safety margin to full range too big! So for instance our sound engineer should keep the signal as big as possible, without ever reaching full range to avoid clipping the signal.

The other assumption we made concerned the type of signal we quantize.

What if we don't have a uniformly distributed signal?

As we saw, speech and audio signals are best modelled by a Laplacian distribution or a Gaussian mixture model, and similar for audio signals. Even a simple sine wave does not fulfil this assumption of a uniform distribution. What is the pdf of a simple sine wave?

Observe: If a sinusoid represents a full range signal, its values are from $-A/2$ to $+A/2$, as in the previous cases.

What is our SNR if we have a sinusoidal signal? What is its pdf? Basically it is its normalized histogram, such that its integral becomes 1, to obtain a probability distribution.

If we look at the signal, and try to see how probable it is for the signal to be in a certain small interval on the y axis, we see that the signal stays longest around +1 and -1, because there the signal slowly turns around. Hence we would expect a pdf, which has peaks at +1 and -1.
If you calculate the pdf of a sine wave, x=sin(t), with t being continuous and with a range larger than 2pi, then the result is

$$ p(x)=\frac{1} {\pi \cdot \sqrt{1-x^2}} $$
This results from the derivative of the inverse sine function (arcsin). This derivation can be found for instance on Wikipedia. For our pdf we need to know how fast a signal x passes through a given bin in x. This is what we obtain if we compute the inverse function $x=f^{-1}(y)$, and then its derivative $df^{-1}(x)/dy$.

PDF of Time Series¶

Given a signal a = f (t) which is sampled uniformly over a time period T , its PDF, p(a) can be calculated as follows. Because the signal is uniformly sampled we have $p(t) = \frac{1}{T}$ . The function f(t) acts to transform this density from one over t to one over a. Hence, using the method for transforming PDFs, we get:

$$\large p(a)=\dfrac{p(t)}{\left|\frac {da}{dt} \right|} $$

where | | denotes the absolute value and the derivative is evaluated at $t=f^{-1}(x).$

From: https://www.fil.ion.ucl.ac.uk/~wpenny/course/appendixDE.pdf

The inverse sine is only defined for $-\frac{\pi}{2} \leq t \leq +\frac{\pi}{2}$ and p(t) is uniform within this. Hence, $ p(t) = \frac {1}{\pi} $.

Here we can see that p(x) indeed becomes infinite at x=+/-1! We could now use the same approach as before to obtain the expectation of the power, multiplying it with $x^2$ and integrating it. But this seems to be somewhat tedious. But since we now have a deterministic signal, we can also try an alternative solution, since the sine function is not a probabilistic function, but a deterministic function.

We can simply directly compute the power of our sine signal over t, and then take the average over at least one period of the sine function.

$$ \large E(x^2)= \frac{1} {2\pi}\int _ {t=0} ^{2\pi} sin^2(t) dt = \frac{1} {2\pi}\int _ {t=0} ^{2\pi} {\dfrac{\left(1-cos(2t)\right)}{2}}dt $$

Trigonometric Identity: $cos(2x)=1-2sin^2(x)$

The cosine integrated over complete periods becomes 0, hence we get:

$$ n\large =\frac{1} {2\pi} \int _{t=0}^{2\pi} {\dfrac{1}{2}} dt =\frac{1} {2\pi} \cdot \pi=\frac{1}{ 2} $$

What do we get for a sinusoid with a different amplitude, say $A/2 \cdot sin(t)$?

The expected power is:

$$ \large E(x^2)=\frac{A^2}{ 8} $$

So this leads to an SNR of:

$$ \large SNR=\frac{\frac{A^2}{8}} {\frac{\Delta^2}{12}}=\frac{3 \cdot A^2} {2 \cdot \Delta^2} $$

Now assume again we have a A/D converter with N bits, and the sinusoid is at full range for this converter. Then $A=2^N \cdot \Delta$

We can plug in this result into the above equation, and get

$$ \large SNR=\frac{3 \cdot 2^{2N} \cdot \Delta^2} {2 \cdot \Delta^2}={1.5 \cdot 2^{2N}} $$

In dB this will now be:

$$\large 10 \cdot \log_{10}(SNR)=10 \cdot \log_{10}(1.5) + N \cdot 20 \cdot \log_{10}(2)= $$$$\large = 1.76 dB +N \cdot 6.02 dB$$

Here we can see now, that using a sinusoidal signal instead of a uniformly distributed signal gives us a boost of 1.76 dB in SNR. This is because it is more likely to have larger values! We see that our rule of 6dB more SNR for each bit still holds!