This notebook demonstrates how to compute KPIs with xlte. It is based on kpidemo program from that repository.
As outlined in xlte/kpi.py module documentation the pipeline to compute a KPI is as follows:
─────────────
│ Measurement │ Measurements ──────────────── ──────
│ │ ─────────────→ │ MeasurementLog │ ──→ │ Calc │ ──→ KPI
│ driver │ ──────────────── ──────
─────────────
Below we will organize this pipeline and execute it step by step.
Step 1.
Setup driver to provide measurements data.
Such a driver is specific to eNB implementation. Below we use amari.kpi.LogMeasure that is
specific to Amarisoft and retrieves measurements data from enb.xlog . The enb.xlog itself was produced by amari.xlog module via running xamari xlog ws://localhost:9001 stats/15s
command on the eNodeB.
from xlte.amari import kpi as akpi
from urllib.request import urlopen
xlog_uri = "https://lab.nexedi.com/kirr/misc/raw/162307b9/lte/20221211-overload.xlog"
fxlog = urlopen(xlog_uri)
alogm = akpi.LogMeasure(fxlog, # amari.kpi.LogMeasure converts enb.xlog into kpi.Measurements
open('/dev/null', 'r'))
Step 2. Setup kpi.MeasurementLog and load measurements data into it.
The data, as contained in the measurement log, is kept there in the form
of kpi.Measurement, which is driver-independent representation for
KPI-related measurement data.
from xlte import kpi
def load_measurements(alogm: akpi.LogMeasure) -> kpi.MeasurementLog:
mlog = kpi.MeasurementLog()
try:
while 1:
m = alogm.read()
if m is None:
break
mlog.append(m)
finally:
alogm.close()
return mlog
mlog = load_measurements(alogm)
Step 3. Compute E-RAB Accessibility KPI over MeasurementLog with specified granularity period.
We partition entries in the measurement log by specified time period and further use kpi.Calc to compute the KPI over each period.
import numpy as np
from datetime import datetime
# calc_each_period partitions mlog data into periods and yields kpi.Calc for each period.
def calc_each_period(mlog: kpi.MeasurementLog, tperiod: float): # -> yield kpi.Calc
τ = mlog.data()[0]['X.Tstart']
for m in mlog.data()[1:]:
τ_ = m['X.Tstart']
if (τ_ - τ) >= tperiod:
calc = kpi.Calc(mlog, τ, τ+tperiod)
τ = calc.τ_hi
yield calc
tperiod = 1*60 # 1 minute
vτ = []
vInititialEPSBEstabSR = []
vAddedEPSBEstabSR = []
for calc in calc_each_period(mlog, tperiod):
vτ.append(calc.τ_lo)
_ = calc.erab_accessibility() # E-RAB Accessibility
vInititialEPSBEstabSR.append(_[0])
vAddedEPSBEstabSR .append(_[1])
vτ = np.asarray([datetime.fromtimestamp(_) for _ in vτ])
vInititialEPSBEstabSR = np.asarray(vInititialEPSBEstabSR)
vAddedEPSBEstabSR = np.asarray(vAddedEPSBEstabSR)
Step 4. Plot computed KPI.
The E-RAB Accessibility KPI has two parts: initial E-RAB establishment
success rate, and additional E-RAB establishment success rate. kpi.Calc
provides both of them in the form of their confidence intervals. The
lower margin of the confidence interval coincides with 3GPP definition of
the KPI. The upper margin, however, provides information of how
confident, or how unsure we are about described value. For example if
there is enough data to compute the KPI precisely during particular
period, the low and high margins of the confidence interval will be the
same. However if, during a period, there is no measurements data at all,
the confidence interval will be [0,100] meaning full uncertainty - because
there is no measurements data we don't know how accessible eNB was during
that period of time. The width of a confidence interval is not
necessarily 100. For example if during a period, there is no measurement
data only for part of that period, the KPI value is computed from the
other times in the period when there is data, and the confidence interval
will be thinner.
For each of the parts we plot both its lower margin and the whole confidence interval area.
from xlte.demo import kpidemo
import matplotlib.pyplot as plt
kpidemo.figplot_erab_accessibility(plt.gcf(), vτ, vInititialEPSBEstabSR, vAddedEPSBEstabSR, tperiod)
The plot corresponds to scenario with 3 parts:
the first part is "silence" - when there is no mobiles around and so the base station is not being used by UEs at all. As the base station is not being used we don't know the probablity of successfully establishing an E-RAB. This is reflected by [0,100] confidence interval that means "full uncertainty".
the second period is when 5 UE start to attach and detach to eNB constantly. Each mobile attaches for 20 seconds and does a ping test, then goes to power_off for 5 seconds. The base station can handle that just ok with all E-RAB establishment requests succeeding. This is reflected by InititialEPSBEstabSR=100%.
The third period is when 100 UE are around the base station. They all try to do the same power_on+ping+power_off test from "2", but since the number of UE is high, the base station gets overloaded and cannot attach them all. This is reflected by decreased InititialEPSBEstabSR.
In this test there was no additional E-RAB established, so AddedEPSBEstabSR is full uncertainty all the time.
This completes the overview of KPI computation pipeline.
Let's now look at E-UTRAN IP Throughput KPI.
To showcase this KPI we use the same computation pipeline as explained above and enb.xlog from another test. This enb.xlog was produces by running xamari xlog ws://localhost:9001 stats/3s x.drb_stats/3s
and includes information about flows of data on radio bearers.
We load the data as explained above in steps 1 and 2,
xlog_uri = "https://lab.nexedi.com/kirr/misc/raw/1a88e053/lte/20230307-thpdemo.xlog"
fxlog = urlopen(xlog_uri)
alogm = akpi.LogMeasure(fxlog, open('/dev/null', 'r'))
mlog = load_measurements(alogm)
compute the KPI similarly to how it was explained above in step 3,
tperiod = 3 # 3 seconds
vτ = []
vIPThp_qci = []
for calc in calc_each_period(mlog, tperiod):
vτ.append(calc.τ_lo)
_ = calc.eutran_ip_throughput() # E-UTRAN IP Throughput
vIPThp_qci.append(_)
vτ = np.asarray([datetime.fromtimestamp(_) for _ in vτ])
vIPThp_qci = np.asarray(vIPThp_qci)
and plot the result similarly to step 4.
fig = plt.gcf()
fig.set_size_inches(10, 8)
kpidemo.figplot_eutran_ip_throughput(fig, vτ, vIPThp_qci, tperiod)
The plot corresponds to throughput test experiment.
Here it is 5MHz FDD cell with 2x2 MIMO. Ideal downlink throughput is around 48 Mbit/s and ideal uplink throughput is around 18 MBit/s. In the test DL MCS was around 26 - this results in DL radio throughput around 41 Mbit/s (256QAM). In the test UL MCS was around 23-25 - this results in UL radio throughput around 10.5 Mbit/s (64QAM).
The test consists of the following parts:
The graphs display uncertainty window for measured downlink and uplink throughput on non-silent QCIs. The uncertainties are relateively big due to the way how throughput is currently measured with heuristics and estimates in xlte/amari/drb.py .
END