Activity 7 - Violin Plots and Parallel Coordinates¶

This notebook will demonstrate the use of Violin Plots and Parallel Coordinates. This visualisation technique, similar to a box plot, is well suited for comparing multiple distributions, and show a curved distribution plot for each feature under consideration. Below shows a simple violin plot example using Seaborn.

Case Study: Comparing Benign and DDoS behaviours in network traffic analysis using CICIDS2017¶

Let's consider an example where we want to look at benign and malicious network traffic. We will use to CICIDS2017 dataset for this. This dataset captures in the region of 80 numerical features that characterise network activity. Each data instance has been labelled as either benign or as an attack type (we focus on DDoS here, however other attacks are present in the full dataset). What data attributes set these two classes apart? We can use violin plots to judge this visually over the entire dataset.

Load in dataset and clean it up¶

First we will load in the data set, and we will remove all Not-a-Number and Infinity values that may be present. We will also remove columns that contain only zeros (i.e., no separating features).

Normalise each column¶

Each column has its own range of values - some are quite narrow, some are quite large. We often normalise data to make it easier to work with and draw comparisons - this essentially means scaling it to be within a fixed range. Here, we want to normalise each feature indepedently - essentially meaning that each column will have a minimum value of zero and a maximum value of one, and all values for that particular feature will be scaled within this range.

We will use the sci-kit learn library to achieve this.

Separate data based on class¶

We have scaled the entire dataset so that all data for each feature is scaled in a consistent manner. We now want to split our dataset based on the classes of data that exist. Here, we know we have benign and DDoS classes.

Visualise the output¶

We now have our data split into the classes, so we can use the violin plot for each of our classes independently, and compare the two figures.

Findings¶

Comparing the two charts, we can see that the following features are different across the two classes.

• Flow Duration
• Flow IAT (Mean, Std, Max, Min)
• Fwd IAT (Total, Mean, Std, Max, Min)
• Packet Length Variance
• Idle (Mean, Std, Max, Min)

We now have a clearer view (as far as this dataset is concerned) with what makes for a benign packet, and what makes for a malicious DDoS packet.

Extra: PCA decomposition to separate classes¶

Given the high dimensionality of the data, what does the data look like if we perform dimensionality reduction? Can we better separate between the two classes?

*Unfortunately not* - this is not a great surprise, our violin plot shows overlap between the features of the two classes and there is no clear decision boundary that separates the two. PCA is quite poor when there is little variance in many features (as we have here) hence why the plot has artefacts where straight lines appear. Other methods like t-SNE and UMAP may perform better but at greater computational cost.