In this tutorial, we will look at the different data types in Lux and how to change them.

In Lux, data types convey the high-level, semantic roles for each attribute, such as whether it is temporal, nominal, or quantitative. The detected data type information are then used to infer the appropriate types of visualization to display for each attribute. Note that data types in Lux are different from the dtype in Pandas, which involve atomic types such as string, integer, and float. Lux automatically infers the data type information for each column in the dataframe. The following code snippet shows what data types were inferred:

We will be using several different datasets for this tutorial.

In [ ]:
import pandas as pd
import lux
In [ ]:
lux.config.default_display="lux"
In [ ]:
df = pd.read_csv("https://github.com/lux-org/lux-datasets/blob/master/data/college.csv?raw=true")
df

To find what Lux has determined as the data type, use df.data_type. This is a dictionary object.

In [ ]:
df.data_type

Quantitative

Quantitative data is used to describe numerical measures. This data type is typically assigned when Lux a numerical column consisting of floats or integers has large numbers of distinct values. In the example above, the column AcceptanceRate is detected as an quantitative attribute.

By default, Lux displays the Correlation action, displaying the relationship between two quantitative columns as scatterplots or heatmaps.

Lux also shows the Distribution action, displaying the distribution of each quantitative attributes.

In [ ]:
df

Nominal

Nominal data types describes unordered, categorical attributes. For example, the attribute PredominantDegree is nominal because it contains only three distinct values: Associate, Bachelor's, and Certificate. Below: Lux displays the occurence counts of nominal variables under the Occurrence action as bar charts.

In [ ]:
df

Temporal

Temporal data types describe time-related attributes, such as dates and timestamps. Lux infers that an attribute is temporal based on the data format, content, and name of the column. For example, the Year attribute is detected as a temporal data type in this example:

In [ ]:
df = pd.read_csv("https://github.com/lux-org/lux-datasets/blob/master/data/car.csv?raw=true")
df["Year"] = pd.to_datetime(df["Year"], format="%Y")
df

ID

ID data type describes identifier columns, such as zip code, product or user ID. Typically, columns that are detected as ID data type do not contain a lot of useful information and should not be plotted. For example, in the code snipped below, we see that the column enrolee_id has numerical data, it is categorized as an ID data type, so no visualizations correspond to it.

In [ ]:
df = pd.read_csv("https://github.com/lux-org/lux-datasets/blob/master/data/aug_test.csv?raw=true")
df

Changing the Inferred Data Type

Lux attempts to infer the data type information for every column in the dataframe. However, sometimes there is ambiguity in how the data should be modelled. For example, perhaps an attribute is detected as an ID field, but it is better visualized as a quantitative column. Alternatively, there may be a column that looks quantitative, but would be better modelled as a nominal. For example:

In [ ]:
df = pd.read_csv("https://raw.githubusercontent.com/lux-org/lux-datasets/master/data/communities.csv?raw=true")
df
In [ ]:
df.data_type

Lux incorrectly assumes that :state is a quantitative column because the column seems to be made of numbers. If we plot a visualization based on state, a histogram is displayed:

In [ ]:
from lux.vis.Vis import Vis
In [ ]:
Vis(["state"], df)

However, in reality, state does not represent numbers with any meaningful relationship since the numbers discretely map to individual states. In this case, the state column is better suited as a nominal column. To change the inferred data type, use the set_data_type method:

In [ ]:
df.set_data_type({"state":"nominal"})

The user specified data type information overrides the Lux-detected type information. From now on, Lux will interpret the state column as a nominal variable. We can validate this by inspecting df.data_type:

In [ ]:
df.data_type

Now, when we plot the same visualization again, the nominal state column is displayed as a bar chart visualization. This bar chart visualization shows that state 6, 34, and 48 has the largest number of records, an insight that was hidden away in the binned histogram when the data type was misdetected.

In [ ]:
Vis(["state"], df)