Multi variable comparison¶

Assessing both wave height and wind speed at the same time

In [1]:

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from fmskill import ModelResult, ModelResultCollection, PointObservation, TrackObservation

In [2]:

%load_ext autoreload
%autoreload 2

Define observations¶

Below, the observations will take the default variable names from the eum type of the item. Alternatively, the user can give another variable name by providing the variable_name argument.

In [3]:

# wave height
o1 = PointObservation('../tests/testdata/SW/HKNA_Hm0.dfs0', item=0, x=4.2420, y=52.6887, name="HKNA_Hm0")
o2 = PointObservation("../tests/testdata/SW/eur_Hm0.dfs0", item=0, x=3.2760, y=51.9990, name="EPL_Hm0")
o3 = TrackObservation("../tests/testdata/SW/Alti_c2_Dutch.dfs0", item=3, name="c2_Hm0")

# wind speed
wind1 = PointObservation('../tests/testdata/SW/HKNA_wind.dfs0', item=0, x=4.2420, y=52.6887, name="HKNA_wind")
wind2 = PointObservation('../tests/testdata/SW/F16_wind.dfs0', item=0, x=4.01222, y=54.1167, name="F16_wind")
wind3 = TrackObservation("../tests/testdata/SW/Alti_c2_Dutch.dfs0", item=2, name="c2_wind")

In [4]:

o1.variable_name

Out[4]:

'Significant_wave_height'

Define model results¶

Two different model results are defined.

In [5]:

mr1 = ModelResult('../tests/testdata/SW/HKZN_local_2017_DutchCoast.dfsu', name='SW_1')
mr2 = ModelResult('../tests/testdata/SW/HKZN_local_2017_DutchCoast_v2.dfsu', name='SW_2')

In [6]:

mr1.dfs.items

Out[6]:

[Sign. Wave Height <Significant wave height> (meter),
 Max. Wave Height <Maximum wave height> (meter),
 Peak Wave Period <Wave period> (second),
 Wave Period, T01 <Wave period> (second),
 Wave Period, T02 <Wave period> (second),
 Peak Wave Direction <Wave direction> (radian),
 Mean Wave Direction <Mean Wave Direction> (degree),
 Dir. Stand. Deviation <Directional Standard Deviation> (degree),
 x-comp. of wave height vector <u velocity component> (meter per sec),
 y-comp. of wave height vector <v velocity component> (meter per sec),
 Surface elevation <Surface Elevation> (meter),
 Current velocity, U <u velocity component> (meter per sec),
 Current velocity, V <v velocity component> (meter per sec),
 Wind speed <Wind speed> (meter per sec),
 Wind direction <Wind Direction> (degree)]

Associate model and observations and extract¶

We match the observation item and model item by refering to the item name in the ModelResult. Item number can also be used.

In [7]:

mr = ModelResultCollection([mr1, mr2])
#mr = mr1

In [8]:

mr.add_observation(o1, item='Sign. Wave Height')
mr.add_observation(o2, item='Sign. Wave Height')
mr.add_observation(o3, item='Sign. Wave Height')
mr.add_observation(wind1, item='Wind speed')
mr.add_observation(wind2, item='Wind speed')
mr.add_observation(wind3, item='Wind speed')
mr.observations

Out[8]:

{'HKNA_Hm0': PointObservation: HKNA_Hm0, x=4.242, y=52.6887,
 'EPL_Hm0': PointObservation: EPL_Hm0, x=3.276, y=51.999,
 'c2_Hm0': TrackObservation: c2_Hm0, n=298,
 'HKNA_wind': PointObservation: HKNA_wind, x=4.242, y=52.6887,
 'F16_wind': PointObservation: F16_wind, x=4.01222, y=54.1167,
 'c2_wind': TrackObservation: c2_wind, n=298}

In [9]:

cc = mr.extract() 

In [10]:

cc.n_variables

Out[10]:

In [11]:

cc.var_names

Out[11]:

['Significant_wave_height', 'Wind_speed']

Analysis¶

Now that the result has been extracted, we can do analysis. Multiple variables means an extra level in the multi-index of the skill dataframe.

In [12]:

s = cc.skill()
s

Out[12]:

			n	bias	rmse	urmse	mae	cc	si	r2
model	observation	variable
SW_1	EPL_Hm0	Significant_wave_height	66	-0.075335	0.216357	0.202817	0.183641	0.972467	0.073902	0.934093
	F16_wind	Wind_speed	67	2.102049	2.775330	1.812140	2.196785	0.824620	0.136781	0.229514
	HKNA_Hm0	Significant_wave_height	385	-0.195266	0.352283	0.293214	0.251992	0.971082	0.088488	0.904815
	HKNA_wind	Wind_speed	277	-0.880928	1.276179	0.923363	1.023092	0.962845	0.064528	0.860548
	c2_Hm0	Significant_wave_height	113	-0.001210	0.351796	0.351794	0.294585	0.974335	0.118511	0.899507
	c2_wind	Wind_speed	113	0.408558	0.637718	0.489657	0.505665	0.960284	0.049998	0.867103
SW_2	EPL_Hm0	Significant_wave_height	66	-0.007782	0.225996	0.225862	0.193719	0.972467	0.082298	0.928089
	F16_wind	Wind_speed	67	2.102049	2.775330	1.812140	2.196785	0.824620	0.136781	0.229514
	HKNA_Hm0	Significant_wave_height	385	-0.101189	0.293247	0.275235	0.214476	0.971082	0.083062	0.934045
	HKNA_wind	Wind_speed	277	-0.880928	1.276179	0.923363	1.023092	0.962845	0.064528	0.860548
	c2_Hm0	Significant_wave_height	113	0.081431	0.430268	0.422492	0.357138	0.974335	0.142327	0.849675
	c2_wind	Wind_speed	113	0.408558	0.637718	0.489657	0.505665	0.960284	0.049998	0.867103

In [13]:

s.index.names

Out[13]:

FrozenList(['model', 'observation', 'variable'])

In [14]:

s = cc.skill(variable='Significant_wave_height')
s.style()

Out[14]:

		variable	n	bias	rmse	urmse	mae	cc	si	r2
model	observation
SW_1	EPL_Hm0	Significant_wave_height	66	-0.075	0.216	0.203	0.184	0.972	0.074	0.934
	HKNA_Hm0	Significant_wave_height	385	-0.195	0.352	0.293	0.252	0.971	0.088	0.905
	c2_Hm0	Significant_wave_height	113	-0.001	0.352	0.352	0.295	0.974	0.119	0.900
SW_2	EPL_Hm0	Significant_wave_height	66	-0.008	0.226	0.226	0.194	0.972	0.082	0.928
	HKNA_Hm0	Significant_wave_height	385	-0.101	0.293	0.275	0.214	0.971	0.083	0.934
	c2_Hm0	Significant_wave_height	113	0.081	0.430	0.422	0.357	0.974	0.142	0.850

In [15]:

s.sel(observation='c2_Hm0').style(columns='rmse')

Out[15]:

	observation	variable	n	bias	rmse	urmse	mae	cc	si	r2
model
SW_1	c2_Hm0	Significant_wave_height	113	-0.001	0.352	0.352	0.295	0.974	0.119	0.900
SW_2	c2_Hm0	Significant_wave_height	113	0.081	0.430	0.422	0.357	0.974	0.142	0.850

In [16]:

s.plot_line('rmse', ylabel='Hm0 RMSE [m]');

In [17]:

cc.scatter(model=1)

mean skill¶

The mean_skill() method will return a weighted average of the skill score per model and variable. You can get the "normal" mean_skill (per model) by selecting a specific variable either by id or name.

In [18]:

s = cc.mean_skill()
s

Out[18]:

		n	bias	rmse	urmse	mae	cc	si	r2
model	variable
SW_1	Significant_wave_height	564	-0.090604	0.306812	0.282609	0.243406	0.972628	0.093634	0.912805
SW_1	Wind_speed	457	0.543226	1.563076	1.075053	1.241847	0.915916	0.083769	0.652388
SW_2	Significant_wave_height	564	-0.009180	0.316503	0.307863	0.255111	0.972628	0.102563	0.903936
SW_2	Wind_speed	457	0.543226	1.563076	1.075053	1.241847	0.915916	0.083769	0.652388

In [19]:

s.plot_bar('si', title='scatter index');

In [20]:

cc.mean_skill(model='SW_2')

Out[20]:

	model	n	bias	rmse	urmse	mae	cc	si	r2
variable
Significant_wave_height	SW_2	564	-0.009180	0.316503	0.307863	0.255111	0.972628	0.102563	0.903936
Wind_speed	SW_2	457	0.543226	1.563076	1.075053	1.241847	0.915916	0.083769	0.652388

In [21]:

cc.mean_skill(variable='Significant_wave_height').style(columns=[])

Out[21]:

	variable	n	bias	rmse	urmse	mae	cc	si	r2
model
SW_1	Significant_wave_height	564	-0.091	0.307	0.283	0.243	0.973	0.094	0.913
SW_2	Significant_wave_height	564	-0.009	0.317	0.308	0.255	0.973	0.103	0.904

score¶

In [22]:

cc.score()

Out[22]:

{'SW_1': 0.9349438989181208, 'SW_2': 0.9397895582539258}

In [23]:

cc.score(model='SW_1')

Out[23]:

0.9349438989181208

In [24]:

cc.score(variable='Wind_speed')

Out[24]:

{'SW_1': 1.5630757331111964, 'SW_2': 1.5630757331111964}

In [ ]: