Res1D - Strategies for handling large files¶

In [1]:

import mikeio1d

All static network data is loaded into memory by default. Dynamic result data is only loaded when required.

In [2]:

res = mikeio1d.open("../tests/testdata/network.res1d")
loaded_time_steps = res.data.NumberOfTimeSteps
loaded_time_steps

Out[2]:

Calling read() loads dynamic data for the entire network.

In [3]:

res.read()
loaded_time_steps = res.data.NumberOfTimeSteps
loaded_time_steps

Out[3]:

All network dynamic data is loaded into memory regardless of how read() is called. In the future, it would nice to make this work lazily.

In [4]:

res = mikeio1d.open("../tests/testdata/network.res1d")
# this still reads all network data, but only creates a DataFrame with the requested data
res.nodes["1"].WaterLevel.read().head()

Out[4]:

	WaterLevel:1
1994-08-07 16:35:00.000	195.052994
1994-08-07 16:36:01.870	195.052994
1994-08-07 16:37:07.560	195.052994
1994-08-07 16:38:55.828	195.052994
1994-08-07 16:39:55.828	195.052994

To avoid loading the entire network's dynamic data into memory, you can specify which IDs to load when opening the file.

In [5]:

res = mikeio1d.open("../tests/testdata/network.res1d", nodes=["1"], reaches=["100l1"])
res.nodes

Out[5]:

Names (1)

1

Quantities (1)

Water level (m)

Derived Quantities (3)

NodeFlooding
NodeWaterDepth
NodeWaterLevelAboveCritical

In [6]:

res.reaches

Out[6]:

Names (1)

100l1

Quantities (2)

Water level (m)
Discharge (m^3/s)

Derived Quantities (6)

ReachAbsoluteDischarge
ReachFilling
ReachFlooding
ReachQQManning
ReachWaterDepth
ReachWaterLevelAboveCritical

In [7]:

res.read()

Out[7]:

	WaterLevel:1	WaterLevel:100l1:0	WaterLevel:100l1:47.6827	Discharge:100l1:23.8414
1994-08-07 16:35:00.000	195.052994	195.441498	194.661499	0.000006
1994-08-07 16:36:01.870	195.052994	195.441498	194.661621	0.000006
1994-08-07 16:37:07.560	195.052994	195.441498	194.661728	0.000006
1994-08-07 16:38:55.828	195.052994	195.441498	194.661804	0.000006
1994-08-07 16:39:55.828	195.052994	195.441498	194.661972	0.000006
...	...	...	...	...
1994-08-07 18:30:07.967	195.119919	195.455109	194.689072	0.000588
1994-08-07 18:31:07.967	195.118607	195.455063	194.688934	0.000583
1994-08-07 18:32:07.967	195.117310	195.455002	194.688812	0.000579
1994-08-07 18:33:07.967	195.115753	195.453049	194.688354	0.000526
1994-08-07 18:35:00.000	195.112534	195.450409	194.686172	0.000343

110 rows × 4 columns

Similarly, you can also specify the time range to read.

In [8]:

start, end = "1994-08-07 18:30:07.967", "1994-08-07 18:33:07.967"
res = mikeio1d.open(
    "../tests/testdata/network.res1d", nodes=["1"], reaches=["100l1"], time=(start, end)
)
loaded_time_steps = res.data.NumberOfTimeSteps
loaded_time_steps

Out[8]:

In [9]:

res.read()
loaded_time_steps = res.data.NumberOfTimeSteps
loaded_time_steps

Out[9]:

In [10]:

res.read().head()

Out[10]:

	WaterLevel:1	WaterLevel:100l1:0	WaterLevel:100l1:47.6827	Discharge:100l1:23.8414
1994-08-07 18:30:07.967	195.119919	195.455109	194.689072	0.000588
1994-08-07 18:31:07.967	195.118607	195.455063	194.688934	0.000583
1994-08-07 18:32:07.967	195.117310	195.455002	194.688812	0.000579
1994-08-07 18:33:07.967	195.115753	195.453049	194.688354	0.000526