rubicon-ml experiments can be tagged with special identifiers to denote a parent/child relationship.
This can be used to track hierarchical or iterative experiments, among other things.
First, let's create a project.
from rubicon_ml import Rubicon
rubicon = Rubicon(persistence="memory")
project = rubicon.create_project(name="hierarchical experiments")
project
<rubicon_ml.client.project.Project at 0x121d7af50>
Now we can log some experiments in a nested loop. Imagine logging an experiment for each node of a gradient boosted tree, or something along those lines.
We can use parent_experiment.add_child_experiment(child_experiment) to automatically add tags
to both parent_experiment and child_experiment that represent their relationship.
root_experiment = project.log_experiment(name="root")
for n in range(3):
node_experiment = project.log_experiment(name=f"node_{n}")
root_experiment.add_child_experiment(node_experiment)
for m in range(2):
nested_node_experiment = project.log_experiment(name=f"node_{n}_{m}")
node_experiment.add_child_experiment(nested_node_experiment)
To retrieve experiments, start at the root experiment and call get_child_experiments to return a
list of rubicon-ml objects representing each of the tagged child experiments.
for experiment in root_experiment.get_child_experiments():
print("id:", experiment.id)
print("tags:", [t for t in experiment.tags if "child" in t], "\n")
for nested_experiment in experiment.get_child_experiments():
print("\tid:", nested_experiment.id)
print("\ttags:", nested_experiment.tags, "\n")
id: f8c897d7-852d-4020-8715-264452a5b8ab tags: ['child:754dff35-aa87-4385-9bcd-af5c1f5b0b7e', 'child:fdf25d72-d1bb-47bc-8cb2-4a088ed0ba33'] id: 754dff35-aa87-4385-9bcd-af5c1f5b0b7e tags: ['parent:f8c897d7-852d-4020-8715-264452a5b8ab'] id: fdf25d72-d1bb-47bc-8cb2-4a088ed0ba33 tags: ['parent:f8c897d7-852d-4020-8715-264452a5b8ab'] id: 5f8c14c1-50d9-4c49-b0b2-6a59c6f3d707 tags: ['child:d9012c99-2888-43d1-833b-78d51de75a3a', 'child:c65f8c5e-bf5b-4a82-94cb-8b669545b951'] id: d9012c99-2888-43d1-833b-78d51de75a3a tags: ['parent:5f8c14c1-50d9-4c49-b0b2-6a59c6f3d707'] id: c65f8c5e-bf5b-4a82-94cb-8b669545b951 tags: ['parent:5f8c14c1-50d9-4c49-b0b2-6a59c6f3d707'] id: da33e918-96c0-4e56-9075-7941515cc18f tags: ['child:7c5b2f4b-1e7c-40be-8cda-8f3a00067e98', 'child:cd30f3b2-bd63-4318-974a-6668648bf4ac'] id: 7c5b2f4b-1e7c-40be-8cda-8f3a00067e98 tags: ['parent:da33e918-96c0-4e56-9075-7941515cc18f'] id: cd30f3b2-bd63-4318-974a-6668648bf4ac tags: ['parent:da33e918-96c0-4e56-9075-7941515cc18f']
We can leverage add_child_experiment to maintain iterative relationships too. This could be
used to log metadata about of each iteration of recursive feature elimination and preserve the
linear history of the model training.
current_experiment = project.log_experiment(name="experiment_0")
for n in range(3):
next_experiment = project.log_experiment(name=f"experiment_{n+1}")
current_experiment.add_child_experiment(next_experiment)
current_experiment = next_experiment
last_experiment = current_experiment
Similarly to get_child_experiments, we can use get_parent_experiment to return a rubicon-ml
object representing the tagged parent experiment.
experiments = [last_experiment]
while len(experiments) != 0:
experiment = experiments[0]
print("name:", experiment.name)
print("\tid:", experiment.id)
print("\ttags:", experiment.tags, "\n")
experiments = experiment.get_parent_experiments()
name: experiment_3 id: a85ba9d9-1473-44c2-b3e9-8a744534485b tags: ['parent:47ecf8a7-b799-4308-994b-aa6d8698dc2b'] name: experiment_2 id: 47ecf8a7-b799-4308-994b-aa6d8698dc2b tags: ['child:a85ba9d9-1473-44c2-b3e9-8a744534485b', 'parent:aea5f005-9792-442c-b98c-8d9b9e39f99b'] name: experiment_1 id: aea5f005-9792-442c-b98c-8d9b9e39f99b tags: ['child:47ecf8a7-b799-4308-994b-aa6d8698dc2b', 'parent:f4a393ef-0b32-4f70-ac82-07a0877da328'] name: experiment_0 id: f4a393ef-0b32-4f70-ac82-07a0877da328 tags: ['child:aea5f005-9792-442c-b98c-8d9b9e39f99b']