IPython holds a reference (in a few places) on the last error and its traceback. This can cause problems, because any object references in any frame in the traceback will not get garbage collected naturally.
If we can find the references, we can clean them up (and maybe suggest some improvements to IPython)
Prompted by https://mastodon.social/@hannorein/113697884922216639
Setup: an object that raises an exception in a method
import sys
class Test:
def exception(self):
raise ValueError("test")
def __del__(self):
print("freed!")
t = Test()
print(sys.getrefcount(t))
2
t.exception()
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[2], line 1 ----> 1 t.exception() Cell In[1], line 5, in Test.exception(self) 4 def exception(self): ----> 5 raise ValueError("test") ValueError: test
print(sys.getrefcount(t))
4
del t
That should have printed 'freed', but it didn't.
This is because the last error's traceback still holds a reference on t
and IPython has references to the traceback in 3 places:
exception.__traceback__
on the exception raised (easier than clearing every reference to the Exception itself)sys.last_traceback
for debugger supportget_ipython().InteractiveTB.tb
which is a cache value, and probably a bug that the value is still heldIf we clear these, the refcount goes back to normal and t
is freed
# clear held references to the latest traceback
import gc
sys.last_value.__traceback__ = None
sys.last_traceback = None
get_ipython().InteractiveTB.tb = None
# gc.collect seems to be needed to resolve some cycles somewhere
gc.collect();
freed!
We can use IPython's post_run_cell
hook to do this automatically on every cell execution,
so you can put this in your ipython_config.py
:
# do the above automatically when a cell fails
import gc
import sys
import traceback
def clear_last_tb(result):
if result.error_in_exec:
result.error_in_exec.__traceback__ = None
sys.last_traceback = None
get_ipython().InteractiveTB.tb = None
gc.collect()
get_ipython().events.register("post_run_cell", clear_last_tb)
t = Test()
t.exception()
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[7], line 2 1 t = Test() ----> 2 t.exception() Cell In[1], line 5, in Test.exception(self) 4 def exception(self): ----> 5 raise ValueError("test") ValueError: test
print(sys.getrefcount(t))
del t
2 freed!
Now we have enough information to propose an enhancement to IPython itself, perhaps.