Trying to clear object references in the 'last traceback' in IPython¶

IPython holds a reference (in a few places) on the last error and its traceback. This can cause problems, because any object references in any frame in the traceback will not get garbage collected naturally.

If we can find the references, we can clean them up (and maybe suggest some improvements to IPython)

Prompted by https://mastodon.social/@hannorein/113697884922216639

Setup: an object that raises an exception in a method

In [1]:

import sys

class Test:
    def exception(self):
        raise ValueError("test")

    def __del__(self):
        print("freed!")

t = Test()
print(sys.getrefcount(t))

In [2]:

t.exception()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[2], line 1
----> 1 t.exception()

Cell In[1], line 5, in Test.exception(self)
      4 def exception(self):
----> 5     raise ValueError("test")

ValueError: test

In [3]:

print(sys.getrefcount(t))

In [4]:

del t

That should have printed 'freed', but it didn't. This is because the last error's traceback still holds a reference on t and IPython has references to the traceback in 3 places:

exception.__traceback__ on the exception raised (easier than clearing every reference to the Exception itself)
sys.last_traceback for debugger support
get_ipython().InteractiveTB.tb which is a cache value, and probably a bug that the value is still held

If we clear these, the refcount goes back to normal and t is freed

In [5]:

# clear held references to the latest traceback
import gc

sys.last_value.__traceback__ = None
sys.last_traceback = None
get_ipython().InteractiveTB.tb = None
# gc.collect seems to be needed to resolve some cycles somewhere
gc.collect();

freed!

We can use IPython's post_run_cell hook to do this automatically on every cell execution, so you can put this in your ipython_config.py:

In [6]:

# do the above automatically when a cell fails
import gc
import sys
import traceback

def clear_last_tb(result):
    if result.error_in_exec:
        result.error_in_exec.__traceback__ = None
        sys.last_traceback = None
        get_ipython().InteractiveTB.tb = None
        gc.collect()
    
get_ipython().events.register("post_run_cell", clear_last_tb)

In [7]:

t = Test()
t.exception()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[7], line 2
      1 t = Test()
----> 2 t.exception()

Cell In[1], line 5, in Test.exception(self)
      4 def exception(self):
----> 5     raise ValueError("test")

ValueError: test

In [8]:

print(sys.getrefcount(t))
del t

2
freed!

Now we have enough information to propose an enhancement to IPython itself, perhaps.