Notebook

For How do I construct a python function where the input in python code and output is ipython rich output (in HTML)?, based on Can I capture rich outputs in plain python sessions ?, and Module: utils.capture ¶

In [1]:

pycode = '''
from matplotlib import pyplot as plt
plt.plot([1, 2, 3])
plt.show()
'''
from IPython.utils import io
with io.capture_output() as captured:
    exec(pycode)

In [2]:

type(captured)

Out[2]:

IPython.utils.capture.CapturedIO

Based on and Module: utils.capture

In [3]:

captured.outputs[0]

Out[3]:

In [4]:

type(captured.outputs[0])

Out[4]:

IPython.utils.capture.RichOutput

In [5]:

captured.show()

In [6]:

captured()

In [7]:

from IPython.display import display
for o in captured.outputs:
    display(o)

Want a figure?¶

In [8]:

pycode = '''
from matplotlib import pyplot as plt
plt.plot([1, 2, 3])
plt.savefig('my_plot.png')
plt.show()
'''
from IPython.utils import io
with io.capture_output() as captured:
    exec(pycode)

In [9]:

from IPython.display import Image
Image("my_plot.png")

Out[9]:

So adding in the saving the plot as an image in the original use of matplotlib works.

If you include a line assigning the plot to a Matplotlib figure object before you make the plot, you can also decide after-the-fact to save it as an image.

In [10]:

pycode = '''
from matplotlib import pyplot as plt
fig = plt.figure()
plt.plot([1, 2, 3])
plt.show()
'''
from IPython.utils import io
with io.capture_output() as captured:
    exec(pycode)

In [11]:

fig.savefig("after_the_fact_save.png")

In [12]:

from IPython.display import Image
Image("after_the_fact_save.png")

Out[12]:

So adding in the saving the plot as an image via matplotlib works.

But can you save the 'RichOuput' from Matplotlib, captured.outputs[0] in this case, to an image, too? So you can keep the matplotlib handling simpler and decide after-the-fact if you want the image and get it progammatically. (Because when in JupyterLab, you have the option to right-click on the plot and choose 'Create New View for Ouput' and then right-clicking on that in the 'Output View' and choosing to 'Save Image As..' to collect an image file by hand.)

Based on here, I didn't think that is easy to do. And I was going to suggest digging into how nbgather works, sugged here may provide a way; howver, it didn't seem straightforward right now.

However, I found two options based on search of 'jupyter %%capture send to image file'. First based on a reply to 'Exporting a single jupyter cell output', where use of '%%capture to save the png output' is discussed.

So I tried:

with open('from_richoutputs.png', 'wb') as f:
    f.write(captured.outputs[0].data['image/png'])

However, that gives 'TypeError: a bytes-like object is required, not 'str'' because it turns out that seems written for something already outputing .png, I think. Even though the pseudocode doesn't work that way. So similar with the other promising idea that turns out to be useful only when the RichOutput was a .png. And so I'm back to thinking not easy with matplotlib and be at least be happy one route works. (The page with the other promising idea also includes metnion of nbinteract/scrapbook and maybe that is appropriate here?)

Can you make the plot HTML code?¶

nbconvert of a notebook with just that code and output to get HTML¶

If you are working in Jupyter than one option to get HTML version of the plot would be to use nbconvert to make the HTML, see here. Example:

jupyter nbconvert --to html so74683522.ipynb

The produced HTML will contain code needed to make the plot.
You can imagine just making a notebook where the only cell is what produces the plot to make it easy to find the pertinent code.
In fact, let's demonstrate that by making a notebook that will only have the following single cell, with no output yet:

pycode = '''
from matplotlib import pyplot as plt
plt.plot([1, 2, 3])
plt.show()
'''
exec(pycode)

To do that, I'll us nbformat to make a Jupyter notebook .ipynb with that code as a single cell in it by running the next cell:

In [13]:

# based on https://stackoverflow.com/a/62105736/8508004 (setting metadata to python based on https://github.com/ParaToolsInc/taucmdr/blob/70daea6e775903a04d3008f4500c0ea60970e023/packages/taucmdr/analysis/analysis.py)
from nbformat import v4 as nbf
import nbformat
code_source = "pycode = '''\nfrom matplotlib import pyplot as plt\nplt.plot([1, 2, 3])\nplt.show()\n'''\nexec(pycode)"
new_ntbk = nbf.new_notebook()
cells = [nbf.new_code_cell(code_source)]
new_ntbk['cells'] = cells
new_ntbk['metadata']['kernelspec'] = nbformat.NotebookNode()
new_ntbk['metadata']['kernelspec']['name'] = 'python3'
new_ntbk['metadata']['kernelspec']['language'] = 'python'
new_ntbk['metadata']['kernelspec']['display_name'] = 'Python 3 (ipykernel)'
nbformat.write(new_ntbk, "single_cell_exec_without_output_via_nbf.ipynb")

We can see that is all that is there:

In [14]:

cat single_cell_exec_without_output_via_nbf.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c761258",
   "metadata": {},
   "outputs": [],
   "source": [
    "pycode = '''\n",
    "from matplotlib import pyplot as plt\n",
    "plt.plot([1, 2, 3])\n",
    "plt.show()\n",
    "'''\n",
    "exec(pycode)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

That's it. Just contains that small block of code we made above. Let's excute it

In [15]:

# based on https://nbconvert.readthedocs.io/en/latest/execute_api.html#executing-notebooks-using-the-python-api-interface
notebook_filename = "single_cell_exec_without_output_via_nbf.ipynb"
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
with open(notebook_filename) as f:
    nb = nbformat.read(f, as_version=4)
ExecutePreprocessor().preprocess(nb, {'metadata': {'path': './'}})
with open('executed_notebook.ipynb', 'w', encoding='utf-8') as f:
    nbformat.write(nb, f)

That will execute the notebook. Now to get the HTML we'll convert it and specify we don't want the input cell source in the result, see here for more about the --no-input flag to leave out the code cells from the produced HTML.

In [16]:

!jupyter nbconvert --no-input --to html executed_notebook.ipynb

[NbConvertApp] Converting notebook executed_notebook.ipynb to html
[NbConvertApp] Writing 587936 bytes to executed_notebook.html

Let's view the first few lines of HTML code produced.

In [17]:

!head executed_notebook.html

<!DOCTYPE html>
<html>
<head><meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0">

<title>executed_notebook</title><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>

Replace !head part of the command above with cat and re-run it to see it all. There's a lot and so I just wanted to show the top to demonstrate it indeed produced an HTML file. You should be able to take the code and embed it elsewhere. In fact, if you really examine it you'll see it has an image of the plot embedded as base64. You can deduce how to embed only that elsewhere from examples below.

To view the HTML as HTML, in JupyterLab you can just double-click on the HTML icon in the file browser pane to the left. You'll note there is just a plot.

Make a image of the plot after-the-fact and converting that to HTML¶

Also, using the figure, you can create a Base64 version of the plot image to embed in HTML, see here. In fact let's see that demonstrated in this notebook, based on a variation here, in reply to Imbed [sic] matplotlib figure into iPython HTML. Plus, updating that to Python 3 using here and here.

In [18]:

from IPython.core.display import HTML
import binascii
#from io import StringIO
from io import BytesIO
import base64
import matplotlib.pyplot as plt

# open IO object
#sio = StringIO()
sio = BytesIO()

pycode = '''
from matplotlib import pyplot as plt
fig = plt.figure()
plt.plot([1, 2, 3])
plt.show()
'''
from IPython.utils import io
with io.capture_output() as captured:
    exec(pycode)

# print raw canvas data to IO object
#fig.canvas.print_png(sio)
fig.savefig(sio, format="png")

# convert raw binary data to base64
# I use this to embed in an img tag
#img_data = binascii.b2a_base64(sio.getvalue())
img_data = base64.encodebytes(sio.getvalue()).decode()

# keep img tag outter html in its own variable
img_html = '<img src="data:image/png;base64,{}&#10;">'.format(img_data)

HTML("<h1 style='color:purple'>Hello, this is all HTML in this output block:</h1><hr/>"+img_html+"<hr>")

Out[18]:

Hello, this is all HTML in this output block:

Making a figure object, the fig = plt.figure() step, was introduced in the section above.

Use the Matplotlib figure object to go to embeddable HTML¶

However, the OP was seeking to be away from Jupyter and IPython. Can you make HTML from matplotlib objects?

Using mpld3 you can. That package has an fig_to_html() method. Let's install that here:

In [19]:

%pip install mpld3

Collecting mpld3
  Downloading mpld3-0.5.8-py3-none-any.whl (201 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.0/201.0 KB 1.2 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: matplotlib in /srv/conda/envs/notebook/lib/python3.7/site-packages (from mpld3) (3.5.2)
Requirement already satisfied: jinja2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from mpld3) (3.1.1)
Requirement already satisfied: MarkupSafe>=2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jinja2->mpld3) (2.1.1)
Requirement already satisfied: pillow>=6.2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (9.2.0)
Requirement already satisfied: pyparsing>=2.2.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (3.0.7)
Requirement already satisfied: fonttools>=4.22.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (4.34.4)
Requirement already satisfied: kiwisolver>=1.0.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (1.4.4)
Requirement already satisfied: packaging>=20.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (21.3)
Requirement already satisfied: python-dateutil>=2.7 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (0.11.0)
Requirement already satisfied: numpy>=1.17 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->mpld3) (1.21.6)
Requirement already satisfied: typing-extensions in /srv/conda/envs/notebook/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib->mpld3) (4.1.1)
Requirement already satisfied: six>=1.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from python-dateutil>=2.7->matplotlib->mpld3) (1.16.0)
Installing collected packages: mpld3
Successfully installed mpld3-0.5.8
Note: you may need to restart the kernel to use updated packages.

Let's connect mpld3 use to this process to make an HTML version of the plot:

In [20]:

pycode = '''
from matplotlib import pyplot as plt
fig = plt.figure()
plt.plot([1, 2, 3])
plt.show()
'''
from IPython.utils import io
with io.capture_output() as captured:
    exec(pycode)

import mpld3
html_string= mpld3.fig_to_html(fig)

How to make a figure object was covered in the section above. Besides, using that, we've just import mpl and used the fig_to_html() method using the Matplotlib figure object.

That should have produced HTML; let's examine it to see:

In [21]:

print(html_string)


<style>

</style>

<div id="fig_el1201399120396793769970057120"></div>
<script>
function mpld3_load_lib(url, callback){
  var s = document.createElement('script');
  s.src = url;
  s.async = true;
  s.onreadystatechange = s.onload = callback;
  s.onerror = function(){console.warn("failed to load library " + url);};
  document.getElementsByTagName("head")[0].appendChild(s);
}

if(typeof(mpld3) !== "undefined" && mpld3._mpld3IsLoaded){
   // already loaded: just create the figure
   !function(mpld3){
       
       mpld3.draw_figure("fig_el1201399120396793769970057120", {"width": 432.0, "height": 288.0, "axes": [{"bbox": [0.125, 0.125, 0.775, 0.755], "xlim": [-0.1, 2.1], "ylim": [0.9, 3.1], "xdomain": [-0.1, 2.1], "ydomain": [0.9, 3.1], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120139912318424592", "lines": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "id": "el120139912318079696", "color": "#1F77B4", "linewidth": 1.5, "dasharray": "none", "alpha": 1, "zorder": 2, "drawstyle": "default"}], "paths": [], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[0.0, 1.0], [1.0, 2.0], [2.0, 3.0]]}, "id": "el120139912039679376", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]});
   }(mpld3);
}else if(typeof define === "function" && define.amd){
   // require.js is available: use it to load d3/mpld3
   require.config({paths: {d3: "https://d3js.org/d3.v5"}});
   require(["d3"], function(d3){
      window.d3 = d3;
      mpld3_load_lib("https://mpld3.github.io/js/mpld3.v0.5.8.js", function(){
         
         mpld3.draw_figure("fig_el1201399120396793769970057120", {"width": 432.0, "height": 288.0, "axes": [{"bbox": [0.125, 0.125, 0.775, 0.755], "xlim": [-0.1, 2.1], "ylim": [0.9, 3.1], "xdomain": [-0.1, 2.1], "ydomain": [0.9, 3.1], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120139912318424592", "lines": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "id": "el120139912318079696", "color": "#1F77B4", "linewidth": 1.5, "dasharray": "none", "alpha": 1, "zorder": 2, "drawstyle": "default"}], "paths": [], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[0.0, 1.0], [1.0, 2.0], [2.0, 3.0]]}, "id": "el120139912039679376", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]});
      });
    });
}else{
    // require.js not available: dynamically load d3 & mpld3
    mpld3_load_lib("https://d3js.org/d3.v5.js", function(){
         mpld3_load_lib("https://mpld3.github.io/js/mpld3.v0.5.8.js", function(){
                 
                 mpld3.draw_figure("fig_el1201399120396793769970057120", {"width": 432.0, "height": 288.0, "axes": [{"bbox": [0.125, 0.125, 0.775, 0.755], "xlim": [-0.1, 2.1], "ylim": [0.9, 3.1], "xdomain": [-0.1, 2.1], "ydomain": [0.9, 3.1], "xscale": "linear", "yscale": "linear", "axes": [{"position": "bottom", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}, {"position": "left", "nticks": 11, "tickvalues": null, "tickformat_formatter": "", "tickformat": null, "scale": "linear", "fontsize": 10.0, "grid": {"gridOn": false}, "visible": true}], "axesbg": "#FFFFFF", "axesbgalpha": null, "zoomable": true, "id": "el120139912318424592", "lines": [{"data": "data01", "xindex": 0, "yindex": 1, "coordinates": "data", "id": "el120139912318079696", "color": "#1F77B4", "linewidth": 1.5, "dasharray": "none", "alpha": 1, "zorder": 2, "drawstyle": "default"}], "paths": [], "markers": [], "texts": [], "collections": [], "images": [], "sharex": [], "sharey": []}], "data": {"data01": [[0.0, 1.0], [1.0, 2.0], [2.0, 3.0]]}, "id": "el120139912039679376", "plugins": [{"type": "reset"}, {"type": "zoom", "button": true, "enabled": false}, {"type": "boxzoom", "button": true, "enabled": false}]});
            })
         });
}
</script>

Does it work? Because I'm demonstrating this in a Jupyter notebook and it has the ability to render HTML, I should be able to see.

In [22]:

from IPython.core.display import HTML
HTML(html_string)

Out[22]:

The D3 version shows the same infomation as the original plot. The aesthetics are slightly different. Importantly, it is HTML generated via Python that the OP wanted with not too much addition or change to the original code block posted.

Interestingly, the D3 plots have additional abilities you can access in the bottom right corner so that these are interactive. In fact, as highlighted in this notebook here, which I learned of here, you can use additional plugins from mpld3, to set the zoom settings at the outset and add a reset button.

For How do I construct a python function where the input in python code and output is ipython rich output (in HTML)?, based on Can I capture rich outputs in plain python sessions ?, and Module: utils.capture¶