Notebook

Section 7.1: A First Plotly Streaming Plot¶

Welcome to Plotly's Python API User Guide.

Links to the other sections can be found on the User Guide's homepage

Section 7 is divided, into separate notebooks, as follows:

Check which version is installed on your machine and please upgrade if needed.

In [1]:

# (*) Import plotly package
import plotly
 
# Check plolty version (if not latest, please upgrade)
plotly.__version__

Out[1]:

'1.9.5'

Import a few modules and sign in to Plotly using our credentials file:

In [3]:

# (*) To communicate with Plotly's server, sign in with credentials file
import plotly.plotly as py  
 
# (*) Useful Python/Plotly tools
import plotly.tools as tls   
 
# (*) Graph objects to piece together plots
from plotly.graph_objs import *
 
import numpy as np  # (*) numpy for math functions and arrays

Finally, retrieve the stream ids in our credentials file as set up in subsection 7.1:

In [6]:

stream_ids = tls.get_credentials_file()['stream_ids']

7.1 A first Plotly streaming plot¶

Making Plotly streaming plots sums up to working with two objects:

A stream id object (Stream in the plotly.graph_objs module),
A stream link object (py.Stream).

The stream id object is a graph object that embeds a particular stream id to each of your plot's traces. As all graph objects, the stream id object is equipped with extensive help() documentation, key validation and a nested update method. In brief, the stream id object initializes the connection between a trace in your Plotly graph and a data stream(for that trace).

Meanwhile, the stream link object, like all objects in the plotly.plotly module, links content in your Python/IPython session to Plotly's servers. More precisely, it is the interface that updates the data in the plotted traces in real-time(as identified with the unique stream id used in the corresponding stream id object).

If you find the Stream/py.Stream terminalogy too confusing --- and you do not mind not having access to the methods associated with Plotly graph objects' --- you can forgo the use of the stream id object and substitute it by a python dict() in the following examples.

So, we start by making an instance of the stream id object:

In [4]:

help(Stream)  # call help() to see the specifications of the Stream object!

Help on class Stream in module plotly.graph_objs.graph_objs:

class Stream(PlotlyDict)
 |  Valid attributes for 'stream' at path [] under parents ():
 |  
 |      ['token', 'maxpoints']
 |  
 |  Run `<stream-object>.help('attribute')` on any of the above.
 |  '<stream-object>' is the object at []
 |  
 |  Method resolution order:
 |      Stream
 |      PlotlyDict
 |      __builtin__.dict
 |      PlotlyBase
 |      __builtin__.object
 |  
 |  Methods inherited from PlotlyDict:
 |  
 |  __copy__(self)
 |  
 |  __deepcopy__(self, memodict={})
 |  
 |  __dir__(self)
 |      Dynamically return the existing and possible attributes.
 |  
 |  __getattr__(self, key)
 |      Python only calls this when key is missing!
 |  
 |  __getitem__(self, key)
 |      Calls __missing__ when key is not found. May mutate object.
 |  
 |  __init__(self, *args, **kwargs)
 |  
 |  __missing__(self, key)
 |      Mimics defaultdict. This is called from __getitem__ when key DNE.
 |  
 |  __setattr__(self, key, value)
 |      Maps __setattr__ onto __setitem__
 |  
 |  __setitem__(self, key, value, _raise=True)
 |      Validates/Converts values which should be Graph Objects.
 |  
 |  force_clean(self, **kwargs)
 |      Recursively remove empty/None values.
 |  
 |  get_data(self, flatten=False)
 |      Returns the JSON for the plot with non-data elements stripped.
 |  
 |  get_ordered(self, **kwargs)
 |      Return a predictable, OrderedDict version of self.
 |  
 |  help(self, attribute=None, return_help=False)
 |      Print help string for this object or an attribute of this object.
 |      
 |      :param (str) attribute: A valid attribute string for this object.
 |      :param (bool) return_help: Return help_string instead of printing it?
 |      :return: (None|str)
 |  
 |  strip_style(self)
 |      Recursively strip style from the current representation.
 |      
 |      All PlotlyDicts and PlotlyLists are guaranteed to survive the
 |      stripping process, though they made be left empty. This is allowable.
 |      
 |      Keys that will be stripped in this process are tagged with
 |      `'type': 'style'` in graph_objs_meta.json. Note that a key tagged as
 |      style, but with an array as a value may still be considered data.
 |  
 |  to_string(self, level=0, indent=4, eol='\n', pretty=True, max_chars=80)
 |      Returns a formatted string showing graph_obj constructors.
 |      
 |      :param (int) level: The number of indentations to start with.
 |      :param (int) indent: The indentation amount.
 |      :param (str) eol: The end of line character(s).
 |      :param (bool) pretty: Curtail long list output with a '..' ?
 |      :param (int) max_chars: The max characters per line.
 |      
 |      Example:
 |      
 |          print(obj.to_string())
 |  
 |  update(self, dict1=None, **dict2)
 |      Update current dict with dict1 and then dict2.
 |      
 |      This recursively updates the structure of the original dictionary-like
 |      object with the new entries in the second and third objects. This
 |      allows users to update with large, nested structures.
 |      
 |      Note, because the dict2 packs up all the keyword arguments, you can
 |      specify the changes as a list of keyword agruments.
 |      
 |      Examples:
 |      # update with dict
 |      obj = Layout(title='my title', xaxis=XAxis(range=[0,1], domain=[0,1]))
 |      update_dict = dict(title='new title', xaxis=dict(domain=[0,.8]))
 |      obj.update(update_dict)
 |      obj
 |      {'title': 'new title', 'xaxis': {'range': [0,1], 'domain': [0,.8]}}
 |      
 |      # update with list of keyword arguments
 |      obj = Layout(title='my title', xaxis=XAxis(range=[0,1], domain=[0,1]))
 |      obj.update(title='new title', xaxis=dict(domain=[0,.8]))
 |      obj
 |      {'title': 'new title', 'xaxis': {'range': [0,1], 'domain': [0,.8]}}
 |      
 |      This 'fully' supports duck-typing in that the call signature is
 |      identical, however this differs slightly from the normal update
 |      method provided by Python's dictionaries.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from PlotlyDict:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Methods inherited from __builtin__.dict:
 |  
 |  __cmp__(...)
 |      x.__cmp__(y) <==> cmp(x,y)
 |  
 |  __contains__(...)
 |      D.__contains__(k) -> True if D has a key k, else False
 |  
 |  __delitem__(...)
 |      x.__delitem__(y) <==> del x[y]
 |  
 |  __eq__(...)
 |      x.__eq__(y) <==> x==y
 |  
 |  __ge__(...)
 |      x.__ge__(y) <==> x>=y
 |  
 |  __getattribute__(...)
 |      x.__getattribute__('name') <==> x.name
 |  
 |  __gt__(...)
 |      x.__gt__(y) <==> x>y
 |  
 |  __iter__(...)
 |      x.__iter__() <==> iter(x)
 |  
 |  __le__(...)
 |      x.__le__(y) <==> x<=y
 |  
 |  __len__(...)
 |      x.__len__() <==> len(x)
 |  
 |  __lt__(...)
 |      x.__lt__(y) <==> x<y
 |  
 |  __ne__(...)
 |      x.__ne__(y) <==> x!=y
 |  
 |  __repr__(...)
 |      x.__repr__() <==> repr(x)
 |  
 |  __sizeof__(...)
 |      D.__sizeof__() -> size of D in memory, in bytes
 |  
 |  clear(...)
 |      D.clear() -> None.  Remove all items from D.
 |  
 |  copy(...)
 |      D.copy() -> a shallow copy of D
 |  
 |  fromkeys(...)
 |      dict.fromkeys(S[,v]) -> New dict with keys from S and values equal to v.
 |      v defaults to None.
 |  
 |  get(...)
 |      D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.
 |  
 |  has_key(...)
 |      D.has_key(k) -> True if D has a key k, else False
 |  
 |  items(...)
 |      D.items() -> list of D's (key, value) pairs, as 2-tuples
 |  
 |  iteritems(...)
 |      D.iteritems() -> an iterator over the (key, value) items of D
 |  
 |  iterkeys(...)
 |      D.iterkeys() -> an iterator over the keys of D
 |  
 |  itervalues(...)
 |      D.itervalues() -> an iterator over the values of D
 |  
 |  keys(...)
 |      D.keys() -> list of D's keys
 |  
 |  pop(...)
 |      D.pop(k[,d]) -> v, remove specified key and return the corresponding value.
 |      If key is not found, d is returned if given, otherwise KeyError is raised
 |  
 |  popitem(...)
 |      D.popitem() -> (k, v), remove and return some (key, value) pair as a
 |      2-tuple; but raise KeyError if D is empty.
 |  
 |  setdefault(...)
 |      D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
 |  
 |  values(...)
 |      D.values() -> list of D's values
 |  
 |  viewitems(...)
 |      D.viewitems() -> a set-like object providing a view on D's items
 |  
 |  viewkeys(...)
 |      D.viewkeys() -> a set-like object providing a view on D's keys
 |  
 |  viewvalues(...)
 |      D.viewvalues() -> an object providing a view on D's values
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes inherited from __builtin__.dict:
 |  
 |  __hash__ = None
 |  
 |  __new__ = <built-in method __new__ of type object>
 |      T.__new__(S, ...) -> a new object with type S, a subtype of T
 |  
 |  ----------------------------------------------------------------------
 |  Methods inherited from PlotlyBase:
 |  
 |  to_graph_objs(self, **kwargs)
 |      Everything is cast into graph_objs. Here for backwards compat.
 |  
 |  validate(self)
 |      Everything is *always* validated now. keep for backwards compat.

In [ ]:

# Get stream id from stream id list 
stream_id = stream_ids[0]

# Make instance of stream id object 
stream = Stream(
    token=stream_id,  # (!) link stream id to 'token' key
    maxpoints=80      # (!) keep a max of 80 pts on screen
)

The 'maxpoints' key set the maxiumum number of points to keep on the plot from an incoming stream.

Streaming Plotly plots are initialized with a standard (i.e. REST API) call to py.plot() or py.iplot() that embeds your unique stream ids in each of the plot's traces.

Each Plotly trace object (e.g. Scatter, Bar, Histogram, etc. More in Section 0) has a 'stream' key made available to link the trace object in question to a corresponding stream object.

In our first example, we link a scatter trace object to the stream:

In [6]:

# Initialize trace of streaming plot by embedding the unique stream_id
trace1 = Scatter(
    x=[],
    y=[],
    mode='lines+markers',
    stream=stream         # (!) embed stream id, 1 per trace
)

data = Data([trace1])

Then, add a title to the layout object and initialize your Plotly streaming plot:

In [7]:

# Add title to layout object
layout = Layout(title='Time Series')

# Make a figure object
fig = Figure(data=data, layout=layout)

# (@) Send fig to Plotly, initialize streaming plot, open new tab
unique_url = py.plot(fig, filename='s7_first-stream')

Great! Your Plotly streaming plot is intialized. Here's a screenshot:

Now, let's add data on top of it, or more precisely, send a stream of data to it.

So, first

In [8]:

help(py.Stream)  # run help() of the Stream link object

Help on class Stream in module plotly.plotly.plotly:

class Stream
 |  Interface to Plotly's real-time graphing API.
 |  
 |  Initialize a Stream object with a stream_id
 |  found in https://plotly.com/settings.
 |  Real-time graphs are initialized with a call to `plot` that embeds
 |  your unique `stream_id`s in each of the graph's traces. The `Stream`
 |  interface plots data to these traces, as identified with the unique
 |  stream_id, in real-time.
 |  Every viewer of the graph sees the same data at the same time.
 |  
 |  View examples and tutorials here:
 |  https://plotly.com/python/streaming/
 |  
 |  Stream example:
 |  # Initialize a streaming graph
 |  # by embedding stream_id's in the graph's traces
 |  import plotly.plotly as py
 |  from plotly.graph_objs import Data, Scatter, Stream
 |  stream_id = "your_stream_id" # See https://plotly.com/settings
 |  py.plot(Data([Scatter(x=[], y=[],
 |                        stream=Stream(token=stream_id, maxpoints=100))]))
 |  # Stream data to the import trace
 |  stream = Stream(stream_id) # Initialize a stream object
 |  stream.open() # Open the stream
 |  stream.write(dict(x=1, y=1)) # Plot (1, 1) in your graph
 |  
 |  Methods defined here:
 |  
 |  __init__(self, stream_id)
 |      Initialize a Stream object with your unique stream_id.
 |      Find your stream_id at https://plotly.com/settings.
 |      
 |      For more help, see: `help(plotly.plotly.Stream)`
 |      or see examples and tutorials here:
 |      https://plotly.com/python/streaming/
 |  
 |  close(self)
 |      Close the stream connection to plotly's streaming servers.
 |      
 |      For more help, see: `help(plotly.plotly.Stream)`
 |      or see examples and tutorials here:
 |      https://plotly.com/python/streaming/
 |  
 |  heartbeat(self, reconnect_on=(200, '', 408))
 |      Keep stream alive. Streams will close after ~1 min of inactivity.
 |      
 |      If the interval between stream writes is > 30 seconds, you should
 |      consider adding a heartbeat between your stream.write() calls like so:
 |      >>> stream.heartbeat()
 |  
 |  open(self)
 |      Open streaming connection to plotly.
 |      
 |      For more help, see: `help(plotly.plotly.Stream)`
 |      or see examples and tutorials here:
 |      https://plotly.com/python/streaming/
 |  
 |  write(self, trace, layout=None, validate=True, reconnect_on=(200, '', 408))
 |      Write to an open stream.
 |      
 |      Once you've instantiated a 'Stream' object with a 'stream_id',
 |      you can 'write' to it in real time.
 |      
 |      positional arguments:
 |      trace - A valid plotly trace object (e.g., Scatter, Heatmap, etc.).
 |              Not all keys in these are `stremable` run help(Obj) on the type
 |              of trace your trying to stream, for each valid key, if the key
 |              is streamable, it will say 'streamable = True'. Trace objects
 |              must be dictionary-like.
 |      
 |      keyword arguments:
 |      layout (default=None) - A valid Layout object
 |                              Run help(plotly.graph_objs.Layout)
 |      validate (default = True) - Validate this stream before sending?
 |                                  This will catch local errors if set to True.
 |      
 |      Some valid keys for trace dictionaries:
 |          'x', 'y', 'text', 'z', 'marker', 'line'
 |      
 |      Examples:
 |      >>> write(dict(x=1, y=2))  # assumes 'scatter' type
 |      >>> write(Bar(x=[1, 2, 3], y=[10, 20, 30]))
 |      >>> write(Scatter(x=1, y=2, text='scatter text'))
 |      >>> write(Scatter(x=1, y=3, marker=Marker(color='blue')))
 |      >>> write(Heatmap(z=[[1, 2, 3], [4, 5, 6]]))
 |      
 |      The connection to plotly's servers is checked before writing
 |      and reconnected if disconnected and if the response status code
 |      is in `reconnect_on`.
 |      
 |      For more help, see: `help(plotly.plotly.Stream)`
 |      or see examples and tutorials here:
 |      http://nbviewer.ipython.org/github/plotly/python-user-guide/blob/master/s7_streaming/s7_streaming.ipynb

In [ ]:

# (@) Make instance of the `Stream link` object 
#     with the same stream id as  the `Stream Id` object
s = py.Stream(stream_id)

# (@) Open the stream
s.open()

We can now use the Stream Link object s in order to stream data to our plot.

As an example, we will send a time stream and some random numbers(for 200 iterations):

In [10]:

# (*) Import module keep track and format current time
import datetime 
import time   
 
i = 0    # a counter
k = 5    # some shape parameter
N = 200  # number of points to be plotted

# Delay start of stream by 5 sec (time to switch tabs)
time.sleep(5) 

while i<N:
    i += 1   # add to counter
    
    # Current time on x-axis, random numbers on y-axis
    x = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')
    y = (np.cos(k*i/50.)*np.cos(i/50.)+np.random.randn(1))[0] 
    
    # (-) Both x and y are numbers (i.e. not lists nor arrays)
    
    # (@) write to Plotly stream!
    s.write(dict(x=x, y=y))  
    
    # (!) Write numbers to stream to append current data on plot,
    #     write lists to overwrite existing data on plot (more in 7.2).
            
    time.sleep(0.08)  # (!) plot a point every 80 ms, for smoother plotting
    
# (@) Close the stream when done plotting
s.close() 

A stream of data totalling 200 points is sent to Plotly's servers in real-time.

Watching this unfold in an opened plot.ly tab looks something like (in the old UI):

In [11]:

from IPython.display import YouTubeVideo
YouTubeVideo('OVQ2Guypp_M', width='100%', height='350')

Out[11]:

It took Plotly around 15 seconds to plot the 200 data points sent in the data stream. After that, the generated plot looks like any other Plotly plot.

With that said, if you have enough computer resources to let a server run indefinitely, why not have

>>> while True:

as the while-loop expression and never close the stream.

Luckily, it turns out that Plotly has access to such computer resources; a simulation generated using the same code as the above has been running since March 2014.

This plot is embedded below:

In [12]:

# Embed never-ending time series streaming plot
tls.embed('streaming-demos','12')

# Note that the time point correspond to internal clock of the servers, 
# that is UTC time.

Out[12]:

Anyone can view your streaming graph in real-time. All viewers will see the same data simultaneously (try it! Open up this notebook up in two different browser windows and observer that the graphs are plotting identical data!).

Simply put, Plotly's streaming API is awesome!

In brief: to make a Plotly streaming plot:

Make a stream id object (Stream in the plotly.graph_objs module) containing the stream id(which is found in the settings of your Plotly account) and the maximum number of points to be keep on screen (which is optional).
Provide the stream id object as the key value for the stream attribute in your trace object.
Make a stream link object (py.Stream) containing the same stream id as the stream id object and open the stream with the .open() method.
Write data to the stream link object with the .write() method. When done, close the stream with the .close() method.

Here are the links to the subsections' notebooks:

Got Questions or Feedback?

Reach us here at: Plotly Community

What's going on at Plotly?

Check out our twitter: @plotlygraphs

In [2]:

from IPython.display import display, HTML

display(HTML('<link href="//fonts.googleapis.com/css?family=Open+Sans:600,400,300,200|Inconsolata|Ubuntu+Mono:400,700" rel="stylesheet" type="text/css" />'))
display(HTML('<link rel="stylesheet" type="text/css" href="http://help.plot.ly/documentation/all_static/css/ipython-notebook-custom.css">'))

! pip install publisher --upgrade
import publisher
publisher.publish(
    's7_streaming_p1-first-stream', 'python/streaming_part1//', 'Getting Started with Plotly Streaming',
    'Getting Started with Plotly Streaming',
    title = 'Getting Started with Plotly Streaming',
    thumbnail='', language='python',
    layout='user-guide', has_thumbnail='false') 

Requirement already up-to-date: publisher in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

In [ ]: