This notebook contains a demonstration of new features present in the 0.44.0 release of Numba. Whilst release notes are produced as part of the CHANGE_LOG
, there's nothing like seeing code in action!
🔥IMPORTANT: Numba is officially deprecating some features and behaviours in this release!🔥
More information about this is given below. _____
It is also worth noting that this release of Numba is now backed by LLVM 8 on all platforms except ppc64le
(due to numerous issues present solely on that platform). As a result of the ppc64le
issues, LLVM 7.0.x and 7.1.x are also still supported on all platforms. This release also fully accommodates new and changed functionality required to be compatible with NumPy 1.16.
Also included are demonstrations of:
First, import the necessary from Numba and NumPy...
from numba import jit, njit, config, __version__
from numba.extending import overload
import numpy as np
assert tuple(int(x) for x in __version__.split('.')[:2]) >= (0, 44)
Numba Version 0.44 is deprecating a number of features and issuing pending-deprecation notices for others. Extensive documentation including examples, recommendations and removal schedules can be found here. The deprecations/pending deprecations most likely to impact your code relate to:
NOTE: Suppressing these warnings is easy and explained along with example code here.
List
and Set
reflection¶Reflection is the jargon used in Numba to describe the process of ensuring that changes made by compiled code to arguments that are mutable Python container data types are visible in the Python interpreter when the compiled function returns. Numba has for some time supported reflection of list and set data types and it is support for this reflection that is scheduled for deprecation with view to replace with a better implementation.
Here is an example of code that would be impacted by such a change:
from numba import njit
@njit
def foo(x):
x.append(10) # changes made here need "reflecting" back to `a` in the outer scope
a = [1, 2, 3]
foo(a)
@jit
¶The numba.jit
decorator has for a long time followed the behaviour of first attempting to compile the decorated function in nopython
mode and should this compilation fail it will fall-back and try again to compile but this time in object mode. It it this fall-back behaviour which is being deprecated, the result of which will be that numba.jit
will by default compile in nopython
mode and object mode
compilation will become opt-in only.
Here is an example of code that would be impacted by such a change:
@jit # `nopython` mode was not explicitly requested, so fall-back is permitted
def bar():
l = []
for x in range(10):
l.append(x)
# no "reversed" support in nopython mode so compilation will fall-back to object mode
reversed(l)
bar()
Along with the above, some lesser used features and behaviours are being deprecated, namely:
For developers writing Numba extensions/leveraging the Numba internal APIs, the following deprecation is made:
Initial support for dictionaries was implemented in Version 0.43 via the "typed" dictionary. As a limitation of the initial work the key and value types had to be specified on construction. Numba Version 0.44 removes this restriction and implements type inference for the typed dictionary.
Specifying the type still works as before:
from numba.typed import Dict
from numba import int32, float32
a_dictionary = Dict.empty(int32, float32)
Type infererence also now works, whereby the type of the dictionary is inferred at compile time by how it is used:
@njit
def infer_dict():
d1 = {}
d1[10] = 's' # Numba works out that d1{} is equivalent to Dict.empty(int64, unicode_type)
return d1
d = infer_dict()
for k, v in d.items():
print("%s:%s" % (k, v))
from numba import typeof
print(typeof(d))
The curly brace initialisation syntax and the dict
reserved word are also both supported (Note: dict()
cannot take arguments yet).
@njit
def curly_braces_and_dict():
curly = {'a': 1.0, 'b': 2.0} # inferred as DictType[unicode_type,float64]
reserved = dict()
reserved[10] = np.zeros(5) # inferred as DictType[int64,array(float64, 1d, C)]
return curly, reserved
for x in curly_braces_and_dict():
print(typeof(x))
for k, v in x.items():
print(k, v)
This is an update to the snake graph example from the previous release notebook, making use of the dictionary type inference enhancement, also, Version 0.44 supports unicode string iteration, so the example now makes use of that too!
@njit
def snake_graph():
"""
This function creates a dictionary (str->int) and then randomly increases the integer
values and finally prints the results as a snake-y graph.
"""
# set up
x = dict() # <-- the type of x will be inferred
bins = "numba"
for v in bins: # <-- iterate a string directly
x[v] = 0 # <-- Numba works out that x is a DictType[unicode_type,int64]
# add values to bins at random
it = 100
for i in range(it):
key = bins[np.random.randint(len(bins))]
x[key] += 1
# iterate the key space and print a "bar" of snakes
for k in x.keys():
print(k + ':' + ''.join(['🐍' for _ in range(x[k])]))
return x
r = snake_graph()
r
A few more features were added to Numba's unicode support bringing complete support ever closer. The added string instance methods are:
.zfill()
.ljust()
.rjust()
.center()
.strip()
.lstrip()
.rstrip()
Support for string multiplication and iteration is also added. This example includes both the new dictionary features and new string features:
@njit
def decode():
char = '#'
w = 35
encode = [(0, 0, 0, False), (0, 0, 0, False), (17, 6, 11, False),
(13, 12, 9, True), (10, 15, 9, False), (8, 14, 12, False),
(6, 13, 15, False), (5, 26, 3, True), (3, 30, 1, True),
(3, 30, 1, True), (2, 31, 1, True), (3, 30, 1, True),
(3, 29, 2, True), (4, 27, 3, True), (8, 21, 5, True),
(19, 8, 7, False), (16, 9, 9, False), (14, 15, 5, False),
(18, 9, 7, False), (17, 7, 10, False), (16, 6, 12, False),
(15, 4, 15, False), (14, 3, 17, False), (13, 2, 19, False)]
slices = {9: ' ####### ################ ',
3: ' ###### #### ',
10: ' ####### ####################',
13: ' ########### ######## ',
14: '# ######## ',
11: ' #################### ########',
7: ' ########## ######',
8: ' ######### ############# ',
12: '################ ####### ',}
ret = dict() # newly supported dictionary use case
for ix, enc in enumerate(encode):
lpad, width, rpad, lookup = enc
if lookup:
if ix % 2:
s = slices[ix].strip()
else:
s = slices[ix].lstrip().rstrip()
buf = ' ' * lpad + s + ' ' * rpad # string multiplication
elif ix % 2:
buf = (char * width).rjust(lpad + width) # string rjust
else:
buf = ' ' * lpad + (char * width).ljust(lpad + rpad) # string ljust
ret[ix] = buf
title = ("Numba " + '.44'.zfill(4)).center(w, '~') # string zfill and center
newtitle = ""
for ix, t in enumerate(title): # string iteration
if t == '~':
if ix % 2:
newtitle += '⚡'
elif ix % 3:
newtitle += '🐍'
else:
newtitle += t
else:
newtitle += t
ret[0] = newtitle
return ret
for k, v in sorted(decode().items()):
print(k, v)
This release contains a number of newly supported NumPy functions:
np.repeat
np.delete
np.ndarray.shape
wiring for completeness: np.shape
np.quantile
and np.nanquantile
@njit
def numpy_new():
arr = np.array([[1,2],[3,4]])
# np.repeat
repeated = np.repeat(arr, 4)
print(repeated)
# np.delete
deleted = np.delete(repeated, 3)
print(deleted)
# np.shape
print(np.shape(deleted))
# np.quantile
print(np.quantile([1, 2, 3, 4], 0.5))
# np.nanquantile
print(np.nanquantile([np.nan, 1, 2, 3, 4], 0.5))
numpy_new()
Some new features were added that don't fit anywhere in particular but are still very useful. It is now possible to .view()
a scalar NumPy type as another NumPy type of the same width, which is particularly helpful if writing low level numerical routines.
import struct
@njit
def extract_fraction_float64(x):
""" extracts the fraction part of an IEEE754 double precision (float64) representation
"""
x_as_int = np.float64(x).view(np.uint64)
w = 64
frac_bits = 52
return np.uint64(x_as_int & (np.uint64(-1) >> w - frac_bits))
val = -np.pi
print("hex bits :", hex(struct.unpack('Q', struct.pack('d', val))[0]))
print("extracted fraction: ", hex(extract_fraction_float64(val)))
Also, max
and min
now work on iterables, a small but hugely useful addition e.g.
@njit
def foo():
x = [10, 3, -4]
print(min(x), max(x))
foo()