Jan 17, 2020
This Python and Numpy turorial (part II - Numpy
) was adapted by Badri Adhikari from the original adaptation by Volodymyr Kuleshov and Isaac Caswell from the CS231n
Python tutorial by Justin Johnson (http://cs231n.github.io/python-numpy-tutorial/). Expanded the broadcasting section by adapting this tutorial.
Numpy documentation.
To use Numpy, we first need to import the numpy
package:
import numpy as np
x = [[2,3][4,5]]
print( x[0] )
Caution! Python arrays and Numpy arrays are different objects!
For example, type
only works with Python data structures shape
works with numpy arrays.
a = np.array([1, 2, 3]) # Create a rank 1 array
print(type(a), a.shape, a[0], a[1], a[2])
a[0] = 's' # Change an element of the array
print(a)
<class 'numpy.ndarray'> (3,) 1 2 3
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-90-c0c003a6e1a8> in <module>() 1 a = np.array([1, 2, 3]) # Create a rank 1 array 2 print(type(a), a.shape, a[0], a[1], a[2]) ----> 3 a[0] = 's' # Change an element of the array 4 print(a) ValueError: invalid literal for int() with base 10: 's'
b = np.array([[1,2,3],[4,5,6]]) # Create a rank 2 array
print(b)
[[1 2 3] [4 5 6]]
print(b.shape)
print(b[0, 0], b[0, 1], b[1, 0])
(2, 3) 1 2 4
"shape
" is your hammer!
Numpy also provides many functions to create arrays:
a = np.zeros((2,2)) # Create an array of all zeros
print(a)
[[0. 0.] [0. 0.]]
b = np.ones((1,2, 5, 6, 7)) # Create an array of all ones
print(b)
[[[[[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]]] [[[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]] [[1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1.]]]]]
c = np.full((2,2), 7) # Create a constant array
print(c)
[[7 7] [7 7]]
d = np.eye(3) # Create a 2x2 identity matrix
print(d)
[[1. 0. 0.] [0. 1. 0.] [0. 0. 1.]]
e = np.random.random((2,2)) * 100 # Create an array filled with random values
print(e)
[[76.37361261 20.04861709] [97.33716423 67.42584384]]
Here is an example matrix of shape (3, 4)
(3 rows and 4 columns) and rank 2
:
1 | 2 | 3 | 4 |
5 | 6 | 7 | 8 |
9 | 10 | 11 | 12 |
import numpy as np
a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
# Use slicing to pull out the subarray consisting of the first 2 rows and columns 1 and 2
b = a[:2, 1:3]
print( b )
[[2 3] [6 7]]
A slice of an array is a view into the same data, so modifying it will modify the original array.
1 | 2 | 3 | 4 |
5 | 6 | 7 | 8 |
9 | 10 | 11 | 12 |
b[0, 0] = 77 # b[0, 0] is the same piece of data as a[0, 1]
print( a)
[[ 1 77 3 4] [ 5 6 7 8] [ 9 10 11 12]]
Then, how can we make a copy (not reference) of an array?
c = np.copy(a)
c[0, 0] = 100
print(c)
print(a)
[[100 77 3 4] [ 5 6 7 8] [ 9 10 11 12]] [[ 1 77 3 4] [ 5 6 7 8] [ 9 10 11 12]]
You can also mix integer indexing with slice indexing. However, doing so will yield an array of lower rank than the original array.
# Create the following rank 2 array with shape (3, 4)
a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
print(a)
[[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12]]
Two ways of accessing the data in the middle row of the array:
1 | 2 | 3 | 4 |
5 | 6 | 7 | 8 |
9 | 10 | 11 | 12 |
row_r1 = a[1, :] # Rank 1 view of the second row of a
row_r2 = a[1:2, :] # Rank 2 view of the second row of a
row_r3 = a[[1], :] # Rank 2 view of the second row of a
print( row_r1, row_r1.shape )
print( row_r2, row_r2.shape )
print( row_r3, row_r3.shape )
[5 6 7 8] (4,) [[5 6 7 8]] (1, 4) [[5 6 7 8]] (1, 4)
# We can make the same distinction when accessing columns of an array:
col_r1 = a[:, 1]
col_r2 = a[:, 1:2]
print( col_r1, col_r1.shape )
print( col_r2, col_r2.shape )
[ 2 6 10] (3,) [[ 2] [ 6] [10]] (3, 1)
Integer array indexing
When you index into numpy arrays using slicing, the resulting array view will always be a subarray of the original array. In contrast, integer array indexing allows you to construct arbitrary arrays using the data from another array. Here is an example:
a = np.array([[1,2], [3, 4], [5, 6]])
print ('a = ')
print(a)
a = [[1 2] [3 4] [5 6]]
# An example of integer array indexing.
# The returned array will have shape (2,)
print (' => ')
print( a[[0, 1], [0, 1]] )
=> [1 4]
# The returned array will have shape (3,)
print (' => ')
print( a[[0, 1, 2], [0, 1, 0]] )
=> [1 4 5]
# The above example of integer array indexing is equivalent to this:
print (' => ')
print( np.array([a[0, 0], a[1, 1], a[2, 0]]) )
=> [1 4 5]
# When using integer array indexing, you can reuse the same
# element from the source array:
print(a[[0, 0], [1, 1]])
# Equivalent to the previous integer array indexing example
print(np.array([a[0, 1], a[0, 1]]))
[2 2] [2 2]
One useful trick with integer array indexing is selecting or mutating one element from each row of a matrix:
# Create a new array from which we will select elements
a = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])
print(a)
[[ 1 2 3] [ 4 5 6] [ 7 8 9] [10 11 12]]
# Create an array of indices
b = np.array([0, 2, 0, 1])
print(b)
[0 2 0 1]
print(np.arange(4))
# Select one element from each row of a using the indices in b
print( a[np.arange(4), b] ) # Prints "[ 1 6 7 11]"
[0 1 2 3] [ 1 6 7 11]
# Mutate one element from each row of a using the indices in b
a[np.arange(4), b] += 10
print(a)
[[11 2 3] [ 4 5 16] [17 8 9] [10 21 12]]
Boolean array indexing
Boolean array indexing lets you pick out arbitrary elements of an array. Frequently this type of indexing is used to select the elements of an array that satisfy some condition. Here is an example:
a = np.array([[1,2], [3, 4], [5, 6]])
print(a)
[[1 2] [3 4] [5 6]]
bool_idx = (a > 2) # Find the elements of a that are bigger than 2;
# this returns a numpy array of Booleans of the same
# shape as a, where each slot of bool_idx tells
# whether that element of a is > 2.
print(bool_idx)
[[False False] [ True True] [ True True]]
# We use boolean array indexing to construct a "rank 1 array"
# consisting of the elements of a corresponding to the True values
# of bool_idx
print(a[bool_idx])
[3 4 5 6]
# We can do all of the above in a single concise statement:
print(a[a > 2])
[3 4 5 6]
x = np.array([1, 2]) # Let numpy choose the datatype
y = np.array([1.0, 2.0]) # Let numpy choose the datatype
z = np.array([1, 2], dtype = np.int32) # Force a particular datatype
print( x.dtype, y.dtype, z.dtype )
int64 float64 int32
Datatypes documentation.
x = np.array([[1,2],[3,4]], dtype = np.float64)
y = np.array([[5,6],[7,8]], dtype = np.float64)
# Elementwise sum; both produce the array
print(x + y) # operator overload
print(np.add(x, y)) # add available as function
[[ 6. 8.] [10. 12.]] [[ 6. 8.] [10. 12.]]
# Elementwise difference; both produce the array
print(x - y)
print(np.subtract(x, y))
[[-4. -4.] [-4. -4.]] [[-4. -4.] [-4. -4.]]
# Elementwise product; both produce the array
print(x * y)
print(np.multiply(x, y))
[[ 5. 12.] [21. 32.]] [[ 5. 12.] [21. 32.]]
# Elementwise division; both produce the array
print(x / y)
print(np.divide(x, y))
[[0.2 0.33333333] [0.42857143 0.5 ]] [[0.2 0.33333333] [0.42857143 0.5 ]]
# Elementwise square root; produces the array
print(np.sqrt(x))
[[1. 1.41421356] [1.73205081 2. ]]
Elementwise Multiplication vs Dot Product:
*
is elementwise multiplication, not matrix multiplicationdot
function to compute inner products of vectors, to multiply a vector by a matrix, and to multiply matricesdot
is available both as a function in the numpy module and as an instance method of array objectsDot Product
.dot()
function returns the dot product of two arraysExamples:
v = np.array([9,10])
w = np.array([11, 12])
# Inner product of vectors
print(v.dot(w))
print(np.dot(v, w))
219 219
x = np.array([[1,2],[3,4]])
# Matrix / vector product; both produce the rank 1 array
print(x.dot(v))
print(np.dot(x, v))
[29 67] [29 67]
y = np.array([[5,6],[7,8]])
# Matrix / matrix product; both produce the rank 2 array
print(x * y)
print(np.dot(x, y))
[[ 5 12] [21 32]] [[19 22] [43 50]]
Numpy provides many useful functions for performing computations on arrays; one of the most useful is sum
:
import numpy as np
x = np.array([[1,2],[3,4]])
print('x =')
print(x)
print('sum =')
print(np.sum(x)) # Compute sum of all elements
print('sum along axis 0 =')
print(np.sum(x, axis=0)) # Compute sum of each column
print('sum along axis 1 =')
print(np.sum(x, axis=1)) # Compute sum of each row
x = [[1 2] [3 4]] sum = 10 sum along axis 0 = [4 6] sum along axis 1 = [3 7]
To transpose a matrix, simply use the T attribute of an array object:
print(x)
print(x.T)
[[1 2] [3 4]] [[1 3] [2 4]]
Numpy arrays can also be reshaped using .reshape()
x = np.array([[1,2],[3,4]])
print(x.shape)
y = np.reshape(x, (1, 4))
print(y.shape)
print(y)
(2, 2) (1, 4) [[1 2 3 4]]
Mathematical functions documentation.
Suppose that we want to add a constant vector to each row of a matrix:
# We will add the vector v to each row of the matrix x,
# storing the result in the matrix y
x = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])
v = np.array([1, 0, 1])
y = np.empty_like(x) # Create an empty matrix with the same shape as x
print('x: ', x)
print('v: ', v)
x: [[ 1 2 3] [ 4 5 6] [ 7 8 9] [10 11 12]] v: [1 0 1]
We would like to add v
to x
. Here is a naive way of doing so (Method 1):
# Add the vector v to each row of the matrix x with an explicit loop
for i in range(4):
x[i, :] = x[i, :] + v
print(x)
[[ 2 2 4] [ 5 5 7] [ 8 8 10] [11 11 13]]
This works; however when the matrix x
is very large, computing an explicit loop in Python could be slow.
Note that adding the vector v to each row of the matrix x
is equivalent to forming a matrix vv
by stacking multiple copies of v
vertically, then performing elementwise summation of x
and vv
. We could implement this approach like this (Method 2):
vv = np.tile(v, (4, 1)) # Stack 4 copies of v on top of each other
print(vv)
[[1 0 1] [1 0 1] [1 0 1] [1 0 1]]
y = x + vv # Add x and vv elementwise
print(y)
[[ 3 2 5] [ 6 5 8] [ 9 8 11] [12 11 14]]
Numpy broadcasting allows us to perform this computation without actually creating multiple copies of v. Consider this version, using broadcasting (Method 3):
import numpy as np
# We will add the vector v to each row of the matrix x,
x = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])
v = np.array([1, 0, 1])
y = x + v # Add v to each row of x using broadcasting
print(x)
print('')
print(v)
print('')
print(y)
[[ 1 2 3] [ 4 5 6] [ 7 8 9] [10 11 12]] [1 0 1] [[ 2 2 4] [ 5 5 7] [ 8 8 10] [11 11 13]]
The simplest broadcasting example occurs when an array and a scalar value are combined in an operation:
import numpy as np
a = np.array([1.0, 2.0, 3.0])
b = 2.0
print(a * b)
[2. 4. 6.]
Explanation:
When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing dimensions, and works its way forward. Two dimensions are compatible for broadcasting when
If these conditions are not met, a ValueError: operands could not be broadcast together
exception is thrown, indicating that the arrays have incompatible shapes. The size of the resulting array is the maximum size along each dimension of the input arrays.
Arrays do not need to have the same number of dimensions. For example, if you have a 256x256x3 array of RGB values, and you want to scale each color in the image by a different value, you can multiply the image by a one-dimensional array with 3 values. Lining up the sizes of the trailing axes of these arrays according to the broadcast rules, shows that they are compatible:
Image (3d array): 256 x 256 x 3
Scale (1d array): 3
Result (3d array): 256 x 256 x 3
When either of the dimensions compared is one, the other is used. In other words, dimensions with size 1 are stretched or “copied” to match the other. Here, the scaling vector can be thought of having the dimensions 1 x 1 x 3
.
Examples where broadcasting works:
A (4d array): 8 x 1 x 6 x 1
B (3d array): 7 x 1 x 5
Result (4d array): 8 x 7 x 6 x 5
A (2d array): 5 x 4
B (1d array): 1
Result (2d array): 5 x 4
A (2d array): 5 x 4
B (1d array): 4
Result (2d array): 5 x 4
A (3d array): 15 x 3 x 5
B (3d array): 15 x 1 x 5
Result (3d array): 15 x 3 x 5
A (3d array): 15 x 3 x 5
B (2d array): 3 x 5
Result (3d array): 15 x 3 x 5
A (3d array): 15 x 3 x 5
B (2d array): 3 x 1
Result (3d array): 15 x 3 x 5
Examples of shapes that do not broadcast:
A (1d array): 3
B (1d array): 4 # trailing dimensions do not match
A (2d array): 2 x 1
B (3d array): 8 x 4 x 3 # second from last dimensions mismatched
Practice Example 1:
x = np.arange(4)
y = np.ones(5)
print(x, x.shape)
print(y, y.shape)
[0 1 2 3] (4,) [1. 1. 1. 1. 1.] (5,)
# What will be the output?
print((x + y).shape)
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-136-ba3d9005e40b> in <module>() ----> 1 print((x + y).shape) ValueError: operands could not be broadcast together with shapes (4,) (5,)
Practice Example 2:
xx = x.reshape(4,1)
print(xx, xx.shape)
print(y, y.shape)
[[0] [1] [2] [3]] (4, 1) [1. 1. 1. 1. 1.] (5,)
# What will be the output?
print((xx + y).shape)
(4, 5)
Broadcasting provides a convenient way of taking the outer product (or any other outer operation) of two arrays. The following example shows an outer product operation of two 1-d arrays:
import numpy as np
a = np.array([0.0, 10.0, 20.0, 30.0])
b = np.array([1.0, 2.0, 3.0])
print(a, a.shape)
print(b, b.shape)
[ 0. 10. 20. 30.] (4,) [1. 2. 3.] (3,)
print(a * b)
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-140-21f961b7a520> in <module>() ----> 1 print(a * b) ValueError: operands could not be broadcast together with shapes (4,) (3,)
#Here the newaxis index operator inserts a new axis into a, making it a two-dimensional 4x1 array
print( a * b.reshape(3,1) )
[[ 0. 10. 20. 30.] [ 0. 20. 40. 60.] [ 0. 30. 60. 90.]]
Functions that support broadcasting are known as universal functions. Here is the list.
Broadcasting documentation.