#!/usr/bin/env python # coding: utf-8 # **Note**: Click on "*Kernel*" > "*Restart Kernel and Clear All Outputs*" in [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) *before* reading this notebook to reset its output. If you cannot run this file on your machine, you may want to open it [in the cloud

](https://mybinder.org/v2/gh/webartifex/intro-to-python/develop?urlpath=lab/tree/11_classes/04_content.ipynb). # # Chapter 11: Classes & Instances (continued) # In this fourth part of the chapter, we finalize our `Vector` and `Matrix` classes. As both `class` definitions have become rather lengthy, we learn how we to organize them into a Python package and import them in this Jupyter notebook. # ## Packages vs. Modules # In [Chapter 2

](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/02_functions/02_content.ipynb#Local-Modules-and-Packages), we introduce the concept of a Python module that is imported with the `import` statement. Essentially, a **module** is a single plain text \*.py file on disk that contains Python code (e.g., [*sample_module.py*

](https://github.com/webartifex/intro-to-python/blob/develop/02_functions/sample_module.py) in [Chapter 2's folder

](https://github.com/webartifex/intro-to-python/tree/develop/02_functions)). # # Conceptually, a **package** is a generalization of a module whose code is split across several \*.py to achieve a better organization of the individual parts. The \*.py files are stored within a folder (e.g., [*sample_package*

](https://github.com/webartifex/intro-to-python/tree/develop/11_classes/sample_package) in [Chapter 11's folder

](https://github.com/webartifex/intro-to-python/tree/develop/11_classes)). In addition to that, a "*\_\_init\_\_.py*" file that may be empty must be put inside the folder. The latter is what the Python interpreter looks for to decide if a folder is a package or not. # # Let's look at an example with the final version of our `Vector` and `Matrix` classes. # # `!pwd` shows the location of this Jupyter notebook on the computer you are running [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) on: It is the local equivalent of [Chapter 11's folder

](https://github.com/webartifex/intro-to-python/tree/develop/11_classes) in this book's [GitHub repository

](https://github.com/webartifex/intro-to-python). # In[1]: get_ipython().system('pwd') # `!ls` lists all the files and folders in the current location: These are Chapter 11's Jupyter notebooks (i.e., the \*.ipynb files) and the [*sample_package*

](https://github.com/webartifex/intro-to-python/tree/develop/11_classes/sample_package) folder. # In[2]: get_ipython().system('ls') # If we run `!ls` with the `sample_package` folder as the argument, we see the folder's contents: Four \*.py files. Alternatively, you can use [JupyterLab' File Browser](https://jupyterlab.readthedocs.io/en/stable/user/interface.html?highlight=file%20browser#left-sidebar) on the left to navigate into the package. # In[3]: get_ipython().system('ls sample_package') # The package is organized such that the [*matrix.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py) and [*vector.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) modules each define just one class, `Matrix` and `Vector`. That is intentional as both classes consist of several hundred lines of code and comments. # # The [*utils.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py) module contains code that is shared by both classes. Such code snippets are commonly called "utilities" or "helpers," which explains the module's name. # # Finally, the [*\_\_init\_\_.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file contains mostly meta information and defines what objects should be importable from the package's top level. # # With the `import` statement, we can import the entire package just as we would import a module from the [standard library

](https://docs.python.org/3/library/index.html). # In[4]: import sample_package as pkg # The above cell runs the code in the [*\_\_init\_\_.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file from top to bottom, which in turn runs the [*matrix.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py), [*utils.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py), and [*vector.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) modules (cf., look at the `import` statements in the four \*.py files to get the idea). As both [*matrix.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py) and [*vector.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) depend on each other (i.e., the `Matrix` class needs the `Vector` class to work and vice versa), understanding the order in that the modules are executed is not trivial. Without going into detail, we mention that Python guarantees that each \*.py file is run only once and figures out the order on its own. If Python is unable to do that, for example, due to unresolvable cirular imports, it aborts with an `ImportError`. # # Below, `pkg` is an object of type `module` ... # In[5]: pkg # In[6]: type(pkg) # ... and we use the built-in [dir()

](https://docs.python.org/3/library/functions.html#dir) function to check what attributes `pkg` comes with. # In[7]: dir(pkg) # The package's meta information and documentation are automatically parsed from the [*\_\_init\_\_.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file. # In[8]: help(pkg) # The meta information could also be accessed separately and individually. # In[9]: pkg.__name__ # In[10]: pkg.__version__ # follows the semantic versioning format # We create `Vector` and `Matrix` instances in the usual way by calling the `Vector` and `Matrix` classes from the package's top level. # In[11]: pkg.Vector([1, 2, 3]) # In[12]: pkg.Matrix([(1, 2, 3), (4, 5, 6), (7, 8, 9)]) # A common practice by package authors is to put all the objects on the package's top level that they want the package users to work with directly. That is achieved via the `import` statements in the [*\_\_init\_\_.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file. # # However, users can always reach into a package and work with its internals. # # For example, the `Vector` and `Matrix` classes are also available via their **qualified name** (cf., [PEP 3155

](https://www.python.org/dev/peps/pep-3155/)): First, we access the `vector` and `matrix` modules on `pkg`, and then the `Vector` and `Matrix` classes on the modules. # In[13]: pkg.vector.Vector # In[14]: pkg.matrix.Matrix # Also, let's import the [*utils.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py) module with the `norm()` function into the global scope. As this function is integrated into the `Vector.__abs__()` and `Matrix.__abs__()` methods, there is actually no need to work with it explicitly. # In[15]: from sample_package import utils # In[16]: help(utils.norm) # Many tutorials on the internet begin by importing "everything" from a package into the global scope with `from ... import *`. # # That is commonly considered a *bad* practice as it may overwrite already existing variables. However, if the package's [*\_\_init\_\_.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file defines an `__all__` attribute, a `list` with all the names to be "exported," the **star import** is safe to be used, in particular, in *interactive* sessions like Jupyter notebooks. We emphasize that the star import should *not* be used *within* packages and modules as then it is not directly evident from a name where the corresponding object is defined. # # For more best practices regarding importing we refer to, among others, [Google's Python Style Guide](https://google.github.io/styleguide/pyguide.html#22-imports). # # The following `import` statement makes the `Vector` and `Matrix` classes available in the global scope. # In[17]: from sample_package import * # In[18]: Vector # In[19]: Matrix # For further information on modules and packages, we refer to the [official tutorial

](https://docs.python.org/3/tutorial/modules.html). # ## The final `Vector` and `Matrix` Classes # The final implementations of the `Vector` and `Matrix` classes are in the [*matrix.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py) and [*vector.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) files: They integrate all of the functionalities introduced in this chapter. In addition, the code is cleaned up and fully documented, including examples of common usages. # # We strongly suggest the eager student go over the files in the [*sample_package*

](https://github.com/webartifex/intro-to-python/tree/develop/11_classes/sample_package) in detail at some point to understand what well-written and (re-)usable code looks like. # In[20]: v = Vector([1, 2, 3]) # In[21]: v # In[22]: type(v) # In[23]: m = Matrix([(1, 2, 3), (4, 5, 6), (7, 8, 9)]) # In[24]: m # In[25]: type(m) # Furthermore, the classes are designed for easier maintenence in the long-run. # # For example, the `Matrix/Vector.storage` and `Matrix/Vector.typing` class attributes replace the "hard coded" [tuple()

](https://docs.python.org/3/library/functions.html#func-tuple) and [float()

](https://docs.python.org/3/library/functions.html#float) built-ins in the `.__init__()` methods: As `self.storage` and `self.typing` are not defined on the *instances*, Python automatically looks them up on the *classes*. # In[26]: Vector.storage # In[27]: Vector.typing # In[28]: get_ipython().run_line_magic('pinfo2', 'Vector.__init__') # Both `Matrix/Vector.storage` and `Matrix/Vector.typing` themselves reference the `DEFAULT_ENTRIES_STORAGE` and `DEFAULT_ENTRY_TYPE` constants in the [*utils.py*

](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py) module. This way, we could, for example, change only the constants and thereby also change how the `._entries` are stored internally in both classes. Also, this single **[single source of truth

](https://en.wikipedia.org/wiki/Single_source_of_truth)** ensures that both classes are consistent with each other at all times. # In[29]: utils.DEFAULT_ENTRIES_STORAGE # In[30]: utils.DEFAULT_ENTRY_TYPE # For the same reasons, we also replace the "hard coded" references to the `Vector` and `Matrix` classes within the various methods. # # Every instance object has an automatically set `.__class__` attribute referencing its class. # In[31]: m.__class__ # Of course, we could also use the [type()

](https://docs.python.org/3/library/functions.html#type) built-in instead. # In[32]: type(m) # So, for example, the `Matrix.transpose()` method makes a `self.__class__(...)` instead of a `Matrix(...)` call. # In[33]: get_ipython().run_line_magic('pinfo2', 'Matrix.transpose') # Whenever we need a `str` representation of a class's name, we use the `.__name__` attribute on the class, ... # In[34]: Matrix.__name__ # ... or access it via the `.__class__` attribute on an instance. # In[35]: m.__class__.__name__ # For example, the `.__repr__()` and `.__str__()` methods make use of that. # In[36]: get_ipython().run_line_magic('pinfo2', 'Matrix.__repr__') # In order to not have to "hard code" the name of *another* class (e.g., the `Vector.as_matrix()` method references the `Matrix` class), we apply the following "hack:" First, we store a reference to the other class as a class attribute (e.g., `Matrix.vector_cls` and `Vector.matrix_cls`), and then reference that attribute within the methods, just like `.storage` and `.typing` above. # In[37]: Matrix.vector_cls # In[38]: Vector.matrix_cls # As an example, the `Vector.as_matrix()` method makes a `self.matrix_cls(...)` instead of a `Matrix(...)` call. # In[39]: get_ipython().run_line_magic('pinfo2', 'Vector.as_matrix') # For completeness sake, we mention that in the final `Vector` and `Matrix` classes, the `.__sub__()` and `.__rsub__()` methods use the negation operator implemented in `.__neg__()` and then dispatch to `.__add__()` instead of implementing the subtraction logic themselves. # ## "Real-life" Experiment # Let's do some math with bigger `Matrix` and `Vector` instances. # In[40]: import random # In[41]: random.seed(42) # We initialize `m` as a $100x50$ dimensional `Matrix` with random numbers in the range between `0` and `1_000`. # In[42]: m = Matrix((1_000 * random.random() for _ in range(50)) for _ in range(100)) # We quickly lose track with all the numbers in the `Matrix`, which is why we implemented the `__str__()` method as a summary representation. # In[43]: m # In[44]: print(m) # Similarily, `v` is now a `Vector` with $50$ entries. # In[45]: v = Vector(1_000 * random.random() for _ in range(50)) # In[46]: v # In[47]: print(v) # The arithmetic works as before. # In[48]: w = m * v # In[49]: print(w) # We can multiply `m` with its transpose or the other way round. # In[50]: n = m * m.transpose() # In[51]: print(n) # In[52]: o = m.transpose() * m # In[53]: print(o) # ## Comparison with [numpy](https://www.numpy.org/) # We started out in this chapter by realizing that Python provides us no good data type to model a vector $\vec{x}$ or a matrix $\bf{A}$. Then, we built up two custom data types, `Vector` and `Matrix`, that wrap a simple `tuple` object for $\vec{x}$ and a `tuple` of `tuple`s for $\bf{A}$ so that we can interact with their `._entries` in a "natural" way, which is similar to how we write linear algebra tasks by hand. By doing this, we extend Python with our own little "dialect" or **[domain-specific language

](https://en.wikipedia.org/wiki/Domain-specific_language)** (DSL). # # If we feel like sharing our linear algebra library with the world, we could easily do so on either [GitHub

](https://github.com) or [PyPI](https://pypi.org). However, for the domain of linear algebra this would be rather pointless as there is already a widely adopted library with [numpy](https://www.numpy.org/) that not only has a lot more features than ours but also is implemented in C, which makes it a lot faster with big data. # # Let's model the example in the [first part

](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/11_classes/00_content.ipynb#Example:-Vectors-&-Matrices) with both [numpy](https://www.numpy.org/) and our own DSL and compare them. # In[54]: x = (1, 2, 3) A = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] # In[55]: A * x # The creation of vectors and matrices is similar to our DSL. However, numpy uses the more general concept of an **n-dimensional array** (i.e., the `ndarray` type) where a vector is only a special case of a matrix and a matrix is yet another special case of an even higher dimensional structure. # In[56]: import numpy as np # In[57]: x_arr = np.array(x) A_arr = np.array(A) # In[58]: x_vec = Vector(x) A_mat = Matrix(A) # The text representations are very similar. However, [numpy](https://www.numpy.org/)'s `ndarray`s keep the entries as `int`s while our `Vector` and `Matrix` objects contain `float`s. # In[59]: x_arr # In[60]: x_vec # In[61]: A_arr # In[62]: A_mat # [numpy](https://www.numpy.org/)'s `ndarray`s come with a `.shape` instance attribute that returns a `tuple` with the dimensions ... # In[63]: x_arr.shape # In[64]: A_arr.shape # ... while `Matrix` objects come with `.n_rows` and `.n_cols` properties. # In[65]: A_mat.n_rows, A_mat.n_cols # The built-in [len()

](https://docs.python.org/3/library/functions.html#len) function does not return the number of entries in an `ndarray` but the number of the rows instead. This is equivalent to the first element in the `.shape` attribute. # In[66]: len(x_arr) # In[67]: len(x_vec) # In[68]: len(A_arr) # In[69]: len(A_mat) # The `.transpose()` method also exists for `ndarray`s. # In[70]: A_arr.transpose() # In[71]: A_mat.transpose() # To perform matrix-matrix, matrix-vector, or vector-matrix multiplication in [numpy](https://www.numpy.org/), we use the `.dot()` method. If we use the `*` operator with `ndarray`s, an *entry-wise* multiplication is performed. # In[72]: A_arr.dot(x_arr) # In[73]: A_arr * x_arr # In[74]: A_mat * x_vec # Scalar multiplication, however, works as expected. # In[75]: 10 * x_arr # In[76]: 10 * x_vec # Because we implemented our classes to support the sequence protocol, [numpy](https://www.numpy.org/)'s *one*-dimensional `ndarray`s are actually able to work with them: The `*` operator is applied on a per-entry basis. # In[77]: x_arr + x_vec # In[78]: x_arr * x_vec # In[79]: A_arr + A_mat # We conclude that it is rather easy to extend Python in a way that makes the resulting application code read like core Python again. As there are many well established third-party packages out there, it is unlikely that we have to implement a fundamental library ourselves. Yet, we can apply the concepts introduced in this chapter to organize the code in the applications we write.