Extending Types by Subclassing¶

Allows you to customize or extend the behavior of built-in types with user-defined class

In [1]:

class MyList(list):
    def __str__(self):
        print("Content: ")
        return list.__str__(self)


ml = MyList([1, 2, 3])
print(ml)

Content: 
[1, 2, 3]

The "New Style" Class Model¶

In Python3, all classes are "new style"
In Python2, only explicitly inheritence from object would be regarded as "new style"

In [2]:

class C(object):
    pass

New-Style Class Changes¶

1. Attribute fetch for built-ins: instance skipped¶

__getattr__, __getattribute__ can no longer be called by other built-in operations for __X__
The search for such names begins at classes, not instances

In [3]:

class GetAttrTest(object):
    data = "spam"

    def __getattr__(self, attrname):
        print(attrname)
        return getattr(self.data, attrname)


g = GetAttrTest()

g.__add__ = lambda y: 88 + y
g + 1

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-4fc49c6ca986> in <module>()
      8 
      9 g.__add__ = lambda y: 88+y
---> 10 g + 1

TypeError: unsupported operand type(s) for +: 'GetAttrTest' and 'int'

Direct calls to built-in method names still work, but equivalent expression is not

In [4]:

# Direct calls still work

g.__add__(1)

Out[4]:

The effect¶

To code a proxy of an object whose interface may in part be invoked by built-in operations, new style classes require both --getattr__ for normal names, as well as method redefinitions for all names accessed by built-in operations

2. Classes and types merges¶

type(I) return the class an instance is made from, instead of generic instance type

In [5]:

type(5)

Out[5]:

int

type checking is ususally not recommended in Python
- isinstance might be prefered, but still not recommened

In [6]:

isinstance(5, (int, str))

Out[6]:

True

type and class hasve merged - type is a kind of object while object is a kind of type

In [7]:

isinstance(type, object)

Out[7]:

True

In [8]:

isinstance(object, type)

Out[8]:

True

3. Automatic object root class¶

A small set of default operator overloading method (e.g. __repr__)

4. Inheritance search order: MRO and diamonds (Also mentioned in Ch31)¶

Classic Class (Python2) -> DFLR
New-style Class (Python3) -> MRO
- New-Style class inheritance works the same for most other inheritance tree structures

Pros of MRO¶

Avoids visiting the same superclass more than once when it's accessible from multiple subclasses (performance optimization)
Without the new-style MRO, object would always override redefinitions in user-coded classes

In [9]:

%%python2
# To make this cell magic runs ok please ensure python2 can be run when you type python2 in terminal
# Reference: http://stackoverflow.com/questions/30201431/ipython-cell-magics

# DFLR: D -> B -> A -> C -> A

class A: attr =1
class B(A): pass
class C(A): attr = 2
class D(B, C): pass

x = D()
print(x.attr)   # x -> D -> B -> A

In [10]:

# MRO: D -> B -> C -> A

class A:
    attr = 1

class B(A):
    pass

class C(A):
    attr = 2

class D(B, C):
    pass

x = D()
print(x.attr)  # x -> D -> B -> C

You can simply resolve this by using attr = C.attr in in class D

In [11]:

%%python2

class A: attr =1
class B(A): pass
class C(A): attr = 2
class D(B, C): attr = C.attr

x = D()
print(x.attr)   # x -> D -> B -> A

`mro`¶

In [12]:

class A:
    attr = 1

class B(A):
    pass

class C(A):
    attr = 2

class D(B, C):
    pass

D.__mro__

Out[12]:

(__main__.D, __main__.B, __main__.C, __main__.A, object)

How MRO works¶

List all the classes using the classic class's DFLR and include a class multiple times if it's vistied more than once
Scan the resulting list for duplicate classes, remove all but the last occurrence of the duplicates in the list

5. Inhertitance algorithm¶

Will be mentioned in Ch40

6. New advanced tools¶

slot: Attribute Declarations¶

slots should be used only in applications that clearly warrant the added complexity

Basic Example¶

In [13]:

class limiter(object):
    __slots__ = ["age", "name", "job"]


x = limiter()
x.age

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-13-a33d945640ff> in <module>()
      3 
      4 x = limiter()
----> 5 x.age

AttributeError: age

In [14]:

x.age = 40  # must be assign first
print(x.age)

In [15]:

x.ape = 1000

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-15-483ef4f9f9f3> in <module>()
----> 1 x.ape = 1000

AttributeError: 'limiter' object has no attribute 'ape'

To save space, instead of allocating dictionary for each instance, Python reserves just enough space in each instance to hold a value for each slot attribute, along with inherited attributes in the common class to manage slot access
- Best reserved for rare cases where there are large number of instances in memory-critical applications
- In Python3.3, non-slots attribute space requirements have been reduced with a key-sharing dictionay model, where the __dict__ used for objects' attributes may share part of their internal storage, including that of their keys. This may lessen some of the value of __slots__ as an optimization tool

Classes with __slots__ do not have __dict__ by defualt

In [16]:

class C(object):
    __slots__ = ["a"]


c = C()
c.__dict__

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-16-4b9d1d768956> in <module>()
      3 
      4 c = C()
----> 5 c.__dict__

AttributeError: 'C' object has no attribute '__dict__'

In [17]:

dir(c)

Out[17]:

['__class__',
 '__delattr__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__gt__',
 '__hash__',
 '__init__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__slots__',
 '__str__',
 '__subclasshook__',
 'a']

However, __dict__ can still be included in __slots__

In [18]:

class C(object):
    __slots__ = ["a", "b", "__dict__"]
    d = 3

    def __init__(self):
        self.d = 5
        self.a = 1


c = C()
c.e = 5
c.__dict__

Out[18]:

{'d': 5, 'e': 5}

Without an attribute namespace dict, it's not possible to assign new names to instances that are not in slots list

In [19]:

class C(object):
    __slots__ = ["a", "b"]

    def __init__(self):
        self.d = 4


c = C()

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-19-f57b79b6a9a7> in <module>()
      3     def __init__(self):
      4         self.d = 4
----> 5 c = C()

<ipython-input-19-f57b79b6a9a7> in __init__(self)
      2     __slots__ = ['a', 'b']
      3     def __init__(self):
----> 4         self.d = 4
      5 c = C()

AttributeError: 'C' object has no attribute 'd'

Slot Usage Rules¶

Slots in subs are pointless when absent in supers
- __dict__ created for the superclass will always be accessible
- Subclass still manages its slots, but doesn't compute their values in anyway, and doesn't avoid a dict
Slots in supers are pointless when absent in subs
- Subclasses will produce and instacne __dict__
Redefintion renders super slots pointless
- A class defines the same slot name as a superclass, its redefinition hides the slot in superclass
Slots prevent class-level defaults
- Class attributes of the same name to provide default cannot be uesd
Slots and __dict__
- Slots preclude __dict__, unless it's listed explicitly

slots essentially require both universal and careful deployment to be effective
slots are not generally recommended, except in pathologicall cases where their space reduction is significant

Properties: Attribute Accessors¶

Similar to proerties in languages like Java and C#
proerties inercept accesss and compute values arbitrarily

Basics¶

A property is a type of object assigned to a class attribute name
By calling property and passing in up to three accessor methods (handlers for get, set and delete)
- If any of them is None or omitted, then that operation is not supported
- The result property object is typically assigned to a name at the top level of a class
- After thus assigned, accesses to the class property name are automatically routed to one of the accessor methods

In [20]:

class Proper(object):
    def getage(self):
        return 40

    age = property(getage, None, None, None)  # (get, set, del, docs)


p = Proper()
p.age

Out[20]:

`getattribute` and Descriptors¶

__getattribute__ is available for new-style classes only and used to intercept all attribute
- It's prone to loops much like __setattr__
Python supports the notion of attribute descriptors - classes with __get__ and __set__

In [21]:

class AgeDesc(object):
    def __get__(self, instance, owner):
        return 30

    def __set__(self, instance, value):
        instance._age = value


class D(object):
    age = AgeDesc()


d = D()
d.age

Out[21]:

In [22]:

d.age = 42
d._age

Out[22]:

Static and Class Methods¶

Static Methods¶

Static Methods in Python2 and Python3¶

To call a method without instance
- In Python2, staticmethod is a must
- In Python3, staticmethod is needed only if it would be called through an instance

In [23]:

# Works both Python2 and Python3


class C(object):
    @staticmethod
    def print_one():
        print(1)


c = C()
c.print_one()

In [24]:

# Python3 Only

class C(object):
    def print_one():
        print(1)

C.print_one()

Look at the code above.
Since print_one does not operate any class or instance data, it may be a good idea to make it static

Class Method¶

Methods of a class that are passed a class object in their first argument instead of an instance, regardless of whether they are called through an instance or a class
Receive the lowest class of the call's subject

In [25]:

class C(object):
    counter = 0

    @classmethod
    def cmethod(cls):
        cls.counter += 1
        print(cls.counter)


c1 = C()
c1.cmethod()

c2 = C()
c2.cmethod()

1
2

Static Method VS Class Method¶

static method: process data local to a class
class method: process data that may differ for each class in hierarchy
- code that needs to manage per-class instance counters, for example, might be best off leveragin class method

Why using static/class method instead of simple function?¶

Localized the function name in class scope
Move the function closer to where it's used
Allows subclasses to customize

Decorators and Metaclasses (Detail in Ch39, Ch40)¶

Decorators
- General tool for adding logic that manages both functions and classes, or later calls to them
- e.g. log, count calls, check its argument types

Function decorator¶

Augment function definitions
Wrap class method in an extra layer of logic implemented as another function, usually called metafunction
Similar to delegation design pattern but designed to augment a specific funtcion
A sort of runtime declaration

What the function decorators do¶

In [26]:

# Decorator version


class C:
    @staticmethod
    def meth():
        pass

In [27]:

class C:
    def meth():
        pass

    meth = staticmethod(meth)

User-Defined Function Decotator¶

Use __call__

In [28]:

class tracer:
    def __init__(self, func):
        self.calls = 0
        self.func = func

    def __call__(self, *args):
        self.calls += 1
        print("Call {} to {}".format(self.calls, self.func.__name__))
        return self.func(*args)


@tracer
def spam(a, b, c):  # Same as spam=tracer(spam)
    return a + b + c


print(spam(1, 2, 3))
print(spam(3, 4, 5))

Call 1 to spam
6
Call 2 to spam
12

Explanation¶

The spam function is run through the tracer decorator, when the original spam is called it actually triggers the __call__ in the class
*name argument is used to pack and unpack the passed-in arguments, because of this, this decorator can be used to wrap any function with any number of positional arguments

Class dectorator and metaclass¶

Augment class definitions
Add supoort for management of whole objects and their interface (metaclass)
Manage an object's entire interface by intercepting construction calls
metaclass generally redeines the __new__ or __init__ of the type class that normally intercepts this call
- meta class need not be a class at all

What the class decorators do¶

In [29]:

def decorator(a_class):
    pass


@decorator
class C(object):
    pass

In [30]:

def decoraor(a_class):
    pass

class C(object):
    pass

C = decoraor(C)

Super¶

Skip

Class Gotachas¶

Class Are Mutable Objects (Side Effect)¶

Chaning class attributes can have side effects