Chapter 6: Text & Bytes (continued)¶

Special Characters¶

In [1]:

"This is a sentence\nthat is printed\non three lines."

Out[1]:

'This is a sentence\nthat is printed\non three lines.'

In [2]:

print("This is a sentence\nthat is printed\non three lines.")

This is a sentence
that is printed
on three lines.

In [3]:

print("ABC\bX")

ABX

In [5]:

print("ABC\rX")

XBC

In [8]:

print("Jump\tfrom\ttab\tstop\tto\ttab\tstop.\nThe\tsecond\tline\tdoes\tso\ttoo.")

Jump	from	tab	stop	to	tab	stop.
The	second	line	does	so	too.

Raw Strings¶

In [9]:

print("C:\Programs\new_application")

C:\Programs
ew_application

<>:1: SyntaxWarning: invalid escape sequence '\P'
<>:1: SyntaxWarning: invalid escape sequence '\P'
/tmp/ipykernel_159416/1102122489.py:1: SyntaxWarning: invalid escape sequence '\P'
  print("C:\Programs\new_application")

In [10]:

print("C:\Users\Administrator\Desktop\Project")

  Cell In[10], line 1
    print("C:\Users\Administrator\Desktop\Project")
          ^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

In [11]:

print("C:\\Programs\\new_application")

C:\Programs\new_application

In [12]:

print("C:\\Users\\Administrator\\Desktop\\Project")

C:\Users\Administrator\Desktop\Project

In [13]:

print(r"C:\Programs\new_application")

C:\Programs\new_application

In [14]:

print(r"C:\Users\Administrator\Desktop\Project")

C:\Users\Administrator\Desktop\Project

Characters are Numbers with a Convention¶

ASCII¶

In [15]:

ord("A")

Out[15]:

In [16]:

chr(65)

Out[16]:

'A'

In [17]:

ord("\n")

Out[17]:

In [18]:

chr(10)

Out[18]:

'\n'

In [19]:

for number in range(48, 58):
    print(number, bin(number), "-> ", chr(number))

48 0b110000 ->  0
49 0b110001 ->  1
50 0b110010 ->  2
51 0b110011 ->  3
52 0b110100 ->  4
53 0b110101 ->  5
54 0b110110 ->  6
55 0b110111 ->  7
56 0b111000 ->  8
57 0b111001 ->  9

In [20]:

for i, number in enumerate(range(65, 91), start=1):
    end = "\n" if i % 3 == 0 else "\t"
    print(number, bin(number), "-> ", chr(number), end=end)

65 0b1000001 ->  A	66 0b1000010 ->  B	67 0b1000011 ->  C
68 0b1000100 ->  D	69 0b1000101 ->  E	70 0b1000110 ->  F
71 0b1000111 ->  G	72 0b1001000 ->  H	73 0b1001001 ->  I
74 0b1001010 ->  J	75 0b1001011 ->  K	76 0b1001100 ->  L
77 0b1001101 ->  M	78 0b1001110 ->  N	79 0b1001111 ->  O
80 0b1010000 ->  P	81 0b1010001 ->  Q	82 0b1010010 ->  R
83 0b1010011 ->  S	84 0b1010100 ->  T	85 0b1010101 ->  U
86 0b1010110 ->  V	87 0b1010111 ->  W	88 0b1011000 ->  X
89 0b1011001 ->  Y	90 0b1011010 ->  Z

In [21]:

for i, number in enumerate(range(97, 123), start=1):
    end = "\n" if i % 3 == 0 else "\t"
    print(str(number).rjust(3), bin(number), "-> ", chr(number), end=end)

 97 0b1100001 ->  a	 98 0b1100010 ->  b	 99 0b1100011 ->  c
100 0b1100100 ->  d	101 0b1100101 ->  e	102 0b1100110 ->  f
103 0b1100111 ->  g	104 0b1101000 ->  h	105 0b1101001 ->  i
106 0b1101010 ->  j	107 0b1101011 ->  k	108 0b1101100 ->  l
109 0b1101101 ->  m	110 0b1101110 ->  n	111 0b1101111 ->  o
112 0b1110000 ->  p	113 0b1110001 ->  q	114 0b1110010 ->  r
115 0b1110011 ->  s	116 0b1110100 ->  t	117 0b1110101 ->  u
118 0b1110110 ->  v	119 0b1110111 ->  w	120 0b1111000 ->  x
121 0b1111001 ->  y	122 0b1111010 ->  z

In [22]:

symbols = (
    list(range(32, 48))
    + list(range(58, 65))
    + list(range(91, 97))
    + list(range(123, 127))
)

In [23]:

for i, number in enumerate(symbols, start=1):
    end = "\n" if i % 3 == 0 else "\t"
    print(str(number).rjust(3), bin(number).rjust(10), "-> ", chr(number), end=end)

 32   0b100000 ->   	 33   0b100001 ->  !	 34   0b100010 ->  "
 35   0b100011 ->  #	 36   0b100100 ->  $	 37   0b100101 ->  %
 38   0b100110 ->  &	 39   0b100111 ->  '	 40   0b101000 ->  (
 41   0b101001 ->  )	 42   0b101010 ->  *	 43   0b101011 ->  +
 44   0b101100 ->  ,	 45   0b101101 ->  -	 46   0b101110 ->  .
 47   0b101111 ->  /	 58   0b111010 ->  :	 59   0b111011 ->  ;
 60   0b111100 ->  <	 61   0b111101 ->  =	 62   0b111110 ->  >
 63   0b111111 ->  ?	 64  0b1000000 ->  @	 91  0b1011011 ->  [
 92  0b1011100 ->  \	 93  0b1011101 ->  ]	 94  0b1011110 ->  ^
 95  0b1011111 ->  _	 96  0b1100000 ->  `	123  0b1111011 ->  {
124  0b1111100 ->  |	125  0b1111101 ->  }	126  0b1111110 ->  ~

Unicode¶

In [24]:

"\U0001f604"

Out[24]:

'😄'

In [25]:

"\N{FACE WITH TEARS OF JOY}"

Out[25]:

'😂'

In [26]:

"\U00000041"  # hex(65) == 0x41

Out[26]:

'A'

In [27]:

"\u0041"

Out[27]:

'A'

In [28]:

"\x41"

Out[28]:

'A'

In [29]:

len("A")

Out[29]:

In [30]:

"\N{SNAKE}"

Out[30]:

'🐍'

In [31]:

len("\N{SNAKE}")

Out[31]:

Multi-line Strings¶

In [35]:

multi_line = """
I am a multi-line string
consisting of four lines.
"""

In [37]:

print(multi_line)

I am a multi-line string
consisting of four lines.

In [38]:

for i, line in enumerate(multi_line.split("\n"), start=1):
    print(i, line)

1 
2 I am a multi-line string
3 consisting of four lines.
4

The `bytes` Type¶

In [41]:

with open("full_house.bin", mode="rb") as binary_file:
    data = binary_file.read()

In [42]:

id(data)

Out[42]:

139880714782512

In [43]:

type(data)

Out[43]:

bytes

In [44]:

data

Out[44]:

b'\xf0\x9f\x82\xa7\xf0\x9f\x82\xb7\xf0\x9f\x83\x97\xf0\x9f\x83\x8e\xf0\x9f\x83\x9e'

In [45]:

len(data)

Out[45]:

In [46]:

for byte in data:
    print(byte, end=" ")

240 159 130 167 240 159 130 183 240 159 131 151 240 159 131 142 240 159 131 158

In [47]:

data[-1]

Out[47]:

In [48]:

data[::2]

Out[48]:

b'\xf0\x82\xf0\x82\xf0\x83\xf0\x83\xf0\x83'

Character Encodings¶

In [49]:

cards = data.decode()

In [50]:

type(cards)

Out[50]:

str

In [51]:

cards

Out[51]:

'🂧🂷🃗🃎🃞'

In [52]:

place = "Café Kastanientörtchen"

In [53]:

place.encode()

Out[53]:

b'Caf\xc3\xa9 Kastanient\xc3\xb6rtchen'

In [54]:

place.encode("iso-8859-1")

Out[54]:

b'Caf\xe9 Kastanient\xf6rtchen'

In [55]:

place.encode("iso-8859-1").decode()

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
Cell In[55], line 1
----> 1 place.encode("iso-8859-1").decode()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 3: invalid continuation byte

In [56]:

"Dobrý den, přátelé!".encode("iso-8859-1")

---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
Cell In[56], line 1
----> 1 "Dobrý den, přátelé!".encode("iso-8859-1")

UnicodeEncodeError: 'latin-1' codec can't encode character '\u0159' in position 12: ordinal not in range(256)

Reading Files (continued)¶

In [57]:

with open("umlauts.txt") as file:
    print("".join(file.readlines()))

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
Cell In[57], line 2
      1 with open("umlauts.txt") as file:
----> 2     print("".join(file.readlines()))

File <frozen codecs>:322, in decode(self, input, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 9: invalid continuation byte

In [58]:

with open("umlauts.txt", encoding="iso-8859-1") as file:
    print("".join(file.readlines()))

Lerchen-Lärchen-Ähnlichkeiten
fehlen. Dieses abzustreiten
mag im Klang der Worte liegen.
Merke, eine Lerch' kann fliegen,
Lärchen nicht, was kaum verwundert,
denn nicht eine unter hundert
ist geflügelt. Auch im Singen
sind die Bäume zu bezwingen.
Die Bätrachtung sollte reichen,
Rächtschreibfählern auszuweichen.
Leicht gälingt's, zu unterscheiden,
wär ist wär nun von dän beiden.

Best Practice: Use UTF-8 explicitly¶

In [59]:

with open("lorem_ipsum.txt", encoding="utf-8") as file:
    content = "".join(file.readlines())

In [60]:

content

Out[60]:

"Lorem Ipsum is simply dummy text of the printing and typesetting industry.\nLorem Ipsum has been the industry's standard dummy text ever since the 1500s\nwhen an unknown printer took a galley of type and scrambled it to make a type\nspecimen book. It has survived not only five centuries but also the leap into\nelectronic typesetting, remaining essentially unchanged. It was popularised in\nthe 1960s with the release of Letraset sheets.\n"

In [61]:

print(content)

Lorem Ipsum is simply dummy text of the printing and typesetting industry.
Lorem Ipsum has been the industry's standard dummy text ever since the 1500s
when an unknown printer took a galley of type and scrambled it to make a type
specimen book. It has survived not only five centuries but also the leap into
electronic typesetting, remaining essentially unchanged. It was popularised in
the 1960s with the release of Letraset sheets.