Python Fundamentals
Glossary#
- Concurrency
-
Concurrency is concerned with managing access to shared state from different threads. Beware concurrency and parallelism are distinct concepts. Parallelism is concerned with utilizing multiple processors/cores to improve the performance of a computation.
- Container Object
-
Some objects contain references to other objects, these objects are called containers. Some examples of container-objects are a tuple, list, and dictionary. The value of an immutable container that contains a reference to a mutable object can be changed if that mutable object is changed. However, the container is still considered immutable because when we talk about the mutability of a container only the identities of the contained objects are implied.
-
But
-
See also Mutability
- Decorator
-
A function that takes a function as input and return another function!
decorators wrap a function, modifying its behavior. i.e the return function is assigned to the initial function variable !
- Deep Copy
-
A deep copy duplicates not just the outer object (like a list), but also all the nested mutable objects inside it. In contrast, a shallow copy only duplicates the outer container — the inner elements are still references to the same objects.
import copy original = [[1, 2], [3, 4]] shallow = copy.copy(original) # Could also be original[:] if original is a sequence # or original.copy() if object supports it (lists, dicts, sets only) # copy.copy is for any object that supports it! deep = copy.deepcopy(original) # Change an inner value original[0][0] = 99 print("Original:", original) # [[99, 2], [3, 4]] print("Shallow:", shallow) # [[99, 2], [3, 4]] — changed! print("Deep:", deep) # [[1, 2], [3, 4]] — unchanged ✅ - Global Interpreter Lock (GIL)
-
The Python threading module uses threads instead of processes. Threads run in the same unique memory heap. Whereas Processes run in separate memory heaps. This, makes sharing information harder with processes and object instances. One problem arises because threads use the same memory heap, multiple threads can write to the same location in the memory heap which is why the default Python interpreter has a thread-safe mechanism, the “GIL” (Global Interpreter Lock). This prevent conflicts between threads, by executing only one statement at a time (serial processing, or single-threading).
-
The Global Interpretor Lock (GIL) in CPython prevents parallel threads of execution on multiple cores, thus the threading implementation on python is useful mostly for concurrent thread implementation in web-servers.
-
More at https://blog.usejournal.com/multithreading-vs-multiprocessing-in-python-c7dc88b50b5b
- Identity
-
~ pointer to memory area (memory address)??? An object as a type, value, and identity -- type and identity of the object stays constant! In Python, a reference is not directly exposed, but you can work with it using the object's memory address (id()) or weak references.
>>> id([1,2]) 4382416128 >>> id([1,2]) 4382416128 >>> a = [1,2] <== variable >>> id([1,2]) 4382422528 >>> id(a) <== variable points to the identity of the assigned object, the identity is used by variables and container objects 4382416128>>> s = 'hello' >>> id(s) 4384848624 >>> sorted(s) ['e', 'h', 'l', 'l', 'o'] >>> id(s) 4384848624 >>> id(sorted(s)) 4385528064 -
See also Type
- Immutable
-
Keys in dictionaries have to be immutable (so can be hashed!) meaning only strings, numbers, frozensets, tuples, bool, range can be dictionary keys
- Method Resolution Order (MRO)
-
A list of classes. The first is the object at hand, the next ones are the parent classES, followed by the grandparent classES...
- Mutability
-
The value of some objects can change (an object as a type, value, and identity -- type and identity of the object stays constant!). Objects whose value can change are said to be mutable (list, dict, set can be changed after their creation); objects whose value is unchangeable once they are created are called immutable (e.g. numbers, tuples, strings, frozensets, bools, ranges)
-
See also Container Object
- Parallelism
-
Parallelism is concerned with utilizing multiple processors/cores to improve the performance of a computation. Beware parallelism and concurrency are distinct concepts. Concurrency is concerned with managing access to shared state from different threads.
- Reference
-
In Python, a reference is a name that points to an object in memory. When you assign a variable to a value, you're actually assigning a reference to that object, not copying the data itself.
The variables hold references, not values
a = [1, 2, 3] # 'a' holds a reference to the list [1, 2, 3] b = a # 'b' is another reference to the same list # Both a and b reference the same list in memory. # Modifying one affects the other. b.append(4) # Modify the list using the 'b' variable print(a) # [1, 2, 3, 4] print(b) # [1, 2, 3, 4] (same list)a = [1, 2, 3] b = a.copy() # Creates a new list (a shallow copy) # similar to a[:] ? b.append(4) # Modify 'b' only print(a) # [1, 2, 3] (original remains unchanged) print(b) # [1, 2, 3, 4] (new copy) - Shallow Copy
-
A shallow copy only duplicates the outer container — the inner elements are still references to the same objects. In contrast, a deep copy duplicates not just the outer object (like a list), but also all the nested mutable objects inside it.
- Thread
-
Multiple threads live in the same process in the same space, each thread will do a specific task, have its own code, own stack memory, instruction pointer, and share heap/virtual memory. If a thread has a memory leak it can damage the other threads and parent process.
-
See also Global Interpreter Lock
- Type
-
Example
-
See also Identity
- Value
Code Primer#
#!/usr/bin/env python3
import sys
def main(argv):
assert len(argv) >= 3, 'Too few parameters'
assert len(argv) <= 1, 'No parameter to command line provided'
print(argv)
prog_name = argv[0]
argument_1 = argv[1]
if __name__ == "__main__": # when you run the python file, __name__ is set to "__main__"
# Otherwise it is set to the module name
main(sys.argv)
#!/usr/bin/env python3
# Given a list of words, group the anagram in a sublist together
# ['abc', 'bar', 'cab'] ---> [ ['cab', 'abc'], ['bar']]
def ana(L): # (1)
anagrams = {}
for word in L:
K = ''.join(sorted(word)) # (2)
if K not in anagrams.keys(): # (3)
anagrams[K] = []
anagrams[K].append(word) # = anagram_list
dict_values = anagrams.values() # <!> type is dict_values([['abc', 'cab'], ['bar']])
return( list(dict_values) ) # return the values of the anagrams dict in a list format
# ana(['abc', 'bar', 'cab'])[0]
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# TypeError: 'dict_values' object is not subscriptable
# list(ana(['abc', 'bar', 'cab']))[0]
# ['abc', 'cab']
- L is list of words
- Reorder the chars of the word sorted(word) ==> list of ordered characters as if 'word' was a list of char sorted('akc') ==> ['a', 'c', 'k']
- K is NOT already a key in the dictionary ... create key/entry + initialize
x_values_list = [1, 2, 3]
y_values_list = [4, 5, 6]
m = map(lambda x, y: x + y, x_values_list, y_values_list) # m is <map at 0x104a65180>
# similar to: for x, y in zip(x_values_list, y_values_list) ? Almost!
# zip returns an iterable on tuples, not a map object!
type(m) # m is a map
list(m) # a list ;-) or [5, 7, 9]
ret_list = list(map(lambda x, y: x + y, x_values_list, y_values_list)) # Add list for Python3
ret_list = list(map(lambda x: x**2, x_values_list))
>>> _list = [ ('a',34), ('z', 23), ('b', 44)]
>>> sorted(_list) # sort list items based on the first value in the
[('a', 34), ('b', 44), ('z', 23)]
# (DEPRECATED) Sort based on 2nd field of tuple ~~~> reverse tuple
>>> _list = sorted(list(map(lambda el: el[::-1], _list)))
# >>> el_values_list = _list # Those 2 lines ...
# >>> _list = sorted(list(map(lambda el: el[::-1], el_values_list))) # ... do the same as above!
>>> sorted(_list) # Sort the elements of a list... using the first value in each element!
[(23, 'z'), (34, 'a'), (44, 'b')]
data = [("apple", 3), ("banana", 1), ("cherry", 2)]
# Sort by second element (index 1)
sorted_data = sorted(data, key=lambda x: x[1])
print(sorted_data)
# Output: [('banana', 1), ('cherry', 2), ('apple', 3)]
>>> [ c*2 for c in "012345678" ]
['00', '11', '22', '33', '44', '55', '66', '77', '88']
# Dictionary comprehension
# build a new dictionary using a function
my_dictionary_comprehension = {k: f(v) for k, v in d.items() if k == 1}
# Fct
def myfunc(self, *args, **kwargs) : # args is a tuple, kwargs is a dict
import my_module as mm
help(mm) # print the docstring of the module + of functions
help(mm.greet) # print the docstring of the given function in the module
help(WHATEVER) # <== prints docstrings
help() # <== help in interactive mode
help(dir)
dir() # List variables/names in the current scope
dir(int) # dir (<class>) --> list methods of <class>
help(int.to_bytes) # help ( <class>.<method> )
dir(WHATEVER) <== returns a list of variables in scope of methods
assert 3/4 > 1, "not so fast!" # raise AssertionError with explanation
help(assert) # Fails, assert is a statement, not a function, i.e. assert()!
help(assert.__doc__) # Fails
help('assert') # Works! help for the standalone command 'assert'
help(str) # return info for method and class for 'str' objects
help(str.__doc__) # ok, works, but not formatted correctly
print(str.__doc__) # print the help for the 'str' type conversion function
<!> If module was already loaded, you need to reload it!
import importlib
importlib.reload(my_module) # or .reload(mm)
# To run if testmod block is not present: python -m doctest your_file.py
# Python will automatically scan the file for docstring tests and run them.
def add(a, b):
"""
Returns the sum of a and b.
>>> add(2, 3)
5
>>> add(-1, 1)
0
"""
return a + b
# To run the tests (with testmod() block ): python calculator.py
if __name__ == "__main__":
import doctest
doctest.testmod()
import pdb
def add(a, b):
pdb.set_trace() # Execution will pause here (when run with pdb)
return a + b
result = add(3, 5)
print(result)
$ python -m pdb my_script.py # run the pdb
pdb
# operations
n | next
s | step - step into a function call
c | continue - continue exection until next breakbpoint
r | return - continue execution until function returns
q | quit - quit debugger
b 10 - set a breakpoint at line 10
l | list - list all breakpoints
p variable - print the value of a variable
pp variable - pretty print of a variable
Miscellaneous#
Hello world!#
$ python3 --version
Python 3.9.10
$ python3
>>> print("Hello world!") # Python 3
>>> 2+1
>>> 2*(3+4)
#!/usr/bin/env python
#!/usr/bin/env python2 # DEPRECATED
#!/usr/bin/env python3
Data structure & algorithm#
Set ::
List ::
Map ::
Trees ::
Graph ::
Big0 notiation ::
Check this site#
Variables scopes#
Global variables#
globvar = 0
def set_globvar_to_one():
global globvar # Needed to modify global copy of globvar
globvar = 1
def print_globvar():
print(globvar) # No need for global declaration to read value of globvar
set_globvar_to_one()
print_globvar() # Prints 1
name#
Numbers#
Numbers are immutable!
so are strings, tuples, etc.
But beware: it is the content that is immutable, the variable that points to this content can still change!
0
1
1.2
34.56
-56.3
>>> a = 1
>>> id(1)
4329177328
>>> id(a)
4329177328
>>> a = 256 # Variable is pointing to the identity of another immutable number!
>>> id(256) # If you take a higher number, the id can change !!!!
4329185488
>>> id(a)
4329185488
More About (im)mutability @ https://towardsdatascience.com/https-towardsdatascience-com-python-basics-mutable-vs-immutable-objects-829a0cb1530a
integer and float#
5 / 2 == 2.5 # Python 3 only !
5 // 2 == 2 # Python 3 only (equivalent to floor) !
math.floor(5/2) # rounds down towards -oo --> outputs 2
math.floor(-5/2) # Outputs -3
math.ceil(5/2) # rounds up towards +oo --> Outputs 3
math.ceil(-5/2) # rounds up towards +oo --> Outputs -2
round(5/2) # rounds towards 0? NO, round half (.5) to closest even --> Outputs 2 (rounds down)
round(-5/2) # Outputs -2 <-- round is symetric and round up and down (to avoid bias)
round(7/2) # Outputs 4 (rounds up!)
round(-7/2) # Outputs -4
int(3.7) # outputs 3 <-- always round towards 0!
int(-3.7) # outputs -3
math.trunc(3.7) # equivalent to int(3.7)
# Function Returning Multiple Values (Tuple)
def divide(a: int, b: int) -> tuple[int, int]: # Takes two integers, returns a tuple of int
return a // b, a % b # // and % work together!
import math
>>> _pi = math.pi
>>> print(_pi)
3.141592653589793
>>> g1 = round(_pi,2) # Rounding floats
>>> print(g1)
3.14
>>> g2 = float("{1:.2f}".format(0.1234, _pi)) # Same operation with conversion through a formatted string!
# float --> turn string into float
# format string with element 1 in format (which is _pi!)
# and format it with .2f as a float with 2 digits after the comma
>>> g
3.14
Source @ http://www.tutorialspoint.com/python/python_basic_operators.htm
bitwise operation#
<!> Spaces added in binary value for clarity but must be removed in real python code<
>
<!> Prefix 'b' as in b"0001" ==> bytes, prefix '0b' as in 0b0001 ==> integer_coded_in_binary
a = 0b 0011 1100 # a = 60 !
b = 0b 0000 1101 # b = 13 !
-----------------
a&b = 0b 0000 1100 @ a AND B
a|b = 0b 0011 1101 # a OR B
a^b = 0b 0011 0001 # a XOR b
# Note that 2^2 is not 2**2 !!
~a = 0b 1100 0011 # NOT a
Source @ http://www.tutorialspoint.com/python/python_basic_operators.htm
byte#
two_bytes = b'\x01\x02'
number = int.from_bytes(two_bytes, byteorder='big') # Interprets as 258 (1*256 + 2)
print(number)
number = 258
bytes_obj = number.to_bytes(2, 'big') # Outputs: b'\x01\x02'
print(bytes_obj)
byte1 = b'\x05'
byte2 = b'\x03'
# Extract the integer values (5 and 3)
value1 = byte1[0]
value2 = byte2[0]
# Perform arithmetic addition
sum_value = value1 + value2 # 5 + 3 = 8
print(sum_value) # Outputs: 8
# Optionally, convert the result back to a bytes object (if it fits in one byte)
result_byte = sum_value.to_bytes(1, 'big')
print(result_byte) # Outputs: b'\x08'
introspection#
>>> i = 17
>>> type(i)
<class 'int'>
>>> dir(45)
['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'as_integer_ratio', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']
>>> help(int.to_bytes)
... shows method descriptor ...
>>> i.to_bytes(2,"big") # 2-byte representation
# 'big', the most significant byte is at the beginning of the byte array.
b'\x00\x11'
>>> i.to_bytes(4,"big")
b'\x00\x00\x00\x11'
>>> i.to_bytes(4,"little")
b'\x11\x00\x00\x00'
Strings#
Immutability#
A string ~= immutable tuple of characters
In Python, immutability refers to the property of an object that cannot be changed after it is created.
Well, not exactly because their types are different, one is 'str' while the other is 'tuple'. They are also different objects, with different methods!
- Strings are immutable in python (see replace) as are number (int, float, decimal), bool, tuple, and range!
- What is immutable is the content of the string, the variable can be reassigned!
What is unicode and utf-8 @ https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/
Demonstration of immutability
>>> text = "Data Science"
>>> print(id(text))
2450343168944 # also = to id("Data Science") ;-)
>>> text += " with Python"
>>> print(id(text))
2450343426208 # The variable points to a new identity (~pointer)
our_str ='Bonjour Monde'
our_str = 'Hello World' # Override variable <> immutability of pointed content
print(our_str[0]) # At a given index, you can read, but you cannot write!
# our_str[0] = 'h' # Exception: TypeError: 'str' object does not support item assignment
# our_str[0] = our_str[0].lower() # Exception! Strings are immutable
new_str = our_str.replace('World', 'Jackson') # HERE we create a new string, the old string is not changed!
Hands-on#
# <!> sorted(string) return a sorted list of characters contained in string
# <!> This list needs to be turned into a string using join()!
ordered_chars_LIST = sorted(string_of_unordered_chars) # returns an ordered LIST or characters , not a string but a LIST!
ordered_chars = ''.join(sorted(unordered_chars)) # Order a string (anagram)
string1 = "this is a string"
string2 = "%s! %s" % ("hello","toto") # Print a string in a string!
string3 = "%(h)s! %(n)s" % { "n":"toto", "h":"hello"} # Using named variable from a dictionary (great for templating!)
string4 = "a number y is {1:.2f} and x is {0:.3f}".format(x, y) # order from tuple followed by format!
price = 4.55
string5 = f"Price in Swiss Franks: {price * 1.086:5.2f}" # String literal <!> 'price' variable must exists
# Don't forget the 'f' at the beginning of the string!
str(3210) # Convert number to string '3210'
str("hello") # Convert a string into a string, i.e. do nothing!
float("3.1415") # Turn a string to float
int("3") # Turn a string to a integer
# int("3.1415") # <!> This is an error as this is a float
int(float("3.14")) # <!> Ok, this works! ;-) and is 3
string3 = string1 + string2 # Concatenation operation
multiline = "This is a \n\
... multiline string" # Multiline string
#comment
len(var3) # Length of a string
print(program, "arguments") # Python3
print(program, "arguments", sep=" ", end="\n") # Python3 equivalent = Concat 2 strings with a space in between + CR at the end
>>> list("hello world") # A string = tuple of chars --> turn a tuple of char into a list!
['h', 'e', ...., 'o', ' ', 'w', ..., 'd']
# <!> Strings are immutable, but here you are just changing where the variable points to, not the string
s = s.lstrip() # Remove leading spaces
s = s.rstrip() # Remove trailing spaces and CARRIAGE RETURNS
s = s.strip() # Remove leading and trailing spaces and CARRIAGE RETURNS
title = title.strip(',.-\n') # Remove leading and trailing characters
# <!> Will strip any of those characters, not the string ",.-" but individual characters!
number_of_ts = s.count('t') # Return number of character 't' in the string
number_of_totos = s.count('toto') # Return number of substrings
pos_of_first_t = s.find('t')
pos_of_second_t = s.find('t', pos_of_first_t) # Find returns the index of first found 't' (or second t since starting from 1st 't' position)
# If find reaches the end, then returns -1
s.capitalize()
s.join()
s.split() # Split on any-block of spaces/tabs
# , e.g. "12 3 4" --> ['12', '3', '4']
# s.split('') # <!> but this ~~~> ValueError: empty separator!
# s.split("") # <!> but this ~~~> ValueError: empty separator!
s.split("\t") # split on tab characters (\t is the tab character, not 2 chars!)
s.split(" ") # split on space charactes only, not tabs!
"hello world".split("wo") # split on a GROUP of characters <!> Not like strip, which assume individual characters
"arn::account::region".split("::") # returns a list!
# Replace = split and join?
arn = "arn::account::region"
arn = "XX".join(arn.split("::"))
=?= arn.replace("::","XX") # ?
>>> s.translate(None, ",!.;") # Remove unwanted characters, here the punctuation
>>> "Python".center(10)
' Python '
s.endswith('toto')
s.startswith('toto')
s = "one\ttwo\t\tthree\nfout" # Tabs and other special characters in a string
print(s) # Transform tabs and other special characters
string_content = "line 1\nline 2\nline 3\n"
string_file_content.splitlines() # ~ s.split('\n') ?
[ 'line 1', 'line 2', 'line 3']
>>> account_number = "43447879"
>>> account_number.zfill(12) # Zero fill
'000043447879'
>>> account_number.rjust(12,"0") # Right justification
'000043447879'
# <!> Immutability: can only be read
string[0] # first character
string[-1] # very last character
string[1:4] # a slice
string[10:] # a slice to end of string
string[:-1] # everything but last character
string[:] # copy of string
string[::1] # copy of string, step of 1 (default)
string[::2] # print every other characters!
string[::-1] # reverse string, step of -1
string[::-2] # reverse string + take every other characters
s = " a very long string ..... " # pep8 compliance for long string
s =(
"a very long"
"string ... "
)
s = """
A multi-line string
With space as well
and carriage returns
"""
s # '\n A multi-line string\n With space as well\nand carriage returns\n'
print(s) #
# A multi-line string
# With space as well
# and carriage returns
Operations#
Equality/Inequality used for sorting list of strings with sorted!
("a" < "b") is True # True
("a" < "aaa") is True # True
("1" < "a") is True # True
("aaa" < "a") is False # True, it is False ;-)
# Sorting based on ASCII code of characters!
out = sorted(["aaa", "b", "a", "2", "-1", "0"])
print(out) # Outputs: ["-1", "0", "2", "a", "aaa", "b"]
#
Print, format, and string literal#
% is an operation on string!
print(word)
print(word, end='\n') # Same as above (default)
print(word, end='') # Print without \n
print(pyObject) # Use the __str__ method or if missing the __repr__ method
print("%s" % pyObject) # Use the __str__ method or if missing the __repr__ method as well and ...
# ... the module operator of string class!)
# <!> __repr__ method is used for representation of the object in ipython when '>>> object'
print(word.__repr__()) # Print single quoted string as word.__repr__() is "'hello'"
print(str(pyObject)) # Convert using the __str__ method
# <-- this is the correct one!
print('a very very '
'very vreey '
'very long ine'
)
s = 'hello world!'
pi = 3.1415
print(f's is {s} and pi is {pi:.2}') # String literal
# <!> do not forget the 'f' before the quote otherwise not expended!
print(f's is {s} and pi is {pi:.2f}') # <!> also do not forget the f in the formatting of pi to indicate a float
# the first line prints 3.1 and the second 3.14 for the value of pi !!!
>>> print(q, p, p * q, sep= " ", end="\n") # (default) sep -> separator end -> CR
>>> print(q, p, p * q, sep=" :-) ", end="") # sep -> separator end -> no carriage return
459 :-) 0.098 :-) 44.982
FORMAT
The pythonic way to print strings, because strings are objects!
>>> print("average is {f}".format(average, f=1)) # here f is a MIX of variables + named arguments
average is 1 # <!> 1 is not the DEFAULT value for f, but the constant value of f !
# f is a keyword argument (a variable to be called in the formatted string)
>>> print("average is {:f}".format(average, f=1)) # here f is formatting of DEFAULT first element, i.e 0, as a float
# <!> if {} were to reappear in the string, it would point to the second element (which does not exist here)
# {} points to the first element only when appear first, then the counter is incremented!
average is 2.300000
>>> print("average is {0:f}".format(average, f=1)) # here f is formatting of element 0 (float) - same as above but more explicit!
average is 2.300000
>>> >>> print("average is {1:f}".format(average, 3.4, f=1.2)) # here 3.4 is element at pos 1
average is 3.400000 # Is f=1.2 at pos 2? NO! 'f' = keyword argument (**kwargs), while average and 3.4 are positional args (*args)
# >>> print("average is {1:f}".format(average, f=1)) # IndexError: Replacement index 1 out of range for positional args tuple
# Tuple *arg vs **kargs dict ? Yes!
>>> f = 1
>>> print("average is {1:f} {0:f}".format(average, f)) # the last f is the value of the variable 'f' and is a positional argument of the format method
>>> x = 3.1415
>>> float("{0:.2f}".format(x)) # don't forget the 'f' for float!
3.14
>>> print("{:d}".format(7000))
7000
>>> print("{:,d}".format(7000))
7,000
>>> print("{:^15,d}".format(7000)) # 15 is total number of characters, 7,0000 is centered
7,000
>>> print("{:*^15,d}".format(7000))
*****7,000*****
>>> print("{:*^15.2f}".format(7000))
****7000.00****
>>> print("{:*>15,d}".format(7000))
**********7,000
>>> print("{:*<15,d}".format(7000)) # left alignment
7,000**********
>>> print("{:*>15X}".format(7000)) # right alignment
***********1B58
>>> print("{:*<#15x}".format(7000)) # hexadecimal format
0x1b58*********
if not testbed:
raise Exception("No such testbed {}".format(testbed_name))
>>> '{0}{1}{0}'.format('abra', 'cad') # arguments' indices can be repeated
'abracadabra' # ~ format of a tuple, and index in the tuple
>>> 'Coordinates: {latitude}, {longitude}'.format(latitude='37.24N', longitude='-115.81W') # With named arguments
'Coordinates: 37.24N, -115.81W'
>>> "Art: {a:5d}, Price: {p:8.2f}".format(a=453, p=59.058)
'Art: 453, Price: 59.06'
>>> print("I've <{}> years of experience and my salary is <{:,}> USD per annum.".format(10, 75000)) # {} points to first and then {} points to second element
I've <10> years of experience and my salary is <75,000> USD per annum.
>>> data = dict(province="Ontario",capital="Toronto")
>>> data
{'province': 'Ontario', 'capital': 'Toronto'}
>>> print("The capital of {province} is {capital}".format(**data)) # With a dictionary
# SWEET **kwargs --> **dictionary !!!
"{0:<20s} {1:6.2f}".format('Spam & Eggs:', 6.99) # With anchored strings
'Spam & Eggs: 6.99'
>>> class Point(object):
... def __init__(self, x, y):
... self.x, self.y = x, y
... def __str__(self):
... return 'Point({self.x}, {self.y})'.format(self=self) # with objects
... def __repr__(self):
... return 'REPR Point({self.x}, {self.y})'.format(self=self)
...
>>> str(Point(4, 2))
'Point(4, 2)'
>>> Point(4,2)
REPR Point(4, 2)
Source @ https://www.python-course.eu/python3_formatted_output.php
More @ https://www.techbeamers.com/python-format-string-list-dict/
SHORT FORMAT (aka string literal -- see above) * <!> If variable does not exist ---> Exception: !NameError: name 'ss' is not defined
Print in a file#
from __future__ import print_function
s = 'hello world!'
# f = open("file.txt", "a")
with open("file.txt", "a") as f:
print(s, end="", file=f) # Python3
Source @ https://stackoverflow.com/questions/9236198/python-3-operator-to-print-to-file
Conversion#
- List are indexed at 0
- spliting a list gives you a list of string which you need to reformat
abc = "abcde....z" # A string is an immutable tuple of characters
char_list = list(abc)
s = 'afdadf 5 dfad 5.0 dfdsdf'
# token = s.split(' ') # Split on space character
token = s.split() # Split on any space char, including tab and multiple space
sum = sum + int(token[1]) + float(token[3]) # sum = sum + 5 + 5.0
Char in strings (char = string of len 1)#
Char is a string of length 1 !
abc = "abcde....z" # sring = an immutalbe tuple of characters!
char_list = list(abc)
>>> type('a')
<class 'str'>
>>> chr(97)
'a'
>>> ord('a') # works only on string of length 1
97
type(chr(97)) # <class 'str'>
>>> s = "asjo,fdjk;djaso,oio!kod.kps"
>>> s.translate(None, ",!.;") # Remove unwanted characters
'asjofdjkdjasooiokodkjodsdkps'
list('cat') # Turn a list in list of chars
['c', 'a',''t' ]
>>> for char in "python": # iterate on the char
... print(char)
>>> for char in list(string): # string --> list of chars (same as above)
print(char)
for pos, char in enumerate(string): # With index/position!
print(pos, char)
print(pos, char, sep=' ', end='\n') # same as above (with explicit default values!)
Words in string (not regex)#
Instead of regex, use method or operations of strings
new_str = our_str.replace('World', 'Jackson') # Don't change the same string
# <!> strings are immutable!
if "blah" in "otherstringblahtoto":
print('found group of chars')
s = 'tatatititata'
if s.find('toto') == -1:
print("Not found!")
s.find('ata') # return the position of the first occurence
s = "worl"
S = "Hello world!"
>>> S.find(s)
7
s.endswith('toto')
s.startswith('toto')
with open("myfile", r) as file:
# for line in file.readlines():
for line in file: # readlines from files (.readlines is implied)
# print(line)
# print(line, end='\n') # Same as line above, but <!> the line also includes a carriage return! ==> 2 carriage returns!
print(line, end='') # Here we remove the extra carriage return added by the print statement!
print(line.rstrip()) # Here we remove the \n from the string, but add the one from the print!
for word in string.split(): # Process words one at a time (split on spaces)
print(word, end='') # (Python3)
REGEX overkill!
# Using regex module!
import re
words = re.split('\W+', 'Words1, words2, words3.') # \W+ matches **one or more non-word characters**
print(words) # Outputs ['Words', 'words', 'words', '']
Str method of Classes/Objects#
>>> class Point(object):
... def __init__(self, x, y):
... self.x, self.y = x, y
... def __str__(self):
... """Used for string conversion, i.e. str(point) or print(point) or '%s' % point """
... return 'Point({self.x}, {self.y})'.format(self=self)
... def __repr__(self):
... """Used for repr(point) or in ipython when >>> P
... but is also used whenever the __str__() method would normally be used and when not defined/present!"""
... return('toto')
...
>>> str(Point(4, 2))
'Point(4, 2)'
More @ https://docs.python.org/2/library/string.html
variables, slicing#
2/ STRING, VARIABLES, SLICING
#!/usr/bin/env python
word = input("Enter a word: ") # python 3
print(word[1:] + word[0] + 'ay')
user input#
language = input('Enter language') # Python3
if language in ['C++', 'Python', 'Java']:
print(language, "rocks!")
if language not in [ 'French', 'English']:
print(language, "sucks!")
Introspection#
- <!> used a variable 's' or the type 'str', but not the value of the variable i.e. not 'dir('toto'.replace)
>>> s = 'toto'
>>> type(s)
str
>>> dir(s)
['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
>>> help(s.split)
split(sep=None, maxsplit=-1) method of builtins.str instance
...
Tuple#
Immutability#
Some objects contain references to other objects, these objects are called containers. Some examples of containers are a tuple, list, and dictionary. The value of an immutable container that contains a reference to a mutable object can be changed if that mutable object is changed. <
Immutability
- <!> What is immutable is the content of the tuple, the variable can be reassigned!
- <!> other immutable types are int, float, decimal, bool, string, tuple, and range.
More @ https://www.thegeekstuff.com/2019/03/python-tuple-examples/#more-17801
- <!> TUPLES ARE IMMUTABLE (Value cannot be changed) as are string and numbers
- <!> TUPLES ARE CONTAINER OBJECT, ex tuple of list
Example
t = ([1,2], [0])
id(t[0]) # identity of t[0] is 4368558656
t[0].append(3) # Here we are not changing the tuple t[0] reference! We are changing the value of the list t[0] is pointing to!
print(t) # ([1, 2, 3], [0]) <== Value of list in t[0] has changed!
id(t[0]) # identity of t[0] is 4368558656 <== id has NOT changed!
But
>>> t[0] = 1 # t[0] can only point to the particular list it was initialized to!
TypeError: 'tuple' object does not support item assignment
Hands-on#
tuple = ([0,1,2], 'two', 'three')
tuple = [0,1,2], 'two', 'three' # Same as above without the parentheses!
tuple # ([0, 1, 2], 'two', 'three')
id(tuple) # 4368483584 <-- identity the variable points to
tuple[::-1] # reverse a tuple! But immutability?!?!?
# The identities have not changed in the original tuple!
('three', 'two', [0, 1, 2]) # Here we are just printing/displaying a new tuple!
tuple = tuple[::-1] # Now the variable points to a new immutable tuple
id(tuple) # 4364976768 or different from the previous one!
>>> print(tuple[1])
>>> tuple[1] = 'deux' # BEWARE IMMUTABILITY !
# Value cannot change! (Assignment)
TypeError: 'tuple' object does not support item assignment
>>> tuple = ('three', 'two', [0, 'w', 2])
>>> tuple
('three', 'two', [0, 'w', 2])
>>> tuple[2][1] = 'w' # Here you change the mutable type (list) in the tuple
# , but the identity (~pointer) as seen by the tuple is still the same!
>>> tuple[1][1] = 'deux' # Fails because string are immutable, so the id(.) would have to change
>>> tuple[2][1] = 'y' # Works because the id in the list changes, but the id seen by the tuple, doesn't
>>> for el in tuple: # This is what cannot change, the ids(~pointer) in the tuple!
...: print(id(el))
...:
140640316455304
140640317363736
140640317362392
nested_tuple = (1,2), (3, 4) # A tuple of tuple
nested_tuple[1][1] # 4
nested_tuple # ((1, 2), (3, 4))
!!! Tuples are seen in function
def myfunc(self, *args, **kwargs) : # args is a tuple, kwargs is a dict
# args = arguments
# kwargs = keyword_arguments
zip#
#!/usr/bin/env python
names = ["Alice", "Bob", "Charlie", "Emmanuel"]
ages = [25, 30, 35]
# Using zip
zipped = zip(names, ages) # When the iterables passed to zip() are of unequal length, it stops when the shortest iterable is exhausted.
# Converting to a list to see the output
print(list(zipped))
# [('Alice', 25), ('Bob', 30), ('Charlie', 35)]
#!/usr/bin/env python
names = ["Alice", "Bob", "Charlie", "Emmanuel"]
ages = [25, 30, 35]
for name, age in zip(names, ages):
print(f"{name} is {age} years old.")
# Alice is 25 years old.
# Bob is 30 years old.
# Charlie is 35 years old.
# Zip vs map
# Zip is more readable
# map() stops at the shortest iterable, just like ip()
# map() version is slightly slower and more awkward!
a = [1, 2, 3]
b = ['a', 'b', 'c']
z1 = list(zip(a, b))
z2 = list(map(lambda *args: args, a, b))
print(z1) # [(1, 'a'), (2, 'b'), (3, 'c')]
print(z2) # [(1, 'a'), (2, 'b'), (3, 'c')]
Introspection#
>>> t = (1,2)
>>> type(t)
tuple
>>> dir(t)
['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'count', 'index']
>>> help(t.index)
nestedtuple.index((3,4)) # Where the element can be found <!> only first level element!
1
List#
Indexed to 0
- Lists are indexed to 0
- First element in the list is at index 0 !!!
Hands-on#
How do you copy a list?
You do NOT copy a list with 'list_0 = list_1', but with 'list0 = list1[:]' for shallow copies and
list_0 = [0,1]
list_0 = list(range(10)) # Python3: range is an iterator-like, but not an iterator!
id(list_0) # ex: 4368873792
list_1 = list_0 # Here you have 2 variables thet refers to the same list object, not a copy! <-- variables points to the same identity
id(list_1) # ex: 4368873792
list_1[2] = 'X' # Changing one will change the other!
list_0 # [0, 1, 'X', 3, 4, 5, 6, 7, 8, 9]
list_1 = list_0[:] # This is how to copy a list! <-- 2 different identities!
list_1[3] = 'Y' # Will NOT change what list_0 is pointing to! If you change one, you do NOT change the other!
list_1 # [0, 1, 'X', 'Y', 4, 5, 6, 7, 8, 9]
list_0 # [0, 1, 'X', 3, 4, 5, 6, 7, 8, 9]
a = range(10) # range(0,10) or 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
a
b = range(1,11) # range(1, 11) or 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
b
c = range(30,20,-1) # range(30, 20, -1) or 30, 29, 28, ..., 22, 21
c
>>> square=[i**2 for i in a] # List comprehension
>>> square=[i**2 for i in range(10)]
["Even" if x % 2 == 0 else "Odd" for x in range(5)] # ['Even', 'Odd', 'Even', 'Odd', 'Even']
✅ More Readable: Less boilerplate than loops.
✅ Faster: Optimized for performance compared to for loops.
✅ Memory Efficient: Combined with generators, it can reduce memory usage. (generators create values as they are read instead of first and then reading them from memory (as with regular lists)
. <!> <
>>> l1 = [1, 2]
>>> l2 = [3, 4]
>>> l2 == l2[:] == l2[::] # Same
True
>>> l1.extend(l2) # Concat the 2 lists and modify one of them in-place ! Here, l1 will be modified!
# Same as list_1 += list_2 !! <-- also list_1 keep the same id !!!
>>> l1
[1, 2, 3, 4]
>>> l1.append(l2) # Append 1 element at the end of the list (l1 is modified)
>>> l1
[1, 2, 3, 4, [3, 4]]
>>> l1.insert(0,"Yes") # In-place insert at the beginning of the list ~ pre-pend!
# The opposite of append!
>>> l1[0]
Yes
>>> l1 # Note that insert does NOT replace! (unlike l[0] = 'No' would!)
['Yes', 1, 2, 3, 4, [3, 4]]]
>>> a = 5
>>> l3 = [1, 2, 3]
>>> [a] + l3 # Insert at the beginning with '+'
# Almost same as .extend, except the original list is not changed in place!
[5, 1, 2, 3]
>>> list = [1, 'deux', {'trois': 3 } ] # A list doesn't require its elements to be different
>>> list = ['the','holy','grail'] # List ~ array are represented with square brackets
>>> nested_list = [ 'XXX', list] # One of the elements of the list is a list
>>> 'XXX' in nested_list # Check list membership
True
# The returned lists are not the original list (ie different id)
# You cannot do l[1:].remove(3) and update the list l
>>> my_list[:] # The whole list
>>> my_list[0:] # The whole list (same as above)
>>> my-list[0:len(my_list)] # The whole list (same as above)
>>> my_list[::1] # The whole list (same as above)
>>> my_list[2:] # From 3rd element to end
>>> my_list[2:-2] # From 3rd element to 2nd before last
>>> my_list[-1:] = [9] # from the last element to the end
>>> my_list[-2:] = [8, 9]
>>> my_list[:-2:] = [0, 1, 2, 3, 4, 5, 6, 7]
>>> nested_list[1] = 'awesome' # List are mutable!
>>> my_list.append('for sure') # append 1 element only
>>> my_list.append(another_list) # 'another_list' is placed at the end of the list as a single element
>>> my_list.insert(1,'super') # inserting of an element without deletion
>>> my_list.remove('super') # remove the FIRST 'super' entry from the list
>>> my_list = [x for x in my_list if x != 'super'] # remove all the 'super' entry
>>> my_list.extend(another_list) # contact the 2 lists like the + operation, but my_list is changed in place!
>>> my_list.extend(another_list) # contact the 2 lists like the + operation and change in place my_list !
>>> my_list = my_list + another_list # same as above
>>> myList.index("revolves")
3
>>> "revolves" in myList
True
>>> my_list.index("a")
Exception ValueError: 'a' is not in list
token = string.split('\t') # Split on tab only
token = string.split() # Split on any space character
# Remove all the elements with a given value between 2 indexes of a list
def remove_between_indexes(lst, start, end, value):
return lst[:start] + [x for x in lst[start:end] if x != value] + lst[end:]
# Example list
my_list = ["keep", "remove_me", "keep", "remove_me", "keep", "remove_me", "keep"]
# Remove 'remove_me' between index 1 and 5
new_list = remove_between_indexes(my_list, 1, 5, "remove_me")
print(new_list)
function with list parameter#
len(lst) - length of the list
sorted(lst) - <!> lst.sort() <== sort in place, but sorted(lst) sort the output
sum(lst) - sum of all the element of the list
- <!> add to 0, so must be a number and not char/str/etc. !
lst[::-1] - <!> l.reverse() <== reverse in place, but lst[::-1] reverse the output only!
...
sorting#
. < >
. <!> sort numbers first and then string ! ( = use LSD Radix sort first and then MSD Radix sort?) . <!> Deprecated? Use lst.sort() instead of sorted(l) ? No, lst.sort() = sort in place while sorted(lst) does not change lst! . <!> Inplace sorting ! Not lie l2 = l1.sort() ... like sorted?sorted(lst)#
sorted(range(30,20,-1)) # Turn range into a list and sort in numerical order
>>> sorted(["a", "b", "ab", "z", "tutu", "aaa", "0", "2"]) # Sort in alphabetical/dictionary order
['a', 'aaa', 'ab', 'b', 'tutu', 'z']
>>> sorted([1, 2.3, 0.99, -23]) # Sort in numerical/incremental order
[-23, 0.99, 1, 2.3]
>>> sorted([1, 2, 3, 4, 'a', 'b', 2.4, 3]) # Python3 raise an exception
TypeError: unorderable types: str() < int()
# Sorting list of tuples use the first element of the tuple
>>> l = [ ('a',34), ('z', 23), ('b', 44)]
>>> sorted(l)
[('a', 34), ('b', 44), ('z', 23)]
# Sort based on 2nd field of tuple ~~~> reverse tuple
>>> l = sorted(list(map(lambda el: el[::-1], l)))
>>> sorted(l)
[(23, 'z'), (34, 'a'), (44, 'b')]
lst.sort#
lst.sort() # Like sorted, but change the list in place
lst.sort(reverse=True) # Sort in reverse order!
>>> list1=[[3,5,111],[16,23,21],[1,2,3,4],[100,1,31,12]]
>>> list1.sort(key=lambda x:x[1], reverse=True) # Sort based on a value extracted from each element/x!
[[16, 23, 21], [3, 5, 111], [1, 2, 3, 4], [100, 1, 31, 12]]
>>> list1=[[100,200,300,400,500],[100,200,300],[100,150,200,400]]
>>> list1.sort(key=len) # using a FUNCTION REFERENCE to sort the elements, here the length function of the element
print(list1)
[[100, 200, 300], [100, 150, 200, 400], [100, 200, 300, 400, 500]]
list1=[[10,200,300,40,500],[100,200,300],[100,150,200,400]]
list1.sort(key=sum) # using the key as sum on the element sum([1,2]) == 3
print(list1)
[[100, 200, 300], [100, 150, 200, 400], [10, 200, 300, 40, 500]]
map : Operation on every element of a list#
the output of map(...) is a map and not a list!l = list(range(9)) # Python3: Turn the iterator-like range into a list
l = list(map(str, range(9)) # Python3: Turn the map object into a list
l = [ ('a',34), ('z', 23), ('b', 44)]
r = list(map(lambda el: el[::-1], l)) # [(34, 'a'), (23, 'z'), (44, 'b')]
print(r)
[(44, 'b'), (23, 'z'), (34, 'a')]
# Map with a function instead of a lambda!
def my_function(value):
""" Double provided value """
return value + value
my_values = (1, 2, 3, 4) # Can be a tuple or a list
res = list(map(my_function, my_values))
res = list(map(lambda v: v + v, my_values)) # Map with a lambda function !
>>> d # beware list of chars, not index --> string multiplication!
['0', '1', '2', '3', '4', '5', '6', '7', '8']
>>> [ x*2 for x in d ]
['00', '11', '22', '33', '44', '55', '66', '77', '88']
# List of strings
list_of_strings = ['sat', 'bat', 'cat', 'mat']
# map() can listify the list of strings individually
test = list(map(list, list_of_strings))
test # returns [['s', 'a', 't'], ['b', 'a', 't'], ['c', 'a', 't'], ['m', 'a', 't']]
Working with 2 lists together (map)#
# Add two lists using map and lambda
x_values = [1, 2, 3]
y_values = [4, 5, 6]
result = map(lambda x, y: x + y, x_values, y_values)
iteration on the elements of a list#
ordered_words = ['aa', 'bb', 'ee', 'hh']
words = 'aaa,ddd'
def insert_word(word, ordered_words):
""" change ordered_words in place """
for i, el_word in enumerate(ordered_words):
if word < el_word: # Inequality on strings, sort based on ASCII characters' code! One by one chars
ordered_words.insert(i, word) # Equivalent to ordered_words = ordered_words[:i] + [word] + ordered_words[i:]
break
return(true)
words = arg_w.split(',')
for w in words:
insert_word(w, ordered_words)
print ordered_words
Introspection#
>>> l
[9, 7, 5, 3, 1]
>>> type(l)
list
>>> dir(l)
>>> dir(list)
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']
>>> help(list.extend)
extend(self, iterable, /)
Extend list by appending elements from the iterable.
Range#
Not quite an iterator
Range are iterator-like but not iterator, because they do not keep track of their state
- range is an iterable (not an iterator) that generates numbers lazily (when asked for it).
- You need to call iter(range(n)) to get an iterator from it (and get the 'next' method).
r = iter(range(3)) # Creates an iterator from range (call range.__iter__ method)
print(next(r)) # 0
print(next(r)) # 1
print(next(r)) # 2
print(next(r)) # ❌ Error! StopIteration
- Using range is memory-efficient compared to storing a full list.
Set#
Sets are Unordered immutable elements with no duplicate.
Sets are mutable
What is a set:
- Every element in set is unique (no duplicates)
- Sets can be used to perform mathematical set operations like union, intersection, symmetric difference etc.
- Every element in set must be immutable (which cannot be changed) like strings, numbers, tuple, or range, however, the set itself is mutable. We can add or remove items from it.
More at @ https://www.thegeekstuff.com/2019/04/python-set-examples/#more-17819
Hands-on#
Months={"Jan", "Feb", "Mar", "Feb"} # Will remove 1 "Feb" because not unique!
Dates={21,22,17}
>>> type(Dates)
set
Days = set(["Mon","Tue","Wed","Thu","Fri","Sat","Sun"]) # Turn a list in a set
>>> cars = ['honda','ford','dodge', 'honda'] # List
>>> autos = set(cars) # Create a set from a list
>>> autos
set(['dodge',chevy','honda', 'ford'])
>>> motos & autos # intersection of sets
>>> employees = engineers | programmers | managers # union of sets
>>> engineering_management = engineers & managers # intersection of sets
>>> managers_only = employeses - engineers - programmers # difference of sets
s.update(t) # s |= t return set s with elements added from t
s.intersection_update(t) # s &= t return set s keeping only elements also found in t
s.difference_update(t) # s -= t return set s after removing elements found in t
s.symmetric_difference_update(t) # s ^= t return set s ...
# ... with elements from s or t but not both
s.add(x) # add element x to set s (if x is not already there! If already there, tihs is a no-op!)
s.remove(x) # remove x from set s; raises KeyError if not present
s.discard(x) # removes x from set s if present (no exception)
s.clear() # remove all elements from set s
s.pop() # remove and return an arbitrary element from set s ...
# ... raises KeyError if empty
Source @ [[https://docs.python.org/2/library/sets.html||target='_blank']]
Introspection#
>>> s = set()
>>> type(s)
set
>>> dir(s)
>>> dir(set)
['__and__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__iand__', '__init__', '__ior__', '__isub__', '__iter__', '__ixor__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__or__', '__rand__', '__reduce__', '__reduce_ex__', '__repr__', '__ror__', '__rsub__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__xor__', 'add', 'clear', 'copy', 'difference', 'difference_update', 'discard', 'intersection', 'intersection_update', 'isdisjoint', 'issubset', 'issuperset', 'pop', 'remove', 'symmetric_difference', 'symmetric_difference_update', 'union', 'update']
>>> help(set.update)
update(...)
Update a set with the union of itself and others.
Frozenset#
A frozenset in Python is an immutable version of a set—once created, you cannot add or remove elements from it.
Key properties:
- Unordered collection of unique items (like a set)
- Immutable (unlike a set)
- Hashable, so it can be used as a key in dictionaries or as an element in other sets
When to use:
- When you need a set-like collection that must not change
- When using sets as keys in a dictionary
Hands-on#
# Create a frozenset
fs = frozenset([1, 2, 3, 2]) # duplicates are removed
print(fs) # Output: frozenset({1, 2, 3})
# Attempting to modify it will raise an error
fs.add(4) # ❌ AttributeError: 'frozenset' object has no attribute 'add'
Dictionary#
Immutable keys
keys must be of an immutable data type such as strings, numbers, or tuples.
d = {1: 'hello'} # d[1] = 'hello'
d = {(1,2): 'hi'} # d[(1,2)] = 'hi'
d = {'a_word': 'bonjour'}
d = {'c': 'hello'}
Hands-on#
How to set default values for the dictionary?
Use d.get('c', 'default_value')
How to remove a key from a dictionary?
Use "del d['key']"
# Width dict comprehension
my_dict = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
keys_to_remove = {'b', 'c'}
filtered_dict = {k: v for k, v in my_dict.items() if k not in keys_to_remove}
print(filtered_dict)
# with d.pop('key')
my_dict = {'a': 1, 'b': 2, 'c': 3}
value = my_dict.pop('b')
print(my_dict) # {'a': 1, 'c': 3}
print(value) # 2 (popped value)
>>> d = dict()
>>> d = {'key': 'value'} # Dicts uses curly braces for their representation
>>> d = {'a': 1, 'b': 9}
>>> d['key'] # Key must be unique and MUST EXIST ! Otherwise KeyError Exception
'value'
>>> d.get(key) # Key must be unique, if does NOT exist return None
>>> d.get('c') is None
True
>>> d.get(key) == 'value' # False if key does NOT exist !
False
>>> d.get(key, 'value_if_not_exist') # A key if doesn't exist <!> Unlike setdefault, does not update the dictionary
>>> d.get('c','toto') == 'toto'
True
>>> val = d.setdefault('toto', 2) # Same as g.get('toto', 2) that is returns the dict value, but will update the dictionary in place
>>> val = d.setdefault('titi') # Here the default value is None + update dictionary, same as d.setdefault('titi', None)
>>> d.setdefault('items', []).append(42) # Great usage to init dict entries on the fly, with liss or sets!
>>> d.setdefault('tata', {'sub': 1} ) # Can be a complex default value!
>>> d.setdefault('l', []).append(2) # will creaete the key/value pair if does not exist and append 2 to it <!> can fail if 'l' is different from a list!
>>> d['l2'] = d['l'] # <!> not a copy of the list, but a reference to it!
>>> d['l2'].append(3) # append 3 to both list !!!!!
d = {'a': 1, 'b': 2}
v = d.setdefault('c', 100) # Key 'c' doesn't exist, so it adds 'c': 100 and return 100
print(v) # 100
print(d) # {'a': 1, 'b': 2, 'c': 100}
d['key'] = 'value' # Add a new key/value pair in the dict
'key' in d.keys() # Check that the key is in dict (True)
del d['key'] # remove key/value pair
'key' in d.keys() # Check that the key is in dict (False)
d.keys() # returns all keys
d.values() # Returns all the values (same order as d.keys())
del dict['Name']; # remove entry with key 'Name'
dict.clear(); # remove all entries in dict, dict is empty dictionary
del dict ; # delete entire dictionary, dict variable does not exist anymore
>>> d.items() # Iterator-like, list of tuples, but not an iterator!
dict_items([('a', 1), ('b', 9)])
list(d.items())[1] # ('b', 2)
list(d.items())[0] # ('a', 1)
>>> for k in d # Iterate on the keys (same as d.keys())
... print(k)
>>> for kv in d.items(): # Get the kv tuples
print(kv) # print the tuple
>>> for key, value in d.items(): # Iterate on key and value pairs (was d.iteritems())
# returns a list of tuples
... print(key, ' -> ', value)
>>> d = {'a': 1, 'b': 2}
>>> 'a' in d # A replacement for d.has_key()
True
>>> d.get('a') != None
True
>>> d.get('X') == None # <!> When using 'dict.get', None is the default value if the key does not exist
True
map on dictionary (rebuild ALL k-v pairs)#
. <!> Also known as dictionary comprehension
def f(value):
return value**2
my_dictionary = {k: f(v) for k, v in my_dictionary.items() if k == 1} # Dict comprehension with filtering-condition!
# or less readable
my_dictionary = dict(map(lambda kv: (kv[0], f(kv[1]) ), d.items())) # d.items is a LIST of tuples fed to the lambda function!
# notice that the map is a map of tuples ... that is then turned into a dict
# similar to what d.items() would return... kv tuples!
Source @ https://stackoverflow.com/questions/12229064/mapping-over-values-in-a-python-dictionary
= vs shallow copy method vs copy.deepcopy function#
. <!> with =, the dictionary points to exactly the same content (= identity ~ pointer) - change to immutable/mutable vars are propagated * list or 'list_1 = list_2', this is not a copy but 2 variables pointing to the same memory space . <!> with the copy dictionary method, the immutable are 'duplicated', the mutable are the same and change together . <!> with copy.deepcopy, the 2 dictionaries are completely independent
>>> d = {'a': 1, 'b': 2}
>>> dd = d # d and dd are the same !
# A change in one = a change in the other!
>>> id(d) == id(dd)
True
>>> dd['a'] = 3
>>> print(d['a'])
3
import copy # required for deepcopy, but not for copy!
otherDict = copy.deepcopy(wordsDict) # Create a deep copy of the dictionary
. <!> With shallow copy, ---> copy the object identities (values in particular, since key are immutables) * the dictionary values that are mutable (list,...) when changed in one change in the other * immutable (numbers, strings, tuple values) are NOT shared * copy mutable (list dict, and set) by reference ==> change a mutable's value == change in all dictionary!
# create a Shallow copy the original dictionary
d = {'a': 1, 'b': 2, 'c': [1, 2, 3, 45]}
d_copy = d.copy()
d_copy["c"].append(222)
print d # The original dictionary has been changed!
{'a': 1, 'c': [1, 2, 3, 45, 222], 'b': 2}
>>> d_copy['e'] = [3, 5, 4] # Add new element
>>> d
{'a': 1, 'c': [1, 2, 3, 45, 222], 'b': 2}. # new key is not in original dictionary !!!
>>> d_copy
{'a': 1, 'c': [1, 2, 3, 45, 222], 'b': 2, 'e': [3, 5, 4]}
Source @ [[https://thispointer.com/python-how-to-copy-a-dictionary-shallow-copy-vs-deep-copy/||target='_blank']]
import copy
# Original dictionary
original = {'a': 1, 'b': [2, 3]}
# Creating a shallow copy
shallow = copy.copy(original)
# Modifying the nested list
shallow['b'].append(4)
print(original) # {'a': 1, 'b': [2, 3, 4]}
print(shallow) # {'a': 1, 'b': [2, 3, 4]} (Both changed!)
# other ways to create shallow copies
shallow = original.copy() # Built-in method
shallow = dict(original) # Using dict() constructor
shallow = {k: v for k, v in original.items()} # Dictionary comprehension
# DEEP Copy
✅ All nested objects are also copied.
✅ No shared references—each copy is fully independent.
import copy
# Original dictionary with nested structure
original = {'a': 1, 'b': [2, 3]}
# Creating a deep copy
deep = copy.deepcopy(original)
# Modifying the nested list
deep['b'].append(4)
print(original) # {'a': 1, 'b': [2, 3]} (Original unchanged)
print(deep) # {'a': 1, 'b': [2, 3, 4]} (Deep copy modified separately)
- Use copy.copy() (shallow copy) when your dictionary contains only immutable values (numbers, strings, tuples).
- Use copy.deepcopy() (deep copy) when your dictionary has nested mutable objects (lists, sets, other dictionaries) and you want full independence.
Hidden dictionary#
Given a list of words, group the anagram in a sublist together
['abc', 'bar', 'cab'] ---> [ ['cab', 'abc'], ['bar']]
Def ana(L): # L is list of words
anagrams = {}
For word in L:
C = sorted(word) # Reorder the chars of the word
If C not in anagrams: # C is not a key in the dictionary
anagrams[C] = []
anagrams[C].append(word) # = anagram_list
Return list(anagrams.values()) # return the values of anagrams dict
map on dict = dict comprehension#
. <!> Not quite an iterator but known as dictionary comprehension
def f(value):
return value**2
my_dictionary = {k: f(v) for k, v in my_dictionary.items()}
# or less readable
my_dict = dict(map(lambda kv: (kv[0], f(kv[1])), my_dict.items()))
my_dict = dict(
map(lambda kv: (kv[0], f(kv[1]) ) , my_dict.items()) # dict( [ (k, v), ... ] ) ~~> {k:v, ...}
)
# or
for k, v in d.items():
d['k'] = f(v)
Source @ https://stackoverflow.com/questions/12229064/mapping-over-values-in-a-python-dictionary
A dictionary of functions (switch ... case ...)#
. <!> A dictionary of function . <!> Key in dictionary are immutable numbers
import math
from math import sqrt
def zero(): # zero is a variable-reference to the function object!
print("You typed zero.\n")
type(zero) # <class 'function'>
def sqr():
print("n is a perfect square\n")
def even():
print("n is an even number\n")
def prime():
print("n is a prime number\n")
# All numbers from 0 to 9 are 'switch' keys, or keys in the option dictionary
options = {
0 : zero,
1 : sqr, # function name/reference!
2 : even,
3 : prime,
4 : sqr, # beware function need to be defined before reference is used
5 : prime,
6 : even,
7 : math.sqrt, # import ...
8 : sqrt, # from math ...
9 : lambda x : x*x, # yes, you can have 2 colon-chars in the same line ! The comma is the entry separator!
# * : lambda : None, # See how to set the default function if not found!
}
options[num]()
You gould also handle the missing default case by doing this instead:
options.get(num, lambda : None)() # Default value for the function !
>>> d.get('*', lambda: None)() == None # Beware: This is different from d.get('*', None)() which generate an exception because None() or None is not callable!
True # A lambda is callable!
Which will return None if num is not in options.
<
json.load vs eval#
import json
# Open and read the JSON file
with open('data.json', 'r') as file:
data = json.load(file)
# Use the parsed data
print(data['name']) # Output: Alice
print(data['age']) # Output: 30
print(data['is_student']) # Output: False
- string of a dict to dict
- source @ https://stackoverflow.com/questions/988228/convert-a-string-representation-of-a-dictionary-to-a-dictionary . Ex API call response
introspection#
>>> d = {}
>>> type(d)
<class 'dict'>
>>> dir(dict)
['__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']
>>> help(dict.keys())
keys(...)
D.keys() -> a set-like object providing a view on D's keys
Function#
Object#
class Vector2D:
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
if isinstance(other, Vector2D):
return Vector2D(self.x + other.x, self.y + other.y)
return NotImplemented
def __mul__(self, scalar):
if isinstance(scalar, (int, float)):
return Vector2D(self.x * scalar, self.y * scalar)
return NotImplemented
def __rmul__(self, scalar):
return self.__mul__(scalar) # for scalar * vector
def __repr__(self):
return f"Vector2D({self.x}, {self.y})"