zip函数

zip() 函数用于将多个列表(或者其它可迭代的对象)中对应的元素, 打包成一个个元组, 然后返回由这些元组组成的列表(Python3返回一个对象).

如果各个迭代器的元素个数不一致, 则返回列表长度与最短的对象相同, 利用 * 号操作符, 可以将元组解压为列表.

Note: 在 Python 3 中为了减少内存, zip() 返回的是一个对象. 如需展示列表, 需手动 list() 转换.

1
2
3
4
5
6
7
8
9
10
11
12
13
a = [1,2,3,4]
b = ['a','b','c','d']
c = ['A','B','C','D']
zipped = list(a,b,c) #将3个列表打包成一个对象

for i,j,k in zipped:
    print(i,j,k)
    
#输出
1 a A
2 b B
3 c C
4 d D

zipped的数据类型实际为列表

1
[(1, 'a', 'A'), (2, 'b', 'B'), (3, 'c', 'C'), (4, 'd', 'D')]

列表中的元素类型为元组

1
2
3
4
(1, 'a', 'A') <class 'tuple'>
(2, 'b', 'B') <class 'tuple'>
(3, 'c', 'C') <class 'tuple'>
(4, 'd', 'D') <class 'tuple'>

enumerate

enumerate(iterable, start=0) | docs

Python enumerate() 函数 | 菜鸟教程

enumerate() 是python的内置函数, 常用于在循环中计数. 它有两个参数, 第一个参数必须是列表, 元组等可迭代的数据结构; 第二个参数是开始的数字, 一般取0(默认)和1, 表示从0, 1开始计数.

1
2
3
4
5
6
7
8
array = ['a', 'b', 'c']
for index, item in enumerate(array,1):
    print("%d, %s" % (index,item))
    
#输出
1, a
2, b
3, c

迭代器(Iterators)

python iterators | w3school

python next() | w3school

python iter() | w3school

Shallow and deep copy

面向对象

类的方法

在类的内部，使用 def 关键字来定义一个方法，与一般函数定义不同，类方法必须包含参数 self, 且为第一个参数，self 代表的是类的实例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class People:
    # 定义基本属性
    name = ''
    age = 0
    # 定义构造方法
    def __init__(self,n,a):
        self.name = n
        self.age = a
    # 定义一个方法
    def speak(self):
        print("%s : %d" % (self.name, self.age))

# 创建对象,调用方法
p = People("Zhang", 18)
p.speak()

self代表什么

类的方法与普通的函数只有一个特别的区别——它们必须有一个额外的第一个参数, 按照惯例它的名称是 self。self代表类所定义的实例, 而非类本身.

Note: self并不是python关键词, 将其换成其它单词也是可以的, 只不过惯例用的是self.

构造方法

类有一个名为 __init__() 的特殊方法（构造方法），该方法在类实例化时会自动调用。每个类都会有一个默认的构造方法，即使没有定义也会有一个空的构造方法。一旦用户自定义了构造方法，默认的空的构造方法就会被覆盖。

1
2
3
4
5
6
7
class Complex:
    def __init__(self, realpart, imagpart):
        self.r = realpart
        self.i = imagpart

x = Complex(3.0, -4.5)
print(x.r, x.i)

单继承

子类(派生类, DerivedClass)会继承父类(基类, BaseClass)的属性和方法

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#定义父类
class Base():
    def __init__(self):
        self.arg = 0
        print("Base.__init__ is called.")
    def menthod(self):
        print("Base.menthod is called.")
#定义子类
class DerivedClass(Base):
    def __init__(self):
        Base.__init__(self)

test = DerivedClass()
#输出 Base.__init__ is called.
test.menthod()
#输出 Base.menthod is called.
print(test.arg)
#输出 0

构造方法也同其它方法一样会被继承, 当子类没有特别定义构造方法时, 子类在创建实例(对象)时会默认调用父类的构造方法

1
2
3
4
5
6
7
8
9
10
#父类
class Base():
    def __init__(self):
        print("Base.__init__ is called.")
#子类
class DerivedClass(Base):
    arg = 1
#父类的构造方法被调用
test = DerivedClass()
#输出 Base.__init__ is called.

多继承

多继承的定义形式如下

1
2
3
4
class DerivedClassName(Base1, Base2, Base3):
    <statement-1>
    ...
    <statement-N>

Note: 需要注意圆括号中父类的顺序，若是父类中有相同的方法名，而在子类使用时未指定，python从左至右搜索 即方法在子类中未找到时，从左到右查找父类中是否包含方法

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# 父类1
class Base1():
    def __init__(self):
        print("Base1.__init__ is called.")
    def menthod(self):
        print("Base1.menthod is called.")

#父类2
class Base2():
    def __init__(self):
        print("Base2.__init__ is called.")
    def menthod(self):
        print("Base2.menthod is called.")

#多继承
class DerivedClass(Base1,Base2):
    def __init__(self):
        #调用父类的构造方法
        Base1.__init__(self)
        Base2.__init__(self)

test = DerivedClass()
#输出
#Base1.__init__ is called.
#Base2.__init__ is called.
test.menthod()
#输出
#Base1.menthod is called.

方法重写

如果你的父类方法的功能不能满足你的需求，你可以在子类重写你父类的方法，实例如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#定义父类
class Base():
    def __init__(self):
        print("Base.__init__ is called.")
    def menthod(self):
        print("Base.menthod is called.")
#定义子类
class DerivedClass(Base):
    def __init__(self):
        Base.__init__(self)
    #重写父类的menthod方法
    def menthod(self):
        print("menthod is overwrited.")

test = DerivedClass()
#输出 Base.__init__ is called.
test.menthod()
#输出 menthod is overwrited.

私有

方法名为双下划线开头的方法为私有方法; 属性名为双下划线开头的属性为私有属性.

私有方法和私有属性不能在类外被直接调用或访问, 只能在类内被调用和访问. 需要在类外访问私有属性和调用私有方法时, 只能通过类的"接口"间接地调用.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
class Base():
    __arg = 0
    def __menthod(self):
        print("private __menthod is called.")
    def menthod(self):
        # print("menthod")
        self.__menthod()
        print(self.__arg)

test = Base()
#不能直接访问私有属性
test.__arg
#Error
#AttributeError: 'Base' object has no attribute '__arg'

#不能直接调用私有方法
test.__menthod()
#Error
#AttributeError: 'Base' object has no attribute '__menthod'

#通过接口类外间接访问私有属性和方法
test.menthod()
#输出
#private __menthod is called.
#0

关于super

Python super() 详解 最简单的解释

关于call

装饰器(Decorators)

python decorators | Runoob

装饰器 | 廖雪峰

使用@property | 廖雪峰

装饰器是一种特殊的函数, 它的输入是函数对象, 并且返回一个函数对象

一个简单的例子

定义一个简单的装饰器如下, 这个装饰器返回输入的函数对象, 并在返回之前print了一句说明.

1
2
3
def decorator1(func):
    print("There is a simple decorator.")
    return func

使用@符号来调用装饰器, 对上面所定义的简单装饰器进行调用如下

1
2
3
@decorator1
def show(s):
    print(s)

这样做相当于进行了如下过程

show=decorator1(show)\rm show = decorator1(show)

show 作为一个函数对象传入 decorator1 , decorator1 对 show 进行了一系列操作(进行了print输出)后返回 show. 虽然还是同名为 show , 但是所指向的函数对象不一样了.

对经过 decorator1 修饰后的 show 进行调用 show("abc")

1
2
3
#输出 show("abc")
There is a simple decorator.
abc

此时的 show("abc") 相当于进行了如下操作, 其中show^\hat{\rm show}是经过decorator1修饰后的函数, show\rm show表示原函数.

show^(“abc")=decorator1(show)‾(“abc")=⋯return show(“abc")={⋯ }+show(“abc")\begin{split} \rm \hat{show}(\textcolor{green}{\text{``abc"}}) &\rm = \underline{decorator1(show)}(\textcolor{green}{\text{``abc"}})\\ &\rm = \fbox{$\begin{array}{l}\cdots\\ \textcolor{purple}{\rm return}\,\rm show \end{array}$}(\textcolor{green}{\text{``abc"}})\\ &\rm = \{\cdots\}+ show(\textcolor{green}{\text{``abc"}}) \end{split}

一个略微复杂的例子

下面的装饰器的定义中额外定义了一个函数对象 wrapper , 装饰器返回这个wrapper函数对象

1
2
3
4
5
6
7
def decorator2(func):
    print("decorator")
    def wrapper(*args, **keys):
        print("wrapper")
        func(*args, **keys)

    return wrapper

对这个例子进行简单调用

1
2
3
@decorator2
def show(s):
    print(s)

输出show("abc")

1
2
3
decorator
wrapper
abc

对decorator2修饰的函数进行调用相当于进行了如下操作

show^(“abc")=decorator2(show)‾(“abc")=⋯def wrapper(∗args)⋯return wrapper(“abc")={⋯ }+wrapper(“abc")\begin{split} \rm \hat{show}(\textcolor{green}{\text{``abc"}}) &\rm = \underline{decorator2(show)}(\textcolor{green}{\text{``abc"}})\\ &\rm = \fbox{$ \begin{array}{l} \cdots\\ \fbox{$\begin{array}{c}\textcolor{purple}{\rm def}\, \rm{wrapper(*args)}\\ \cdots\end{array}$} \\ \textcolor{purple}{\rm return}\,\rm{wrapper} \end{array} $} (\textcolor{green}{\text{``abc"}})\\ &\rm = \{\cdots\}+ wrapper(\textcolor{green}{\text{``abc"}}) \end{split}

@functools.wraps(func)

注意在上面的第2个示例中, 输出 show.__name__ 得到的结果是 wrapper. 因为虽然变量名字是 show , 但是实际指向的函数对象是在decorator2中定义的 wrapper.

使用functools工具包中的wraps装饰器可以避免这个现象

在wrapper的定义前加上@functools.wraps(func)即可

1
2
3
4
5
6
7
import functools
def decorator2(func):
    ...
    @functools.wraps(func)
    def wrapper(*args, **keys):
        ...
    return wrapper

如何给装饰器传入参数

当需要给装饰器传入参数的时候需要额外的嵌套一层函数用于传入装饰器的参数

1
2
3
4
5
6
7
8
9
10
11
def decorator3_with_args(*decorator_args):
    print(*decorator_args) #输出装饰器的参数
    #在传参函数的内部再定义一个装饰器decorator3
    def decorator3(func):  
        print("we are in decorator.")
        #在装饰器内部定义一个新的函数对象,作为装饰器的返回对象
        def wrapper(*wrapper_args):
            print("we are in wrapper.")
            func(*wrapper_args)
        return wrapper
    return decorator3

对上述定义的装饰器进行调用

1
2
3
@decorator3_with_args("decorator's args")
def show(s):
    print(s)
1
2
3
4
5
#输出 show("show's args")
decorator's args
we are in decorator.
we are in wrapper.
show's args

这个例子的结构图可以如下表示

show^(“fun’s args")=decorator3_with_args(“dec’s args")‾(show)(“fun’s args")=⋯return decorator3(show)‾(“fun’s args")=⋯def wrapper(∗args)⋯return wrapper(“fun’s args")={⋯ }+wrapper(“fun’s args")\begin{split} \rm \hat{show}(\textcolor{green}{\text{``fun's args"}}) &\rm = \underline{decorator3\_with\_args(\textcolor{green}{\text{``dec's args"}})}(show)(\textcolor{green}{\text{``fun's args"}})\\ &\rm = \underline{\fbox{$\begin{array}{l}\cdots\\ \textcolor{purple}{\rm return}\,\rm decorator3\end{array}$}(show)}(\textcolor{green}{\text{``fun's args"}})\\ &\rm = \fbox{$ \begin{array}{l} \cdots\\ \fbox{$\begin{array}{c}\textcolor{purple}{\rm def}\, \rm{wrapper(*args)}\\ \cdots\end{array}$} \\ \textcolor{purple}{\rm return}\,\rm{wrapper} \end{array} $} (\textcolor{green}{\text{``fun's args"}})\\ &\rm = \{\cdots\}+ wrapper(\textcolor{green}{\text{``fun's args"}}) \end{split}

List(列表)

  • 列表可以被索引和截取，列表被截取后返回一个包含所截取元素的新列表
  • 列表截取的语法格式如下：
1
2
变量[头下标:尾下标:步长]
# 注意: 尾下标不被截取 (下图的蓝色和紫色线的尾点是空心的)

list_slicing1_new2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
L = ['a', 'b', 'c', 'd', 'e']
#idx: 0    1    2    3    4
#idx:-5   -4   -3   -2   -1

print(L[5])     # IndexError: list index out of range
print(L[-6])    # IndexError: list index out of range
# -----------------------------------------------------
print(L[:0])      # []
print(L[:1])      # ['a']
print(L[:4])      # ['a', 'b', 'c', 'd']
print(L[:5])      # ['a', 'b', 'c', 'd', 'e']
print(L[:6])      # ['a', 'b', 'c', 'd', 'e']
# -----------------------------------------------------
print(L[:])       # ['a', 'b', 'c', 'd', 'e']
# -----------------------------------------------------
print(L[:-1])     # ['a', 'b', 'c', 'd']
print(L[:-2])     # ['a', 'b', 'c']
print(L[:-3])     # ['a', 'b']
print(L[:-4])     # ['a']
print(L[:-5])     # []
print(L[:-6])     # []
# -----------------------------------------------------
print(L[0:0])     # []
print(L[0:1])     # ['a']
print(L[0:5])     # ['a', 'b', 'c', 'd', 'e']
print(L[0:6])     # ['a', 'b', 'c', 'd', 'e']
# -----------------------------------------------------
print(L[0:])      # ['a', 'b', 'c', 'd', 'e']
print(L[1:])      #      ['b', 'c', 'd', 'e']
# -----------------------------------------------------
print(L[-1:])     #                     ['e']
print(L[-2:])     #                ['d', 'e']
print(L[-1:0])    # []
print(L[-2:0])    # []
# -----------------------------------------------------
print(L[0:4:2])   # ['a', 'c']
print(L[0:5:2])   # ['a', 'c', 'e']
print(L[0:6:2])   # ['a', 'c', 'e']
# -----------------------------------------------------
print(L[::-1])    # ['e', 'd', 'c', 'b', 'a']
print(L[4:0:-1])  # ['e', 'd', 'c', 'b']
print(L[4:-1:-1]) # []
print(L[4:-2:-1]) # ['e']

random

random.choice
random.choices

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from torchvision import datasets
import random

# loading data
trainset = datasets.MNIST(root="dataset/", train=True, download=True)

# random choice
data = random.choice(trainset)
print(type(data)) # <class 'tuple'>

# random choices
datas = random.choices(trainset, k=3)
print(type(datas)) # <class 'list'> 
print(type(datas[0])) # <class 'tuple'>

trainloader = DataLoader(dataset=trainset, batch_size=16)
loader = random.choice(trainloader)
'''
TypeError: 'DataLoader' object is not subscriptable
'''

os

1
2
3
4
5
import os

os.path.listdir()
os.path.join()
os.path.isdir()

os.path

os.path | python docs

os.path.basename | python docs

In Python, use the os.path.exists() method to see if a directory already exists, and then use the os.makedirs() method to create it.

1
2
3
import os
if not os.path.exists(path_str):
    os.makedirs(path_str)

sys

How to get python version numbers?

1
2
3
4
import sys

print(sys.version_info.major) # 3 in my case
print(sys.version_info.minor) # 8 in my case

glob

glob | python docs

Relative path

Relative paths are relative to the working directory, not where the Python file is.

The working directory is the root of your terminal that runs your code with command (conda_env) Path\to\cwd>python code.py

The getcwd() function from the os module fetches the current working directory location. The function returns the full path starting from the root directory to the current working directory.

1
2
import os
print(os.getcwd())

You can change the working directory using the os.chdir(<path>) method, and you can test whether a path, either directory, or file, exists using the os.path.exists(<path>) method.

Sub folder keeps the same working directory as main.

1
2
3
4
5
6
# Folder Structure:
test.py
package
├── __init__.py
└── module.py : import os
                def returncwd(): return os.getcwd()
1
2
3
4
5
6
7
8
9
10
# D:\Code\Python\test.py

import os
print(os.getcwd())
# D:\Code\Python

from package.module import returncwd
print(returncwd())
# D:\Code\Python
# Note: it is ~\Pthon not ~\Python\package

When debug in vscode, you can change the cwd by modifing the launch.json file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug test",
      "type": "python",
      "request": "launch",
      "program": "${workspaceFolder}/test.py",
      "console": "integratedTerminal",
      "cwd": "${workspaceFolder}/package", // <-- change your cwd to the path that you
                                           // want your relative paths be relative to
      "justMyCode": true
    }
  ]
}

double backslashes \\ & single forward slash /

Python treat a backslash (\) as an escape character. For instance, \n represents a line feed, and \t represents a tab. When specifying a path, a forward slash (/) can be used in place of a backslash. Two backslashes can be used instead of one to avoid a syntax error.

1
2
# D:\Code\Python\text.txt
Hello World!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# single backslash
with open("D:\Code\Python\text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # OSError: [Errno 22] Invalid argument: 'D:\\Code\\Python\text.txt'

# raw strings
with open(r"D:\Code\Python\text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # Hello World!

# double backslashes
with open("D:\\Code\\Python\\text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # Hello World!

# single forward slash
with open("D:/Code/Python/text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # Hello World!

When you use double backslashes // or single forward slash \ before the last item of the path, it workes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# \\ before text.txt
with open("D:\Code\Python\\text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # Hello World!

# mix backslash and forward slash ended with \\
with open("D:/Code\Python\\text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # Hello World!

# mix backslash and forward slash ended with /
with open("D:/Code\Python/text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # Hello World!

# mix backslash and forward slash ended with \
with open("D:/Code\Python\text.txt", encoding="utf-8") as f:
  content = f.read()
  print(content)
  # OSError: [Errno 22] Invalid argument: 'D:/Code\\Python\text.txt'

json

python_json | w3schools

*传参数

1
2
3
4
nums = [1, 2, 3]

print(*nums) # 1 2 3
print(nums)  # [1, 2, 3]

*nums表示把nums这个list的所有元素作为可变参数传进去。这种写法相当有用，而且很常见。
—— 函数的参数 | 廖雪峰 Python 教程

判断变量是否可迭代

1
2
3
4
# 方法一
from collections import Iterable

print(isinstance("str", Iterable)) # True
1
2
# 方法二
print(hasattr("str", "__iter__"))  # True

collections.Counter

collections.Counter | docs.python.org

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
from collections import Counter

iter_list = ["A", "A", "B", "A", "B", "C"]
counter = Counter(iter_list)

print(counter) # Counter({'A': 3, 'B': 2, 'C': 1})
print(type(counter)) # <class 'collections.Counter'>

# 可迭代
print(hasattr(counter, "__iter__")) # True

# 调用自带的.items()方法可以访问counter的内容
for item in counter.items():
    print(item, type(item))
    # ('A', 3) <class 'tuple'>
    # ('B', 2) <class 'tuple'>
    # ('C', 1) <class 'tuple'>

# 统计出现次数最多的元素
most = counter.most_common(1)
print(most, type(most)) #[('A', 3)] <class 'list'>

# 查询具体元素的个数
print(counter["A"]) # 3
print(counter["B"]) # 2
print(counter["C"]) # 1

# elements() 方法按照元素首次出现的顺序, 列出所有元素
for element in counter.elements():
    print(element)
    # A A A B B C
# 可以使用 list() 转换成列表输出
print(list(counter.elements())) # ['A', 'A', 'A', 'B', 'B', 'C']

# 使用 sorted(), 结合 Key Functions 按照元素出现的次数进行排序
freqs = sorted(counter.items(), key=lambda x: x[1], reverse=False)
print(freqs) # [('C', 1), ('B', 2), ('A', 3)]

sorted()

sorted() | docs.python.org
Sorting HOW TO | docs.python.org

len、shape、size

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import numpy as np
import torch as tc

# Elements of array can be numeric or string type:
array = np.array([1, 2, 3])
array = np.array(['a', 'b'])

# Elements of tensor can only be numeric, and string type is not allowed:
tensor = tc.tensor([1, 2, 3])

# Error
tensor = tc.tensor(['a', 'b'])
#     tensor = tc.tensor(['a', 'b'])
# ValueError: too many dimensions 'str'

# Error
tensor = tc.tensor([1, 'b'])
#     tensor = tc.tensor([1, 'b'])
# TypeError: new(): invalid data type 'str'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import numpy as np
import torch as tc

list = [[1, 2, 3],
        [4, 5, 6]]
array = np.array(list)
tensor = tc.tensor(list)

# list
print(list.__len__()) # 2
print(len(list))      # 2

# array
print(array.__len__()) # 2
print(len(array))      # 2

# array-np.shape
print(array.shape)     # (2, 3)
print(np.shape(array)) # (2, 3)
print(np.shape(array)[0]) # 2
print(np.shape(array)[1]) # 3

# array-np.size
print(array.size)      # 6
print(np.size(array))  # 6
print(np.size(array, 0))  # 2
print(np.size(array, 1))  # 3

# tensor
print(tensor.__len__()) # 2
print(len(tensor))      # 2
# tensor.shape
print(tensor.shape)     # torch.Size([2, 3])
print(tensor.shape[0])  # 2
print(tensor.shape[1])  # 3
# np.shape()
print(np.shape(tensor))    # torch.Size([2, 3])
print(np.shape(tensor)[0]) # 2
print(np.shape(tensor)[1]) # 3

# tensor.size()
print(tensor.size)      #<built-in method size of Tensor object at 0x00000245A8814F40>
print(tensor.size())    # torch.Size([2, 3])
# np.size()
print(np.size(tensor))  #<built-in method size of Tensor object at 0x00000245A8814F40>
print(np.size(tensor)())# torch.Size([2, 3])
print(np.size(tensor, 0))  # 2
print(np.size(tensor, 1))  # 3

对于 tensor 即使不导入 numpy 模块，也可以使用 .shape 和 .size() 来返回 tensor 的形状

1
2
3
4
5
6
7
8
9
10
11
12
import torch as tc

list = [[1, 2, 3],
        [4, 5, 6]]
tensor = tc.tensor(list)

print(tensor.shape)   # torch.Size([2, 3])
print(tensor.shape[0])# 2
print(tensor.shape[1])# 3
print(tensor.size())  # torch.Size([2, 3])
print(tensor.size(0)) # 2
print(tensor.size(1)) # 3

len() 只能返回第一层的元素数量

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import numpy as np
import torch as tc

l = list([[1, 2, 3], [4, 5, 6]])
n = np.array(l)
t = tc.tensor(l)

print(l, len(l), type(l))
print(n, len(n), type(n))
print(t, len(t), type(t))

# [[1, 2, 3], [4, 5, 6]] 2 <class 'list'>
# [[1 2 3]
#  [4 5 6]] 2 <class 'numpy.ndarray'>
# tensor([[1, 2, 3],
#         [4, 5, 6]]) 2 <class 'torch.Tensor'>

查看包的安装位置

1
2
import module
print(module.__file__)
1
2
3
import torch
print(torch.__file__)
# C:\Users\xiaophai\.conda\envs\pytorch\lib\site-packages\torch\__init__.py

Something about path

1
2
# filename: test.py
print(os.path.basename(__file__)) # test.py

time

In Python, the time.time() function returns the number of seconds passed since epoch (the point where time begins).

What is epoch time?
Check this Epoch | wikiwand, you will have the answer.

Different systems have different epoch time.
For instance, Unix and POSIX measure time as the number of seconds that have passed since Thursday 1 January 1970 00:00:00 UT, a point in time known as the Unix epoch.
Windows NT systems, up to and including Windows 11 and Windows Server 2022, measure time as the number of 100-nanosecond intervals that have passed since 1 January 1601 00:00:00 UTC, making that point in time the epoch for those systems.

UTC means Coordinated Universal Time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import time

# Record the time of now
start_time = time.time()

# It is a number meaning how many seconds there have been since 1 January 1601 00:00:00 UTC
print(start_time)
# 1691678385.7681801 

# Pause this program for 10 secondes
time.sleep(10)

print("Total time: {:.2f} seconds".format((time.time() - start_time)))
# Total time: 10.01 seconds
print("Total time: {:.2f} minutes".format((time.time() - start_time) / 60))
# Total time: 0.17 minutes

Python format 格式化函数

cProfile

The Python Profilers

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# test.py
import time

def factorial(n):
  if n==0:
    return 1
  elif n>=1:
    return n*factorial(n-1)

def waste(t):
  time.sleep(t)

if __name__ == "__main__":
    for i in range(10):
        print(factorial(i))
        waste(0.25)
  • Using in terminal
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# terminal
PS ...> python -m cProfile  test.py
# ...
#          88 function calls (43 primitive calls) in 2.527 seconds

#    Ordered by: standard name

#    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
#         1    0.000    0.000    2.527    2.527 draft.py:1(<module>)
#     55/10    0.000    0.000    0.000    0.000 draft.py:3(factorial)
#        10    0.000    0.000    2.524    0.252 draft.py:9(waste)
#         1    0.000    0.000    2.527    2.527 {built-in method builtins.exec}
#        10    0.003    0.000    0.003    0.000 {built-in method builtins.print}
#        10    2.524    0.252    2.524    0.252 {built-in method time.sleep}
#         1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
  • Using in python code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import time
import cProfile
import pstats

def factorial(n):
  if n==0:
    return 1
  elif n>=1:
    return n*factorial(n-1)

def waste(t):
  time.sleep(t)

if __name__ == "__main__":
    with cProfile.Profile() as profile:
        for i in range(10):
            print(factorial(i))
            waste(0.25)
    
    results = pstats.Stats(profile)
    results.sort_stats(pstats.SortKey.TIME)
    results.print_stats()
#          87 function calls (42 primitive calls) in 2.581 seconds

#    Ordered by: internal time <-- Note

#    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
#        10    2.577    0.258    2.577    0.258 {built-in method time.sleep}
#        10    0.003    0.000    0.003    0.000 {built-in method builtins.print}
#     55/10    0.000    0.000    0.000    0.000 d:/Code/Python/GeoSeg/draft.py:5(factorial)
#        10    0.000    0.000    2.577    0.258 d:/Code/Python/GeoSeg/draft.py:11(waste)
#         1    0.000    0.000    0.000    0.000 C:\Users\xiaophai\.conda\envs\geoseg\lib\cProfile.py:134(__exit__)
#         1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

tuna

1
(conda_env) ...>pip install tuna
1
2
(conda_env) ...>python -m cProfile -o cprofile_output.prof -s tottime test.py
(conda_env) ...>tuna cprofile_output.prof

for/else

for/else

1
2
3
4
5
6
7
8
for n in range(2, 10):
    for x in range(2, n):
        if n % x == 0:
            print( n, 'equals', x, '*', n/x)
            break
    else:
        # loop fell through without finding a factor
        print(n, 'is a prime number')

Python debugging in VSCode

Debugging | vscode doc

Python debugging in VS Code | vscode doc

Set configuration options | vscode doc

an example of mutipul configurations

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: config1_script",
      "type": "debugpy",
      "request": "launch",
      "program": "${workspaceFolder}/script.py",
      "args" : ["zhang", "--age", "18", "--gender", "male"],
      "console": "integratedTerminal",
      "justMyCode": true
    },
    {
      "name": "Python: config2_train",
      "type": "debugpy",
      "request": "launch",
      "program": "${workspaceFolder}/mmsegmentation/tools/train.py",
      "args": "configs/unet/unet_s5-d16_deeplabv3_4xb4-40k_chase-db1-128x128.py",
      "console": "integratedTerminal",
      "cwd": "${workspaceFolder}/mmsegmentation",
      "justMyCode": true
    }
  ]
}

Settings of VSCode

variables and watch

Packages and __init__.py

Packages | python docs

When there is no __init__.py in the package,

1
2
3
4
5
6
7
# Folder Structure:
mian.py
package
├── moduleA.py : def printA(): print("A")
|
└── subpack
    └── moduleB.py : def printB(): print("B")

If I want to import a module or method from the package, I need to specify the location of the module or method.

1
2
3
4
5
6
# main.py
from package.moduleA import printA
from package.subpack.moduleB import printB

printA() # A
printB() # B

When there is an __init__.py in the package,

1
2
3
4
5
6
7
8
# Folder Structure:
mian.py
package
├── __init__.py
├── moduleA.py : def printA(): print("A")
|
└── subpack
    └── moduleB.py : def printB(): print("B")

Package Relative Imports | python docs

the imports can be done in the __init__.py, so it is more convenient to do the imports in the main.py.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# package/__init__.py
if __name__ == '__main__':
    from moduleA import printA
    from subpack.moduleB import printB
else:
    # use relative imports
    from .moduleA import printA
    from .subpack.moduleB import printB

# -----------------------------
# main.py

# from package.moduleA import printA
# from package.subpack1.moduleB import printB
from package import printA
from package import printB

printA() # A
printB() # B

Importing * From a Package
Besides, you can define a list named __all__ in the __init__.py, which contain the names of modules or methods that you have imported in the __init__.py. When you use from package import * in the main.py, all of the modules or methods in the __all__ list will be imported.

1
2
3
4
5
6
7
8
# Folder Structure:
mian.py
package
├── __init__.py
├── moduleA.py : def printA(): print("A")
|
└── subpack
    └── moduleB.py : def printB(): print("B")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# package/__init__.py
if __name__ == '__main__':
    from moduleA import printA
    from subpack.moduleB import printB
else:
    # use relative imports
    from .moduleA import printA
    from .subpack.moduleB import printB

__all__ = ['printA', 'printB']

# -----------------------------
# main.py

# from package import printA
# from package import printB
from package import *

printA() # A
printB() # B

relative and absolute imports

1
2
3
4
5
6
7
8
# Folder Structure:
mian.py
package
├── moduleA.py : def printA(): print("A")
|
└── subpack
    ├── moduleB.py : def printB(): print("B")
    └── moduleC.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# package/subpack/moduleC.py
# __name__ == 'package.subpack.moduleC'
# use relative imports
from ..moduleA import printA
from .moduleB import printB

# use absolute imports
from package.moduleA import printA
from package.subpack.moduleB import printB

def printAB():
    printA()
    printB()

# -----------------------------
# main.py
from package.subpack.moduleC import printAB 

printAB()
# A
# B

circular imports

1
2
ImportError: cannot import name '<name>' from partially initialized
module '<module>'(most likely due to a circular import)...
1
2
3
4
# Folder Structure:
├── main.py
├── moduleA.py
└── moduleB.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# moduleA.py
from moduleB import funcB
def funcA():
    funcB()

# -----------------------------
# moduleB.py
from moduleA import funcA
def funcB():
    funcA()

# -----------------------------
# main.py
from moduleA import funcA

# ImportError: cannot import name 'funcA' from partially initialized
# module 'moduleA' (most likely due to a circular import)

# -----------------------------
# main.py
from moduleB import funcB
# ImportError: cannot import name 'funcB' from partially initialized
# module 'moduleB' (most likely due to a circular import)

If you want to get Recursion by importing two functions to call each other, you need use import ... instead of from ... import ....

Take the example in the following Recursion section for example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def funcA(count):
    if count >= 10:
        return "A"
    else:
        return funcB(count+1)

def funcB(count):
    if count >= 10:
        return "B"
    else:
        return funcA(count+1)

print(funcA(1)) # B
print(funcB(1)) # A

The code above is a plain demonstration of Recursion. We split these two functions into two different modules, then import and call them in the main.py file.

1
2
3
4
# Folder Structure:
├── main.py
├── moduleA.py
└── moduleB.py

Using the two functions in this way to form Recursion, you will get an ImportError, as shown below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# moduleA.py
from moduleB import funcB

def funcA(count):
    if count >= 10:
        return "A"
    else:
        return funcB(count+1)

# -----------------------------
# moduleB.py
from moduleA import funcA

def funcB(count):
    if count >= 10:
        return "B"
    else:
        return funcA(count+1)

# -----------------------------
# main.py
from moduleA import funcA

# ImportError: cannot import name 'funcA' from partially initialized
# module 'moduleA' (most likely due to a circular import)

Instead of using from ... import ..., using import ... will make this recursion feasible

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# moduleA.py
# from moduleB import funcB
import moduleB

def funcA(count):
    if count >= 10:
        return "A"
    else:
        return moduleB.funcB(count+1)

# -----------------------------
# moduleB.py
# from moduleA import funcA
import moduleA

def funcB(count):
    if count >= 10:
        return "B"
    else:
        return moduleA.funcA(count+1)

# -----------------------------
# main.py
from moduleA import funcA
from moduleB import funcB

print(funcA(1)) # B
print(funcB(1)) # A

Recursion

Python allows functions to call each other. By doing this, you will get a loop in your code, which is called Recursion. If you don’t define the conditions to end the loop, you will get a RecursionError, as what happened in the following code

1
2
3
4
5
6
7
8
9
def funcA():
    funcB()

def funcB():
    funcA()

funcA()
# RecursionError: maximum recursion depth exceeded while calling
# a Python object

To fix this problem, you need to define the condition of ending the recursion.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def funcA(count):
    if count >= 10:
        return "A"
    else:
        return funcB(count+1)

def funcB(count):
    if count >= 10:
        return "B"
    else:
        return funcA(count+1)

print(funcA(1)) # B
print(funcA(2)) # A
print(funcB(1)) # A
print(funcB(2)) # B

import

模块 | 廖雪峰的官方网站

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Folder Structure:
mian.py
package
├── __init__.py
├── moduleA.py : def printA(): print("A")
|
├── subpack1
|   ├── __init__.py
│   └── moduleB.py : def printB(): print("B")
|
└── subpack2
    ├── __init__.py
    ├── moduleC.py : def printC(): print("C")
    |
    ├── test.py
    |
    └── subsubpack
        ├── __init__.py
        └── moduleD.py : def printD(): print("D")

import moduleC

  • from … import …
1
2
3
# package/subpack2/test.py
from moduleC import printC
printC() # C
  • import
1
2
3
# package/subpack2/test.py
import moduleC
moduleC.printC() # C

import moduleD

  • from … import …
1
2
3
# package/subpack2/test.py
from subsubpack.moduleD import printD
printD() # D
  • import
1
2
3
# package/subpack2/test.py
import subsubpack.moduleD
subsubpack.moduleD.printD() # D
  • import … as …
1
2
3
# package/subpack2/test.py
import subsubpack.moduleD as moduleD
moduleD.printD() # D
  • relative import
1
2
3
# package/subpack2/test.py
from .subsubpack.moduleD import printD
# ImportError: attempted relative import with no known parent package
1
2
3
4
5
6
7
8
# package/subpack2/test.py

if __name__ == '__main__':
    from subsubpack.moduleD import printD
else:
    from .subsubpack.moduleD import printD

printD() # D

import moduleA

  • relative import
1
2
3
4
5
6
7
8
9
# Folder Structure:
main.py : from package.subpack2.test import printA
package
├── __init__.py
├── moduleA.py : def printA(): print("A")
|
└── subpack2
    ├── __init__.py
    └── test.py : from ..moduleA import printA
1
2
3
4
5
6
7
8
9
10
11
# main.py
# __name__ == '__main__'
from package.subpack2.test import printA
printA() # A

# ------------------------------------

# package/subpack2/test.py
# __name__ == 'package.subpack2.test'
from ..moduleA import printA
print(__name__)
  • sys.path.append
1
2
3
4
5
6
7
8
# Folder Structure:
package
├── __init__.py
├── moduleA.py : def printA(): print("A")
|
└── subpack2
    ├── __init__.py
    └── test.py <-- main
1
2
3
4
5
6
7
8
9
# package/subpack2/test.py
# __name__ == '__main__'
import sys
# either way is ok
sys.path.append(__file__+'\..\..')
# sys.path.append(sys.path[0]+'\..')

from moduleA import printA
printA() # A

import moduleB

  • sys.path.append
1
2
3
4
5
6
7
8
9
10
11
# Folder Structure:
package
├── __init__.py
|
├── subpack1
|   ├── __init__.py
│   └── moduleB.py : def printB(): print("B")
|
└── subpack2
    ├── __init__.py
    └── test.py <-- main
1
2
3
4
5
6
7
8
9
10
# package/subpack2/test.py
# __name__ == '__main__'
import sys
# either way is ok
sys.path.append(__file__+'\..\..')
# sys.path.append(os.path.join(sys.path[0], '..'))
# sys.path.append(sys.path[0]+'\..')

from subpack1.moduleB import printB
printB() # B

what does sys.path do?

sys.path | python docs

The initialization of the sys.path module search path

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# package/subpack2/test.py
import sys
print(type(sys.path), len(sys.path))
# <class 'list'> 10

# the first elmnt of sys.path is CWD
# Current Working Directory
print(sys.path[0])
# c:\Users\...\package\subpack2

# list.append(elmnt):
# Add elmnt to the end of a list
sys.path.append("..")
# Print the last elmnt of sys.path
print(sys.path[-1])
# ..

# list.insert(pos, elmnt):
# Add elmnt to the head of a list (pos=0)
sys.path.insert(0, "..")
# Print the fitst elmnt of sys.path
print(sys.path[0])
# ..

Common errors

ImportError

1
ImportError: attempted relative import with no known parent package

Answer from vaultah | Relative imports in Python 3 | stackoverflow

1
2
3
# package/subpack2/test.py
from .subsubpack.moduleD import printD
# ImportError: attempted relative import with no known parent package

PEP 328 – Imports: Multi-Line and Absolute/Relative: Relative imports use a module’s __name__ attribute to determine that module’s position in the package hierarchy. If the module’s name does not contain any package information (e.g. it is set to '__main__') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

You can’t excute relative imports directly, the reasons are explained above, the module’s name does not contain any package information when __name__ == '__main__'.

If you call the modules containing relative imports in the programs under their parent folder, package/__init__.py or package/moduleA.py, the relative imports will be resolved correctly.
Because the __name__ == 'subpack2.test' in this case.

1
2
3
4
5
6
7
8
9
10
11
# Folder Structure:
package
├── __init__.py : from subpack2.test import printD
|
└── subpack2
    ├── __init__.py
    ├── test.py : from .subsubpack.moduleD import printD
    |
    └── subsubpack
        ├── __init__.py
        └── moduleD.py : def printD(): print("D")
1
2
3
4
5
6
7
8
9
10
# package/__init__.py
# __name__ == '__main__'
from subpack2.test import printD
printD() # D

# ------------------------------------

# package/subpack2/test.py
# __name__ == 'subpack2.test'
from .subsubpack.moduleD import printD

ValueError

1
ValueError: attempted relative import beyond top-level package
1
2
3
4
5
6
7
8
# Folder Structure:
package
├── __init__.py : from subpack2.test import printA
├── moduleA.py : def printA(): print("A")
|
└── subpack2
    ├── __init__.py
    └── test.py : from ..moduleA import printA
1
2
3
4
5
6
7
8
9
10
11
12
13
# package/__init__.py
# __name__ == '__main__'
from subpack2.test import printA

# ------------------------------------

# package/subpack2/test.py
# __name__ == 'subpack2.test'
from ..moduleA import printA

#-------------------------------------
#     from ..moduleA import printA
# ValueError: attempted relative import beyond top-level package

Type Hints and Variable&Function Annotations

PEP 484 – Type Hints

PEP 526 – Syntax for Variable Annotations

PEP 3107 – Function Annotations

Backslash “\” in line joining

Explicit line joining

Two or more physical lines may be joined into logical lines using backslash characters (), as follows: when a physical line ends in a backslash that is not part of a string literal or comment, it is joined with the following forming a single logical line, deleting the backslash and the following end-of-line character. For example:

1
2
3
4
if 1900 < year < 2100 and 1 <= month <= 12 \
   and 1 <= day <= 31 and 0 <= hour < 24 \
   and 0 <= minute < 60 and 0 <= second < 60:   # Looks like a valid date
        return 1

A line ending in a backslash cannot carry a comment. A backslash does not continue a comment. A backslash does not continue a token except for string literals (i.e., tokens other than string literals cannot be split across physical lines using a backslash). A backslash is illegal elsewhere on a line outside a string literal.

call

super

setattr()

ops

1
2
args = 'C:\\Users\\xiaophai\\Desktop\\code\\Python\\Swin-Transformer-for-semantic-segmentation/mymmlab/tools\\..\\configs\\unet\\unet_s5-d16_deeplabv3_4xb4-40k_chase-db1-128x128.py'
osp.join('./work_dirs',osp.splitext(osp.basename(args))[0])

@classmethod and @staticmethod

Meaning of @classmethod and @staticmethod for beginner

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
class Student:
    name: str
    age: int
    def __init__(self, name:str, age:int) -> None:
        self.name = name
        self.age = age
    
    @classmethod
    def from_tuple_cls(cls, name_age: tuple) -> "Student":
        # "cls" means class itself, not an instance of the class like "self"
        name, age = name_age
        stu = cls(name, age) # <- Notice here
        return stu
    
    @staticmethod
    def from_tuple_stc(name_age: tuple) -> "Student":
        # there is no obligatory parameter like a "cls" or "self"
        name, age = name_age
        stu = Student(name, age) # <- Notice here
        return stu

stu1 = Student.from_tuple_cls(("zhang", 18))
stu2 = Student.from_tuple_stc(("Wen", 22))
print(stu1.name, stu1.age) # zhang 18
print(stu2.name, stu2.age) # Wen 22

map(function, iterable)

1
map(int, ['1', '2', '3'])

Union

1
2
3
4
5
6
7
8
from typing import Callable, Dict, List, Optional, Sequence, Union

def __init__(
    model: Union[nn.Module, Dict],
    load_from: Optional[str] = None,
    train_dataloader: Optional[Union[DataLoader, Dict]] = None,
    env_cfg: Dict = dict(dist_cfg=dict(backend='nccl')),
):

yield

with

*args, **kwargs

Error

1
2
3
4
5
6
7
(openmmlab)...>pip list
...
Could not fetch URL https://pypi.org/simple/pip/: There was a problem
confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org',
port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError
(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.
c:1131)'))) - skipping

multiprocessing.pool

Multiprocessing in Python: Pool | YouTube

multiprocessing.pool | python doc

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
import time
from multiprocessing import Pool

def sum_square(number):
    s = 0
    for i in range(number):
        s += i**2
    return s

def sum_square_with_mp(numbers):
    start_time = time.time()

    p = Pool()
    result = p.map(sum_square, numbers)
    p.close()
    p.join()

    elapsed_time = time.time() - start_time
    print(f"Processing {len(numbers)} numbers took {elapsed_time} time using multiprocessing.")

def sum_square_without_mp(numbers):
    start_time = time.time()

    result = []
    for i in numbers:
        result.append(sum_square(i))

    elapsed_time = time.time() - start_time
    print(f"Processing {len(numbers)} numbers took {elapsed_time} time using serial processing.")
    

if __name__ == "__main__":
    
    import os
    print(os.cpu_count()) # 16

    numbers = range(100)
    sum_square_with_mp(numbers=numbers)
    sum_square_without_mp(numbers=numbers)

    # numbers = range(100):
    # Processing 100 numbers took 0.3546745777130127 time using multiprocessing.
    # Processing 100 numbers took 0.0010056495666503906 time using serial processing.
    # numbers = range(1000):
    # Processing 1000 numbers took 0.33483433723449707 time using multiprocessing.
    # Processing 1000 numbers took 0.10793018341064453 time using serial processing.
    # numbers = range(10000):
    # Processing 10000 numbers took 2.137113571166992 time using multiprocessing.
    # Processing 10000 numbers took 11.323538064956665 time using serial processing.

tracemalloc

tracemalloc | python doc

1
2
3
4
5
6
7
1 Byte = 8 Bits
1 KiloByte(KB) = 1024 Bytes
1 MegaByte(MB) = 1024 KiloBytes
1 GigaByte(GB) = 1024 MegaBytes
1 TeraByte(TB) = 1024 GigaBytes
1 PetaByte(PB) = 1024 TeraBytes
1 ExaBytes(EB) = 1024 PetaBytes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
import time
import tracemalloc
from multiprocessing import Pool

def sum_square(number):
    s = 0
    for i in range(number):
        s += i**2
    return s

def sum_square_with_mp(numbers):
    start_time = time.time()

    p = Pool()
    result = p.map(sum_square, numbers)
    p.close()
    p.join()

    elapsed_time = time.time() - start_time
    print(f"Processing {len(numbers)} numbers took {elapsed_time} time using multiprocessing.")

def sum_square_without_mp(numbers):
    start_time = time.time()

    result = []
    for i in numbers:
        result.append(sum_square(i))

    elapsed_time = time.time() - start_time
    print(f"Processing {len(numbers)} numbers took {elapsed_time} seconds using serial processing.")
    

if __name__ == "__main__":
    
    # import os
    # print(os.cpu_count()) # 16

    numbers = range(1000)

    tracemalloc.start()
    sum_square_with_mp(numbers=numbers)
    size_with_mp, peak_with_mp = tracemalloc.get_traced_memory()

    tracemalloc.clear_traces()
    sum_square_without_mp(numbers=numbers)
    size_without_mp, peak_without_mp = tracemalloc.get_traced_memory()

    print(f"Memery usage of multiprocessing: \n\t\
current size: {size_with_mp/(1024**2):.4f} MB, \
peak size: {peak_with_mp/(1024**2):.4f} MB")
    print(f"Memery usage of serialprocessing: \n\t\
current size: {size_without_mp/(1024**2):.4f} MB, \
peak size: {peak_without_mp/(1024**2):.4f} MB")

# Processing 1000 numbers took 0.3659026622772217 time using multiprocessing.
# Processing 1000 numbers took 0.3618462085723877 seconds using serial processing.
# Memery usage of multiprocessing:
#         current size: 1.3442 MB, peak size: 1.4158 MB
# Memery usage of serialprocessing:
#         current size: 0.0009 MB, peak size: 0.0365 MB

python 指定类型

1
2
def function(x: "int|str", y: int|str) -> tuple:
    return x, y
1
from typing import List