Add Python Performance talk from En Attendant La PyConFr 2020.

This commit is contained in:
Julien Palard 2023-02-07 22:19:54 +01:00
parent 4ca6026735
commit 484f2969db
Signed by: mdk
GPG Key ID: 0EFC1AC1006886F8
19 changed files with 821 additions and 0 deletions

View File

@ -0,0 +1,420 @@
# Performance
<!-- .slide: data-background="static/background.jpg" -->
- cProfile
- pstats
::: notes
Se présenter.
# Le code
```python
def main():
already_checked = []
while True:
c = "".join(choice(ascii_letters) for _ in range(10))
if c in already_checked: continue
already_checked.append(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
```
## Premiers tests
```bash
$ time python perf.py AFPy 00
Searching
[...]
Searching
Found: sha512(5NX3dB0BrO + AFPy) = 00…
real 0m0.048s
user 0m0.040s
sys 0m0.008s
```
## Premiers tests
```bash
$ time python perf.py AFPy 000
Searching
[...]
Searching
Found: sha512(UYb0z6nac1 + AFPy) = 000…
real 0m2.797s
user 0m2.773s
sys 0m0.024s
```
## Premiers tests
```bash
$ time python perf.py AFPy 0000
Searching
[...]
Searching
Found: sha512(dX0oAzvOmm + AFPy) = 0000…
real 0m16.381s
user 0m16.375s
sys 0m0.004s
```
C'est long mais ça passe …
## Premiers tests
```bash
$ time python perf.py AFPy 00000
Searching
[...]
Searching
Searching
Searching
Searching
```
Bon, on a un sushi.
## cProfile
```bash
$ python -m cProfile -o prof perf.py AFPy 0000
```
## pstats
```bash
$ python -m pstats prof
Welcome to the profile statistics browser.
prof% sort cumulative
prof% stats 10
```
## pstats
```txt
ncalls cumtime percall filename:lineno(function)
12/1 17.007 17.007 {built-in method builtins.exec}
1 17.007 17.007 /tmp/perf.py:1(<module>)
1 16.996 16.996 /tmp/perf.py:20(main)
36429 0.869 0.000 {method 'join' of 'str' objects}
```
## snakeviz
```bash
$ pip install snakeviz
Collecting snakeviz
Using cached snakeviz-2.1.0-py2.py3-none-any.whl (282 kB)
Collecting tornado>=2.0
Using cached tornado-6.1-cp39-cp39-manylinux2010_x86_64.whl (427 kB)
Installing collected packages: tornado, snakeviz
Successfully installed snakeviz-2.1.0 tornado-6.1
```
## snakeviz
```bash
$ snakeviz prof
```
## snakeviz
![](static/snakeviz-v1.png)
## vprof
```
$ pip install vprof
Collecting vprof
Using cached vprof-0.38-py3-none-any.whl (319 kB)
Collecting psutil>=3
Using cached psutil-5.7.3-cp39-cp39d-linux_x86_64.whl
Installing collected packages: psutil, vprof
Successfully installed psutil-5.7.3 vprof-0.38
```
## vprof
```
$ vprof -c h "perf.py AFPy 0000"
```
## vprof
![](static/vprof.png)
# Le code, v1
```python [2,5,6]
def main():
already_checked = []
while True:
c = "".join(choice(ascii_letters) for _ in range(10))
if c in already_checked: continue
already_checked.append(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
```
# Le code, v2
```python [2,5,6]
def main():
already_checked = set()
while True:
c = "".join(choice(ascii_letters) for _ in range(10))
if c in already_checked: continue
already_checked.add(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
```
## Les perfs
```bash
$ hyperfine 'python perf.py AFPy 00000'
```
- v1 : ∞
- v2 (`set`) : 23 s ± 23 s
::: notes
Il existe aussi pyperf: https://github.com/psf/pyperf
## cProfile + pstats
```bash
$ python -m cProfile -o prof perf.py AFPy 0000
$ python -m pstats prof
```
## cProfile + pstats
```
ncalls cumtime percall filename:lineno(function)
12/1 1.156 1.156 {built-in method builtins.exec}
1 1.156 1.156 perf.py:1(<module>)
1 1.143 1.143 perf.py:35(main)
34215 0.771 0.000 {method 'join' of 'str' objects}
371647 0.681 0.000 perf.py:39(<genexpr>)
337860 0.526 0.000 /python3.9/random.py(choice)
337860 0.283 0.000 /python3.9/random.py(randbelow)
33786 0.134 0.000 built-in method print
372745 0.037 0.000 method 'getrandbits' of Random'
33786 0.037 0.000 method 'hexdigest' of hashlib
```
## snakeviz
```bash
$ snakeviz prof
```
## snakeviz
![](static/snakeviz-v2.png)
## Le code, v2
```python [4]
def main():
already_checked = set()
while True:
c = "".join(choice(ascii_letters) for _ in range(10))
if c in already_checked: continue
already_checked.add(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
```
# Le code, v3
```python [4]
def main():
already_checked = set()
while True:
c = "".join(choices(ascii_letters, k=10))
if c in already_checked: continue
already_checked.add(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
```
## Les perfs
```bash
$ hyperfine 'python perf.py AFPy 00000'
```
- v1 : ∞
- v2 (`set`) : 23 s ± 23 s
- v3 (`choices`): 8.591 s ± 6.525 s
## snakeviz
![](static/snakeviz-v3.png)
# Le code, v4
```python [3]
def main():
already_checked = set()
for c in product(ascii_letters, repeat=10):
c = "".join(c)
if c in already_checked: continue
already_checked.add(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
```
## Les perfs
```bash
$ hyperfine 'python perf.py AFPy 00000'
```
- v1 : ∞
- v2 (`set`) : 23 s ± 23 s
- v3 (`choices`): 8.591 s ± 6.525 s
- v4 (`deterministic`) : 3.900 s ± 0.121 s
## snakeviz
![](static/snakeviz-v4.png)
# Le code, v5
```python [12]
def main():
already_checked = set()
for c in product(ascii_letters, repeat=10):
c = "".join(c)
if c in already_checked: continue
already_checked.add(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
# print("Searching")
```
## Les perfs
```bash
$ hyperfine 'python perf.py AFPy 00000'
```
- v1 : ∞
- v2 (`set`) : 23 s ± 23 s
- v3 (`choices`): 8.591 s ± 6.525 s
- v4 (`deterministic`) : 3.900 s ± 0.121 s
- v5 (`print`) : 3.120 s ± 0.062 s
## Snakeviz
![](static/snakeviz-v5.png)
Il reste du `hexdigest`, du `encode`, et du `join`.
## vprof
![](static/vprof2.png)
Ligne 26 et 28 !?
# Le code, v6
```python
def main():
for c in product(ascii_letters, repeat=10):
c = "".join(c)
digest = sha512(
(c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
```
## Snakeviz
![](static/snakeviz-v6.png)
Il reste du `hexdigest`, du `encode`, et du `join`.
# Le code, v7
```python
def main():
string = args.string.encode("UTF-8")
pool = ascii_letters.encode("UTF-8")
for c in product(pool, repeat=10):
digest = sha512(bytes(c) + string).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({bytes(c)} + {args.string}) = "
f"{digest}")
sys.exit(0)
```
## Les perfs
```bash
$ hyperfine 'python perf.py AFPy 00000'
```
- v1 : ∞
- v2 (`set`) : 23 s ± 23 s
- v3 (`choices`): 8.591 s ± 6.525 s
- v4 (`deterministic`) : 3.900 s ± 0.121 s
- v5 (`print`) : 3.120 s ± 0.062 s
- v6 (`dead code`): 2.844 s ± 0.059 s
- v7 (`bytes`) : 1.837 s ± 0.067 s
## Encore plus d'expériences
- pypy: 3.8s
- python: 1.8s
- cython (hashlib) 1.3s
- cython (crypto) 0.8s
- c: 0.3s

View File

@ -0,0 +1,63 @@
# Expériences
Toutes les expériences cherchent de la même manière et trouvent le
même résultat (sans quoi les temps ne sont pas comparables: changer un
peu l'alphabet peut permettre de trouver rapidement une bonne
solution, ou de passer à côté d'une solution) :
C'est une recherche d'un sha512 commençant par '00000' d'une chaîne
commençant par un préfix libre de 10 caractères suivi de la chaîne `AFPy`.
Le préfixe utilise tous les caractères ASCII de `a` à `Z`, dans l'ordre ASCII.
Toutes les expériences doivent donc trouver le même résultat :
```
sha512("AAAAAAEfNeAFPy") = 000000c9ddc3e63e34aecc1724fa38d55636a678800250e1bf322c4da065f37f8d251fb68a55bc8ca6ecf1fc226a712a65ae8d8c5d3e11a4527779d74f8fc8b6
```
C'est reproductible en exécutant `run.sh` (avec gcc et cython d'installé).
# Pure C
```
Time (mean ± σ): 298.9 ms ± 7.4 ms [User: 298.5 ms, System: 0.4 ms]
Range (min … max): 286.8 ms … 313.1 ms 10 runs
```
# Cython sans hashlib
Ressemble beaucoup au C pur.
```
Time (mean ± σ): 819.9 ms ± 24.7 ms [User: 817.5 ms, System: 2.4 ms]
Range (min … max): 791.4 ms … 867.4 ms 10 runs
```
# Cython avec hashlib
Utilise hashlib, et donc des objets Python et des strings Python, la
conversion a un coût.
```
Time (mean ± σ): 1.312 s ± 0.043 s [User: 1.309 s, System: 0.003 s]
Range (min … max): 1.258 s … 1.382 s 10 runs
```
# Pure Python
```
Time (mean ± σ): 1.885 s ± 0.066 s [User: 1.882 s, System: 0.003 s]
Range (min … max): 1.808 s … 2.034 s 10 runs
```
# Pypy 7.3.3
```
Time (mean ± σ): 3.834 s ± 0.054 s [User: 3.772 s, System: 0.045 s]
Range (min … max): 3.717 s … 3.897 s 10 runs
```

55
perf-experiments/myperf.c Normal file
View File

@ -0,0 +1,55 @@
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "openssl/sha.h"
static inline void tohex(char *outputbuffer, unsigned char c)
{
outputbuffer[1] = "0123456789abcdef"[c % 16];
outputbuffer[0] = "0123456789abcdef"[c / 16];
}
void sha512(char *string, char outputBuffer[129])
{
unsigned char hash[SHA512_DIGEST_LENGTH];
SHA512_CTX sha512;
SHA512_Init(&sha512);
SHA512_Update(&sha512, string, strlen(string));
SHA512_Final(hash, &sha512);
int i = 0;
for (i = 0; i < SHA512_DIGEST_LENGTH; i++)
tohex(outputBuffer + 2 * i, hash[i]);
outputBuffer[128] = 0;
}
int main(int ac, char **av)
{
unsigned char outputbuffer[129];
unsigned char inputbuffer[7];
int repeat = 10;
char *suffix = "AFPy";
int size = repeat + strlen(suffix);
char *candidate = malloc(size + 1);
memset(candidate, 'A', size);
memcpy(candidate + size - strlen(suffix), suffix, strlen(suffix));
candidate[size] = 0;
long long int i = 0;
while (1)
{
candidate[9] = i % 58 + 65;
candidate[8] = (i / 58) % 58 + 65;
candidate[7] = (i / (58 * 58)) % 58 + 65;
candidate[6] = (i / (58 * 58 * 58)) % 58 + 65;
candidate[5] = (i / (58 * 58 * 58 * 58)) % 58 + 65;
candidate[4] = (i / (58 * 58 * 58 * 58 * 58)) % 58 + 65;
i += 1;
sha512(candidate, outputbuffer);
if (outputbuffer[0] == '0' && outputbuffer[1] == '0' && outputbuffer[2] == '0' && outputbuffer[3] == '0' && outputbuffer[4] == '0') {
printf("%s\n", outputbuffer);
return 0;
}
}
return 1;
}

19
perf-experiments/perf.py Normal file
View File

@ -0,0 +1,19 @@
import sys
from itertools import product
from string import ascii_lowercase, ascii_uppercase
import hashlib
def main():
string = "AFPy".encode("UTF-8")
pool = (ascii_uppercase + "[\\]^_`" + ascii_lowercase).encode("UTF-8")
for c in product(pool, repeat=10):
candidate = bytes(c) + string
digest = hashlib.sha512(candidate).hexdigest()
if digest.startswith("00000"):
print(f"sha512({candidate}) = {digest}")
sys.exit(0)
if __name__ == "__main__":
main()

64
perf-experiments/perf.pyx Normal file
View File

@ -0,0 +1,64 @@
# cython: language_level = 3
import sys
from libc.stdlib cimport malloc
from libc.string cimport memcpy, strlen, memset
cdef extern from "openssl/sha.h":
void SHA512_Init(void *)
void SHA512_Update(void *, const void *, size_t)
int SHA512_Final(unsigned char *, void *)
cdef union u:
unsigned long long d[16]
unsigned char p[16*8]
ctypedef struct SHA512_CTX:
unsigned long long h[8]
unsigned long long Nl, Nh
u u
unsigned int num, md_len
cdef sha512(const char *string, char output_buffer[129]):
cdef unsigned char hash[64]
cdef SHA512_CTX sha512
SHA512_Init(&sha512)
SHA512_Update(&sha512, string, strlen(string))
SHA512_Final(hash, &sha512)
cdef int i
cdef const char *hex = "0123456789abcdef";
for i in range(64):
output_buffer[i * 2] = hex[hash[i] // 16]
output_buffer[i * 2 + 1] = hex[hash[i] % 16]
output_buffer[128] = 0
cpdef search():
cdef int repeat = 10
cdef char *suffix = b"AFPy"
cdef unsigned int size = repeat + strlen(suffix)
cdef char *candidate = <char *> malloc(size + 1)
cdef char *digest;
cdef char output_buffer[129];
memset(candidate, b'-', size)
memcpy(candidate + size - strlen(suffix), suffix, strlen(suffix))
candidate[size] = 0
cdef unsigned int i = 0;
while True:
candidate[9] = i % 58 + 65;
candidate[8] = <char>(i / 58 ** 1) % 58 + 65;
candidate[7] = <char>(i / 58 ** 2) % 58 + 65;
candidate[6] = <char>(i / 58 ** 3) % 58 + 65;
candidate[5] = <char>(i / 58 ** 4) % 58 + 65;
candidate[4] = <char>(i / 58 ** 5) % 58 + 65;
candidate[3] = <char>(i / 58 ** 6) % 58 + 65;
candidate[2] = <char>(i / 58 ** 7) % 58 + 65;
candidate[1] = <char>(i / 58 ** 8) % 58 + 65;
candidate[0] = <char>(i / 58 ** 9) % 58 + 65;
i += 1
sha512(candidate, output_buffer)
if output_buffer.startswith(b"00000"):
print(f"sha512({candidate!r}) = {output_buffer}")
sys.exit(0)
print("Not found")

View File

@ -0,0 +1,32 @@
# cython: language_level = 3
from libc.stdlib cimport malloc
from libc.string cimport memcpy, strlen, memset
import sys
import hashlib
cpdef search():
cdef int repeat = 10
cdef char *suffix = b"AFPy"
cdef unsigned int size = repeat + strlen(suffix)
cdef char *candidate = <char *> malloc(size + 1)
cdef char *digest;
memset(candidate, b'A', size)
memcpy(candidate + size - strlen(suffix), suffix, strlen(suffix))
candidate[size] = 0
cdef unsigned int i = 0;
while True:
candidate[9] = i % 58 + 65;
candidate[8] = <char>(i / 58 ** 1) % 58 + 65;
candidate[7] = <char>(i / 58 ** 2) % 58 + 65;
candidate[6] = <char>(i / 58 ** 3) % 58 + 65;
candidate[5] = <char>(i / 58 ** 4) % 58 + 65;
candidate[4] = <char>(i / 58 ** 5) % 58 + 65;
i += 1
output_buffer = hashlib.sha512(candidate).hexdigest()
if output_buffer.startswith("00000"):
print(f"sha512({candidate!r}) = {output_buffer}")
sys.exit(0)
print("Not found")

28
perf-experiments/run.sh Normal file
View File

@ -0,0 +1,28 @@
#!/bin/sh
echo Pure C
FLAG="--style basic"
# Use --show-output to validate the results
# FLAG="--show-output"
cc -O3 myperf.c -o myperf -lcrypto
hyperfine $FLAG ./myperf
echo Cython sans hashlib
python setup.py build_ext --inplace >/dev/null
hyperfine $FLAG "python3 -c 'from perf import search; search()'"
echo Cython avec hashlib
hyperfine $FLAG "python3 -c 'from perf_hashlib import search; search()'"
echo Pure Python
hyperfine $FLAG 'python3 perf.py'
echo Pypy 7.3.3
hyperfine $FLAG 'pypy3 perf.py'

11
perf-experiments/setup.py Normal file
View File

@ -0,0 +1,11 @@
# python setup.py build_ext --inplace
from setuptools import setup, Extension
from Cython.Build import cythonize
ext_modules = [
Extension("perf", sources=["perf.pyx"], libraries=["crypto"]),
Extension("perf_hashlib", sources=["perf_hashlib.pyx"]),
]
setup(name="Perf", ext_modules=cythonize(ext_modules))

129
static/perf.py Normal file
View File

@ -0,0 +1,129 @@
"""Search seed such that:
sha512(seed + string).startswith(prefix)
Run like:
time for i in $(seq 2 7) ; do hyperfine --show-output "python perf.py --version $i AFPy 00000" > hyperfine-AFPy.$i; done
"""
import argparse
import sys
from itertools import product
from random import choices, choice
from types import SimpleNamespace
from string import ascii_uppercase, ascii_lowercase
from hashlib import sha512
POOL = ascii_uppercase + "[\\]^_`" + ascii_lowercase
def main_v1():
args = parse_args()
already_checked = []
while True:
c = "".join(choice(POOL) for _ in range(10))
if c in already_checked:
continue
already_checked.append(c)
digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
def main_v2():
args = parse_args()
already_checked = set()
while True:
c = "".join(choice(POOL) for _ in range(10))
if c in already_checked:
continue
already_checked.add(c)
digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
# Use choices
def main_v3():
args = parse_args()
already_checked = set()
while True:
c = "".join(choices(POOL, k=10))
if c in already_checked:
continue
already_checked.add(c)
digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
# Use product
def main_v4():
args = parse_args()
already_checked = set()
for c in product(POOL, repeat=10):
c = "".join(c)
if c in already_checked:
continue
already_checked.add(c)
digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
print("Searching")
# Drop print
def main_v5():
args = parse_args()
already_checked = set()
for c in product(POOL, repeat=10):
c = "".join(c)
if c in already_checked:
continue
already_checked.add(c)
digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
# Drop useless lines
def main_v6():
args = parse_args()
for c in product(POOL, repeat=10):
c = "".join(c)
digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({c} + {args.string}) = {digest}")
sys.exit(0)
# Bytes only
def main_v7():
args = parse_args()
string = args.string.encode("UTF-8")
pool = POOL.encode("UTF-8")
for c in product(pool, repeat=10):
digest = sha512(bytes(c) + string).hexdigest()
if digest.startswith(args.sha_prefix):
print(f"sha512({bytes(c)} + {args.string}) = {digest}")
sys.exit(0)
def parse_args():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("string")
parser.add_argument("sha_prefix")
parser.add_argument("--version", default="1")
return parser.parse_args()
if __name__ == "__main__":
globals()["main_v" + parse_args().version]()

BIN
static/snakeviz-v1.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

BIN
static/snakeviz-v2.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

BIN
static/snakeviz-v3.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

BIN
static/snakeviz-v4.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

BIN
static/snakeviz-v5.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

BIN
static/snakeviz-v6.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

BIN
static/snakeviz-v7.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

BIN
static/snakeviz.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

BIN
static/vprof.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

BIN
static/vprof2.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB