Import: Write The Docs Paris 19.

2019-write-the-docs-paris-python-translation.md

@@ -0,0 +1,207 @@
+# docs.python.org/fr
+
+<!-- .slide: data-background="static/background.jpg" -->
+
+<br/>
+
+<b>Julien Palard</b>
+
+<tt>WriteTheDocs Paris 2019</tt>
+
+
+## Julien Palard
+
+- Python core dev and documentation expert
+- Python teacher and coach at:
+  - Sup'Internet
+  - CRI-Paris
+  - Makina Corpus
+  - …
+
+
+## History
+
+![](static/french.python.png)
+
+
+## History
+
+- Python is 28 years old,
+- its doc is written in reStructuredText,
+- compiled in HTML, PDF, epub, txt using Sphinx,
+- more than one million words,
+- daily changes.
+
+Notes:
+28 years old as of 2019 (first release 1991).
+
+More than "daily" for 3.7 stable: 170 commits on Doc/ over 120 days.
+
+
+## History
+
+- 2000: Project frpython on sourceforge
+- 2001: Translating Python 2.0
+- 2007: Python Doc moves from Latex to Sphinx & rst
+- 2010: GSoC Project to add i18n to Sphinx
+- 2011: 2% translated on pottle.python.org
+- 2012: pottle dead, AFPy team migrates to github
+- 2015-12: Resurected the project, alone for one year
+- 2016-03: docs.python.org/fr/ on python-ideas.
+- 2017-03: PEP 545
+
+Notes:
+
+2000: Latex to Latex, scripté en Python 1.5.2 (populaire à l'époque)
+2007: Hello Sphinx, created for Python by Georg Brandl, now used by many like Linux Kernel
+2012: Few people contributed during a month, and left for two years.
+2015: Alone for one year
+
+https://lists.afpy.org/mailman/private/afpy-membres/2012-September/005747.html
+http://frpython.sourceforge.net/
+
+
+## 2000
+
+![](static/sourceforge.png)
+
+Notes: https://web.archive.org/web/20010302160925/http://sourceforge.net/projects/frpython
+
+
+## 2001
+
+![](static/2.0.png)
+
+Notes: http://quentel.pierre.free.fr/python-trad/intro.html
+
+
+## 2012
+
+![](static/pottle.png)
+
+
+## 2019
+
+![](static/docs.python.org.png)
+
+
+## Progression
+
+![](static/fr_translation_percent.png)
+
+Notes:
+- 2016 jan: very short untranslatable strings
+- 2016 june: autofill whatsnew.po
+- 2017: PEP 545
+
+# How do we work?
+
+Mandatory meet point is:
+
+ `github.com/python/python-docs-*`
+
+But every language can use their own tools as long as they push on the meet point.
+
+
+## How do we work?
+
+- Some are using Transifex (ja, zh, pt_BR, ko, ...)
+- Some are using git (fr, it, es)
+- One could use any other tool…
+
+Notes:
+The french fries team (fr,it,es) is using git.
+
+
+## How do we work, in France?
+
+github and pull requests
+
+![](static/prs.png)
+
+----
+
+## How do we work, in France?
+
+It allows us to review and give feedback
+
+![](static/support.png)
+
+----
+
+## But git is hard!
+
+Yes.
+
+Note: But it looks like it's the mandatory way to contribute to most
+open source projects. We want to make the translation a way to learn
+how to contribute to an open source project. It's like a git sandbox :)
+
+Also it allows offline work that a lot of us do (in the train).
+
+
+# Tools
+
+How do we cope with:
+
+- Around 500 ``.po`` files,
+- more than one million words,
+- 45k paragraphs,
+- french in reStructuredText in gettext imbrication?
+
+```
+#: ../Doc/library/stdtypes.rst:373
+msgid "the greatest :class:`~numbers.Integral` <= *x*"
+msgstr "le plus grand :class:`~numbers.Integral` <= *x*"
+```
+
+## Tools
+
+### pypi.org/p/pospell
+
+``pospell`` is a tool using hunspell to spell check inside a
+``.po`` file while ignoring reStructuredText syntax.
+
+
+## Tools
+
+### pypi.org/p/powrap
+
+``powrap`` is a tool using ``msgcat`` to rewrap all ``.po`` files to a
+fixed width of 79 columns. We're enforcing this wrapping via the CI to
+reduce the noise in git logs.
+
+
+## Tools
+
+### pypi.org/p/potodo
+
+``potodo`` helps us listing what still needs to be done in this mess
+of 500 files.  It also synchroniszes with github issues to tell if
+someone is already working on a file, avoiding conflicts.
+
+
+## Tools
+
+### pypi.org/p/pomerge
+
+``pomerge`` helps us propagating translations from a branch to another,
+from a repo to another, or simply from a file to another.
+
+
+## Tools
+
+### poautofill
+
+``poautofill`` uses automatic translation to translate a whole file.
+This is bad from so many aspects, but it just helps me to avoid
+spending time trying to remember some vocabulary when I'm offline.
+
+
+# Et après ?
+
+Venez nous aider :
+
+- Sur github.com/python/python-docs-fr
+- Aux ateliers mensuels (meetup AFPy Paris)
+- Au sprint (2 jours) à La PyCon Fr à Bordeaux à partir du 31 oct 2019.

2019-write-the-docs-paris-python-translation.py

@@ -0,0 +1,141 @@
+from matplotlib import pyplot as plt
+import datetime
+
+"""Using something like:
+for i in $(seq -w 01 12)
+do
+   git clean -dfqx >/dev/null
+   git checkout -- . >/dev/null
+   git checkout $(git log 3.9 --after "2020-$i-01" | grep ^commit | tail -n 1 | cut -d' ' -f2)
+   git show | head -n 3
+   python -c 'import sys; print("{:.1%}".format(int(sys.argv[1]) / int(sys.argv[2])))' \
+       $(ls -1 *.po */*.po | grep -v 'whatsnew/changelog\|whatsnew/2\|whatsnew/3.[0-8]' | xargs msgcat | msgattrib --translated | grep -c '^msgid') \
+       $(ls -1 *.po */*.po | grep -v 'whatsnew/changelog\|whatsnew/2\|whatsnew/3.[0-8]' | xargs msgcat | grep -c '^msgid')
+done
+"""
+
+
+plt.xkcd()
+
+# According to `make progress`
+
+progress = {
+    datetime.date(2012, 10, 1): 0,
+    datetime.date(2012, 12, 1): 4,
+    datetime.date(2013, 12, 1): 4,
+    datetime.date(2014, 11, 1): 4,
+    datetime.date(2015, 12, 1): 5,
+    datetime.date(2016, 1, 4): 6,  # f8933be
+    datetime.date(2016, 2, 1): 15,  # very short untranslatable strings
+    datetime.date(2016, 3, 4): 16,  # 18c7d86
+    datetime.date(2016, 4, 12): 17,  # 4616b26
+    datetime.date(2016, 5, 1): 18,  # dd31bac
+    datetime.date(2016, 6, 13): 20,  # d16d992
+    datetime.date(2016, 7, 3): 20,  # 9e52ea9
+    datetime.date(2016, 10, 17): 20,  # efc39ad...
+    # Changement de repo
+    datetime.date(2016, 11, 5): 19,  # e61fde28
+    datetime.date(2017, 1, 17): 19,  # 384a9e5c
+    datetime.date(2017, 3, 5): 19.3,  # 384a9e5c
+    datetime.date(2017, 4, 2): 19.5,
+    datetime.date(2017, 5, 10): 20,
+    datetime.date(2017, 6, 1): 21.3,
+    datetime.date(2017, 7, 6): 22.2,
+    datetime.date(2017, 8, 1): 22.2,
+    datetime.date(2017, 9, 5): 23.3,
+    datetime.date(2017, 10, 2): 24.5,
+    datetime.date(2017, 11, 5): 26.5,
+    datetime.date(2017, 12, 1): 27.1,
+    datetime.date(2018, 1, 4): 27.3,
+    datetime.date(2018, 1, 30): 27.8,
+    datetime.date(2018, 3, 1): 27.8,
+    datetime.date(2018, 5, 21): 29.9,
+    datetime.date(2018, 7, 2): 30,
+    datetime.date(2018, 8, 1): 33,
+    datetime.date(2018, 9, 15): 33.2,
+    datetime.date(2018, 11, 1): 37.6,
+    datetime.date(2018, 12, 7): 39.6,
+    datetime.date(2019, 1, 2): 40.1,
+    datetime.date(2019, 2, 3): 41,
+    datetime.date(2019, 3, 1): 43.2,
+    datetime.date(2019, 4, 2): 44.6,
+    datetime.date(2019, 5, 6): 45,
+    datetime.date(2019, 6, 3): 45.6,
+    datetime.date(2019, 7, 2): 47.1,
+    datetime.date(2019, 8, 13): 47.4,
+    datetime.date(2019, 9, 3): 49.4,
+    datetime.date(2019, 9, 30): 47.9,  # Coût du passage en 3.8
+    datetime.date(2019, 10, 2): 47.9,
+    datetime.date(2019, 11, 1): 48.5,
+    datetime.date(2019, 12, 1): 48.8,
+    datetime.date(2020, 1, 2): 49.6,
+    datetime.date(2020, 2, 1): 50.0,
+    datetime.date(2020, 3, 3): 50.9,
+    datetime.date(2020, 4, 1): 51.1,
+    datetime.date(2020, 5, 1): 51.4,
+    datetime.date(2020, 6, 5): 53.2,
+    datetime.date(2020, 7, 1): 53.4,
+    datetime.date(2020, 8, 4): 52.8,
+    datetime.date(2020, 9, 9): 53.4,
+    datetime.date(2020, 10, 1): 54.1,
+    datetime.date(2020, 11, 6): 54.1,
+    datetime.date(2020, 12, 2): 54.1,
+    datetime.date(2021, 2, 2): 54.0,
+    datetime.date(2021, 3, 1): 54.4,
+    datetime.date(2021, 4, 1): 54.8,
+    datetime.date(2021, 5, 4): 55.1,
+    datetime.date(2021, 6, 4): 56.2,
+    datetime.date(2021, 7, 14): 56.4,
+    datetime.date(2021, 8, 5): 56.4,
+    datetime.date(2021, 9, 6): 56.4,
+    datetime.date(2021, 11, 1): 58,
+    datetime.date(2022, 1, 1): 58.2,
+    datetime.date(2022, 4, 5): 58.1,
+}
+
+plt.rcParams["figure.figsize"] = (20, 10)
+
+f, ax = plt.subplots(1)
+ax.plot_date(list(progress.keys()), list(progress.values()), linestyle="-")
+ax.annotate(
+    "Bump to 3.10",
+    (datetime.date(2021, 9, 24), 56.4),
+    xytext=(datetime.date(2021, 9, 24), 40),
+    arrowprops={"arrowstyle": "->"},
+)
+ax.annotate(
+    "Bump to 3.9",
+    (datetime.date(2020, 8, 4), 52.8),
+    xytext=(datetime.date(2020, 6, 5), 35),
+    arrowprops={"arrowstyle": "->"},
+)
+ax.annotate(
+    "Bump to 3.8",
+    (datetime.date(2019, 9, 30), 47.9),
+    xytext=(datetime.date(2019, 9, 4), 30),
+    arrowprops={"arrowstyle": "->"},
+)
+ax.annotate(
+    "Meetups AFPy mensuels",
+    (datetime.date(2018, 9, 1), 29.4),
+    xytext=(-10, -40),
+    textcoords="offset points",
+    arrowprops={"arrowstyle": "->"},
+)
+ax.annotate(
+    "/python/python-docs-fr/",
+    (datetime.date(2016, 11, 5), 23.5),
+    xytext=(-100, 50),
+    textcoords="offset points",
+    arrowprops={"arrowstyle": "->"},
+)
+ax.annotate(
+    "Working on very short strings",
+    (datetime.date(2016, 1, 15), (6 + 16) / 2),
+    xytext=(-10, -50),
+    textcoords="offset points",
+    arrowprops={"arrowstyle": "->"},
+)
+plt.ylabel("%")
+ax.set_ylim(ymin=0)
+plt.savefig("fr_translation_percent.png")

2020-en-attendant-la-pycon-fr-perf.md

@@ -0,0 +1,420 @@
+# Performance
+
+<!-- .slide: data-background="static/background.jpg" -->
+
+- cProfile
+- pstats
+
+::: notes
+
+Se présenter.
+
+# Le code
+
+```python
+def main():
+    already_checked = []
+    while True:
+        c = "".join(choice(ascii_letters) for _ in range(10))
+        if c in already_checked: continue
+        already_checked.append(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+```
+
+
+## Premiers tests
+
+```bash
+$ time python perf.py AFPy 00
+Searching
+[...]
+Searching
+Found: sha512(5NX3dB0BrO + AFPy) = 00…
+
+real 0m0.048s
+user 0m0.040s
+sys 0m0.008s
+```
+
+## Premiers tests
+
+```bash
+$ time python perf.py AFPy 000
+Searching
+[...]
+Searching
+Found: sha512(UYb0z6nac1 + AFPy) = 000…
+
+real 0m2.797s
+user 0m2.773s
+sys 0m0.024s
+```
+
+## Premiers tests
+
+```bash
+$ time python perf.py AFPy 0000
+Searching
+[...]
+Searching
+Found: sha512(dX0oAzvOmm + AFPy) = 0000…
+
+real 0m16.381s
+user 0m16.375s
+sys 0m0.004s
+```
+
+C'est long mais ça passe …
+
+
+## Premiers tests
+
+```bash
+$ time python perf.py AFPy 00000
+Searching
+[...]
+Searching
+Searching
+Searching
+Searching
+```
+
+Bon, on a un sushi.
+
+## cProfile
+
+```bash
+$ python -m cProfile -o prof perf.py AFPy 0000
+```
+
+## pstats
+
+```bash
+$ python -m pstats prof
+Welcome to the profile statistics browser.
+prof% sort cumulative
+prof% stats 10
+```
+
+## pstats
+
+```txt
+ncalls  cumtime  percall filename:lineno(function)
+  12/1   17.007   17.007 {built-in method builtins.exec}
+     1   17.007   17.007 /tmp/perf.py:1(<module>)
+     1   16.996   16.996 /tmp/perf.py:20(main)
+ 36429    0.869    0.000 {method 'join' of 'str' objects}
+```
+
+## snakeviz
+
+```bash
+$ pip install snakeviz
+Collecting snakeviz
+  Using cached snakeviz-2.1.0-py2.py3-none-any.whl (282 kB)
+Collecting tornado>=2.0
+  Using cached tornado-6.1-cp39-cp39-manylinux2010_x86_64.whl (427 kB)
+Installing collected packages: tornado, snakeviz
+Successfully installed snakeviz-2.1.0 tornado-6.1
+```
+
+
+## snakeviz
+
+```bash
+$ snakeviz prof
+```
+
+## snakeviz
+![](static/snakeviz-v1.png)
+
+
+## vprof
+
+```
+$ pip install vprof
+Collecting vprof
+  Using cached vprof-0.38-py3-none-any.whl (319 kB)
+Collecting psutil>=3
+  Using cached psutil-5.7.3-cp39-cp39d-linux_x86_64.whl
+Installing collected packages: psutil, vprof
+Successfully installed psutil-5.7.3 vprof-0.38
+```
+
+## vprof
+
+```
+$ vprof -c h "perf.py AFPy 0000"
+```
+
+## vprof
+
+![](static/vprof.png)
+
+
+# Le code, v1
+
+```python [2,5,6]
+def main():
+    already_checked = []
+    while True:
+        c = "".join(choice(ascii_letters) for _ in range(10))
+        if c in already_checked: continue
+        already_checked.append(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+```
+
+# Le code, v2
+
+```python [2,5,6]
+def main():
+    already_checked = set()
+    while True:
+        c = "".join(choice(ascii_letters) for _ in range(10))
+        if c in already_checked: continue
+        already_checked.add(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+```
+
+## Les perfs
+
+```bash
+$ hyperfine 'python perf.py AFPy 00000'
+```
+- v1 : ∞
+- v2 (`set`) : 23 s ± 23 s
+
+::: notes
+
+Il existe aussi pyperf: https://github.com/psf/pyperf
+
+
+## cProfile + pstats
+
+```bash
+$ python -m cProfile -o prof perf.py AFPy 0000
+$ python -m pstats prof
+```
+
+## cProfile + pstats
+```
+  ncalls  cumtime  percall filename:lineno(function)
+    12/1    1.156    1.156 {built-in method builtins.exec}
+       1    1.156    1.156 perf.py:1(<module>)
+       1    1.143    1.143 perf.py:35(main)
+   34215    0.771    0.000 {method 'join' of 'str' objects}
+  371647    0.681    0.000 perf.py:39(<genexpr>)
+  337860    0.526    0.000 /python3.9/random.py(choice)
+  337860    0.283    0.000 /python3.9/random.py(randbelow)
+   33786    0.134    0.000 built-in method print
+  372745    0.037    0.000 method 'getrandbits' of Random'
+   33786    0.037    0.000 method 'hexdigest' of hashlib
+```
+
+## snakeviz
+
+```bash
+$ snakeviz prof
+```
+
+## snakeviz
+
+![](static/snakeviz-v2.png)
+
+
+## Le code, v2
+
+```python [4]
+def main():
+    already_checked = set()
+    while True:
+        c = "".join(choice(ascii_letters) for _ in range(10))
+        if c in already_checked: continue
+        already_checked.add(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+```
+
+
+# Le code, v3
+
+```python [4]
+def main():
+    already_checked = set()
+    while True:
+        c = "".join(choices(ascii_letters, k=10))
+        if c in already_checked: continue
+        already_checked.add(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+```
+
+## Les perfs
+
+```bash
+$ hyperfine 'python perf.py AFPy 00000'
+```
+- v1 : ∞
+- v2 (`set`) : 23 s ± 23 s
+- v3 (`choices`): 8.591 s ± 6.525 s
+
+
+## snakeviz
+
+![](static/snakeviz-v3.png)
+
+
+# Le code, v4
+
+```python [3]
+def main():
+    already_checked = set()
+    for c in product(ascii_letters, repeat=10):
+        c = "".join(c)
+        if c in already_checked: continue
+        already_checked.add(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+```
+
+## Les perfs
+
+```bash
+$ hyperfine 'python perf.py AFPy 00000'
+```
+- v1 : ∞
+- v2 (`set`) : 23 s ± 23 s
+- v3 (`choices`): 8.591 s ± 6.525 s
+- v4 (`deterministic`) : 3.900 s ±  0.121 s
+
+
+## snakeviz
+
+![](static/snakeviz-v4.png)
+
+
+# Le code, v5
+
+```python [12]
+def main():
+    already_checked = set()
+    for c in product(ascii_letters, repeat=10):
+        c = "".join(c)
+        if c in already_checked: continue
+        already_checked.add(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        # print("Searching")
+```
+
+## Les perfs
+
+```bash
+$ hyperfine 'python perf.py AFPy 00000'
+```
+- v1 : ∞
+- v2 (`set`) : 23 s ± 23 s
+- v3 (`choices`): 8.591 s ± 6.525 s
+- v4 (`deterministic`) : 3.900 s ±  0.121 s
+- v5 (`print`) : 3.120 s ±  0.062 s
+
+
+## Snakeviz
+
+![](static/snakeviz-v5.png)
+
+Il reste du `hexdigest`, du `encode`, et du `join`.
+
+## vprof
+
+![](static/vprof2.png)
+
+Ligne 26 et 28 !?
+
+# Le code, v6
+
+```python
+def main():
+    for c in product(ascii_letters, repeat=10):
+        c = "".join(c)
+        digest = sha512(
+            (c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+```
+
+
+## Snakeviz
+
+![](static/snakeviz-v6.png)
+
+Il reste du `hexdigest`, du `encode`, et du `join`.
+
+
+# Le code, v7
+
+```python
+def main():
+    string = args.string.encode("UTF-8")
+    pool = ascii_letters.encode("UTF-8")
+    for c in product(pool, repeat=10):
+        digest = sha512(bytes(c) + string).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({bytes(c)} + {args.string}) = "
+                  f"{digest}")
+            sys.exit(0)
+```
+
+
+## Les perfs
+
+```bash
+$ hyperfine 'python perf.py AFPy 00000'
+```
+- v1 : ∞
+- v2 (`set`) : 23 s ± 23 s
+- v3 (`choices`): 8.591 s ± 6.525 s
+- v4 (`deterministic`) : 3.900 s ±  0.121 s
+- v5 (`print`) : 3.120 s ±  0.062 s
+- v6 (`dead code`): 2.844 s ±  0.059 s
+- v7 (`bytes`) : 1.837 s ±  0.067 s
+
+
+## Encore plus d'expériences
+
+- pypy: 3.8s
+- python: 1.8s
+- cython (hashlib) 1.3s
+- cython (crypto) 0.8s
+- c: 0.3s

Makefile

@@ -11,7 +11,7 @@ output/%.html: %.md
 
 .PHONY: rsync
 rsync: static
-	rsync -vah --delete output/ mdk_fr@mdk.fr:/var/www/mdk.fr/talks/
+	rsync -vah --delete output/ mdk@mdk.fr:/var/www/mdk.fr/talks/
 
 .PHONY: clean
 clean:

perf-experiments/README.md

@@ -0,0 +1,63 @@
+# Expériences
+
+Toutes les expériences cherchent de la même manière et trouvent le
+même résultat (sans quoi les temps ne sont pas comparables: changer un
+peu l'alphabet peut permettre de trouver rapidement une bonne
+solution, ou de passer à côté d'une solution) :
+
+C'est une recherche d'un sha512 commençant par '00000' d'une chaîne
+commençant par un préfix libre de 10 caractères suivi de la chaîne `AFPy`.
+
+Le préfixe utilise tous les caractères ASCII de `a` à `Z`, dans l'ordre ASCII.
+
+Toutes les expériences doivent donc trouver le même résultat :
+
+```
+sha512("AAAAAAEfNeAFPy") = 000000c9ddc3e63e34aecc1724fa38d55636a678800250e1bf322c4da065f37f8d251fb68a55bc8ca6ecf1fc226a712a65ae8d8c5d3e11a4527779d74f8fc8b6
+```
+
+C'est reproductible en exécutant `run.sh` (avec gcc et cython d'installé).
+
+
+# Pure C
+```
+Time (mean ± σ):     298.9 ms ±   7.4 ms    [User: 298.5 ms, System: 0.4 ms]
+Range (min … max):   286.8 ms … 313.1 ms    10 runs
+```
+
+
+# Cython sans hashlib
+
+Ressemble beaucoup au C pur.
+
+```
+Time (mean ± σ):     819.9 ms ±  24.7 ms    [User: 817.5 ms, System: 2.4 ms]
+Range (min … max):   791.4 ms … 867.4 ms    10 runs
+```
+
+
+# Cython avec hashlib
+
+Utilise hashlib, et donc des objets Python et des strings Python, la
+conversion a un coût.
+
+```
+Time (mean ± σ):      1.312 s ±  0.043 s    [User: 1.309 s, System: 0.003 s]
+Range (min … max):    1.258 s …  1.382 s    10 runs
+```
+
+
+# Pure Python
+
+```
+Time (mean ± σ):      1.885 s ±  0.066 s    [User: 1.882 s, System: 0.003 s]
+Range (min … max):    1.808 s …  2.034 s    10 runs
+```
+
+
+# Pypy 7.3.3
+
+```
+Time (mean ± σ):      3.834 s ±  0.054 s    [User: 3.772 s, System: 0.045 s]
+Range (min … max):    3.717 s …  3.897 s    10 runs
+```

perf-experiments/myperf.c

@@ -0,0 +1,55 @@
+#include <stdlib.h>
+#include <string.h>
+#include <stdio.h>
+#include "openssl/sha.h"
+
+static inline void tohex(char *outputbuffer, unsigned char c)
+{
+    outputbuffer[1] = "0123456789abcdef"[c % 16];
+    outputbuffer[0] = "0123456789abcdef"[c / 16];
+}
+
+void sha512(char *string, char outputBuffer[129])
+{
+    unsigned char hash[SHA512_DIGEST_LENGTH];
+    SHA512_CTX sha512;
+    SHA512_Init(&sha512);
+    SHA512_Update(&sha512, string, strlen(string));
+    SHA512_Final(hash, &sha512);
+    int i = 0;
+    for (i = 0; i < SHA512_DIGEST_LENGTH; i++)
+        tohex(outputBuffer + 2 * i, hash[i]);
+    outputBuffer[128] = 0;
+}
+
+int main(int ac, char **av)
+{
+    unsigned char outputbuffer[129];
+    unsigned char inputbuffer[7];
+
+    int repeat = 10;
+    char *suffix = "AFPy";
+    int size = repeat + strlen(suffix);
+    char *candidate = malloc(size + 1);
+    memset(candidate, 'A', size);
+    memcpy(candidate + size - strlen(suffix), suffix, strlen(suffix));
+    candidate[size] = 0;
+
+    long long int i = 0;
+    while (1)
+    {
+        candidate[9] = i % 58 + 65;
+        candidate[8] = (i / 58) % 58 + 65;
+        candidate[7] = (i / (58 * 58)) % 58 + 65;
+        candidate[6] = (i / (58 * 58 * 58)) % 58 + 65;
+        candidate[5] = (i / (58 * 58 * 58 * 58)) % 58 + 65;
+        candidate[4] = (i / (58 * 58 * 58 * 58 * 58)) % 58 + 65;
+        i += 1;
+        sha512(candidate, outputbuffer);
+        if (outputbuffer[0] == '0' && outputbuffer[1] == '0' && outputbuffer[2] == '0' && outputbuffer[3] == '0' && outputbuffer[4] == '0') {
+            printf("%s\n", outputbuffer);
+            return 0;
+        }
+    }
+    return 1;
+}

perf-experiments/perf.py

@@ -0,0 +1,19 @@
+import sys
+from itertools import product
+from string import ascii_lowercase, ascii_uppercase
+import hashlib
+
+
+def main():
+    string = "AFPy".encode("UTF-8")
+    pool = (ascii_uppercase + "[\\]^_`" + ascii_lowercase).encode("UTF-8")
+    for c in product(pool, repeat=10):
+        candidate = bytes(c) + string
+        digest = hashlib.sha512(candidate).hexdigest()
+        if digest.startswith("00000"):
+            print(f"sha512({candidate}) = {digest}")
+            sys.exit(0)
+
+
+if __name__ == "__main__":
+    main()

perf-experiments/perf.pyx

@@ -0,0 +1,64 @@
+# cython: language_level = 3
+
+import sys
+from libc.stdlib cimport malloc
+from libc.string cimport memcpy, strlen, memset
+
+cdef extern from "openssl/sha.h":
+    void SHA512_Init(void *)
+    void SHA512_Update(void *, const void *, size_t)
+    int SHA512_Final(unsigned char *, void *)
+    cdef union u:
+        unsigned long long d[16]
+        unsigned char p[16*8]
+    ctypedef struct SHA512_CTX:
+        unsigned long long h[8]
+        unsigned long long Nl, Nh
+        u u
+        unsigned int num, md_len
+
+
+
+cdef sha512(const char *string, char output_buffer[129]):
+    cdef unsigned char hash[64]
+    cdef SHA512_CTX sha512
+    SHA512_Init(&sha512)
+    SHA512_Update(&sha512, string, strlen(string))
+    SHA512_Final(hash, &sha512)
+    cdef int i
+    cdef const char *hex = "0123456789abcdef";
+    for i in range(64):
+        output_buffer[i * 2] = hex[hash[i] // 16]
+        output_buffer[i * 2 + 1] = hex[hash[i] % 16]
+    output_buffer[128] = 0
+
+
+cpdef search():
+    cdef int repeat = 10
+    cdef char *suffix = b"AFPy"
+    cdef unsigned int size = repeat + strlen(suffix)
+    cdef char *candidate = <char *> malloc(size + 1)
+    cdef char *digest;
+    cdef char output_buffer[129];
+    memset(candidate, b'-', size)
+    memcpy(candidate + size - strlen(suffix), suffix, strlen(suffix))
+    candidate[size] = 0
+
+    cdef unsigned int i = 0;
+    while True:
+        candidate[9] = i % 58 + 65;
+        candidate[8] = <char>(i / 58 ** 1) % 58 + 65;
+        candidate[7] = <char>(i / 58 ** 2) % 58 + 65;
+        candidate[6] = <char>(i / 58 ** 3) % 58 + 65;
+        candidate[5] = <char>(i / 58 ** 4) % 58 + 65;
+        candidate[4] = <char>(i / 58 ** 5) % 58 + 65;
+        candidate[3] = <char>(i / 58 ** 6) % 58 + 65;
+        candidate[2] = <char>(i / 58 ** 7) % 58 + 65;
+        candidate[1] = <char>(i / 58 ** 8) % 58 + 65;
+        candidate[0] = <char>(i / 58 ** 9) % 58 + 65;
+        i += 1
+        sha512(candidate, output_buffer)
+        if output_buffer.startswith(b"00000"):
+            print(f"sha512({candidate!r}) = {output_buffer}")
+            sys.exit(0)
+    print("Not found")

perf-experiments/perf_hashlib.pyx

@@ -0,0 +1,32 @@
+# cython: language_level = 3
+
+from libc.stdlib cimport malloc
+from libc.string cimport memcpy, strlen, memset
+
+import sys
+import hashlib
+
+cpdef search():
+    cdef int repeat = 10
+    cdef char *suffix = b"AFPy"
+    cdef unsigned int size = repeat + strlen(suffix)
+    cdef char *candidate = <char *> malloc(size + 1)
+    cdef char *digest;
+    memset(candidate, b'A', size)
+    memcpy(candidate + size - strlen(suffix), suffix, strlen(suffix))
+    candidate[size] = 0
+
+    cdef unsigned int i = 0;
+    while True:
+        candidate[9] = i % 58 + 65;
+        candidate[8] = <char>(i / 58 ** 1) % 58 + 65;
+        candidate[7] = <char>(i / 58 ** 2) % 58 + 65;
+        candidate[6] = <char>(i / 58 ** 3) % 58 + 65;
+        candidate[5] = <char>(i / 58 ** 4) % 58 + 65;
+        candidate[4] = <char>(i / 58 ** 5) % 58 + 65;
+        i += 1
+        output_buffer = hashlib.sha512(candidate).hexdigest()
+        if output_buffer.startswith("00000"):
+            print(f"sha512({candidate!r}) = {output_buffer}")
+            sys.exit(0)
+    print("Not found")

perf-experiments/run.sh

@@ -0,0 +1,28 @@
+#!/bin/sh
+echo Pure C
+
+FLAG="--style basic"
+
+# Use --show-output to validate the results
+# FLAG="--show-output"
+
+
+cc -O3 myperf.c -o myperf -lcrypto
+hyperfine $FLAG ./myperf
+
+echo Cython sans hashlib
+
+python setup.py build_ext --inplace >/dev/null
+hyperfine $FLAG "python3 -c 'from perf import search; search()'"
+
+echo Cython avec hashlib
+
+hyperfine $FLAG "python3 -c 'from perf_hashlib import search; search()'"
+
+echo Pure Python
+
+hyperfine $FLAG 'python3 perf.py'
+
+echo Pypy 7.3.3
+
+hyperfine $FLAG 'pypy3 perf.py'

perf-experiments/setup.py

@@ -0,0 +1,11 @@
+# python setup.py build_ext --inplace
+
+from setuptools import setup, Extension
+from Cython.Build import cythonize
+
+ext_modules = [
+    Extension("perf", sources=["perf.pyx"], libraries=["crypto"]),
+    Extension("perf_hashlib", sources=["perf_hashlib.pyx"]),
+]
+
+setup(name="Perf", ext_modules=cythonize(ext_modules))

static/perf.py

@@ -0,0 +1,129 @@
+"""Search seed such that:
+sha512(seed + string).startswith(prefix)
+
+Run like:
+
+time for i in $(seq 2 7) ; do hyperfine --show-output "python perf.py --version $i AFPy 00000" > hyperfine-AFPy.$i; done
+"""
+
+import argparse
+import sys
+from itertools import product
+from random import choices, choice
+from types import SimpleNamespace
+from string import ascii_uppercase, ascii_lowercase
+from hashlib import sha512
+
+POOL = ascii_uppercase + "[\\]^_`" + ascii_lowercase
+
+
+def main_v1():
+    args = parse_args()
+    already_checked = []
+    while True:
+        c = "".join(choice(POOL) for _ in range(10))
+        if c in already_checked:
+            continue
+        already_checked.append(c)
+        digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+
+
+def main_v2():
+    args = parse_args()
+    already_checked = set()
+    while True:
+        c = "".join(choice(POOL) for _ in range(10))
+        if c in already_checked:
+            continue
+        already_checked.add(c)
+        digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+
+
+# Use choices
+def main_v3():
+    args = parse_args()
+    already_checked = set()
+    while True:
+        c = "".join(choices(POOL, k=10))
+        if c in already_checked:
+            continue
+        already_checked.add(c)
+        digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+
+
+# Use product
+def main_v4():
+    args = parse_args()
+    already_checked = set()
+    for c in product(POOL, repeat=10):
+        c = "".join(c)
+        if c in already_checked:
+            continue
+        already_checked.add(c)
+        digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+        print("Searching")
+
+
+# Drop print
+def main_v5():
+    args = parse_args()
+    already_checked = set()
+    for c in product(POOL, repeat=10):
+        c = "".join(c)
+        if c in already_checked:
+            continue
+        already_checked.add(c)
+        digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+
+
+# Drop useless lines
+def main_v6():
+    args = parse_args()
+    for c in product(POOL, repeat=10):
+        c = "".join(c)
+        digest = sha512((c + args.string).encode("UTF-8")).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({c} + {args.string}) = {digest}")
+            sys.exit(0)
+
+
+# Bytes only
+def main_v7():
+    args = parse_args()
+    string = args.string.encode("UTF-8")
+    pool = POOL.encode("UTF-8")
+    for c in product(pool, repeat=10):
+        digest = sha512(bytes(c) + string).hexdigest()
+        if digest.startswith(args.sha_prefix):
+            print(f"sha512({bytes(c)} + {args.string}) = {digest}")
+            sys.exit(0)
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("string")
+    parser.add_argument("sha_prefix")
+    parser.add_argument("--version", default="1")
+    return parser.parse_args()
+
+
+if __name__ == "__main__":
+    globals()["main_v" + parse_args().version]()