How APT does it progress bar?

content/blog/fancy-progress.md

@@ -0,0 +1,147 @@
+---
+Title: How APT does its fancy progress bar?
+Date: 2021-10-13 10:12:00
+Summary: It uses good old VT100 instructions.
+---
+
+Today while running an `apt full-upgrade` I asked myself how `apt`
+does this nice progress bar stuck at the bottom line while still
+writing scrolling text.
+
+We needed no more with a coworker to try to reproduce it!
+
+![apt autoremove gif]({static}/images/apt-autoremove.gif)
+
+Fortunately, while being very bored in the tube a few years back, I
+wrote a [Headless VT100
+emulator](https://github.com/JulienPalard/vt100-emulator), so I
+remembered some things and had a few hints on how it could be done, so
+I started poking around with some Python code.
+
+I remembered a few instructions like
+[DECSTBM](https://vt100.net/docs/vt510-rm/DECSTBM.html) to set the
+margins, and the various commands to move the cursor up and down so I
+started with this.
+
+After some trials-and-errors bottom margin reservation and log
+printing worked but the log were displayed on the line near the bottom
+margin, not where the command started: notice you can run an `apt
+upgrade` on the top of your terminal, it displays the progress bar at
+the bottom, but the logs will start from the top, as if there were no
+progress bar.
+
+While trying to solve this with my coworker, we were discussing about
+my implementatin, and were reading a random function:
+
+```c
+static void DECSC(struct lw_terminal *term_emul)
+{
+    /*TODO: Save graphic rendition and charset.*/
+    struct lw_terminal_vt100 *vt100;
+
+    vt100 = (struct lw_terminal_vt100 *)term_emul->user_data;
+    vt100->saved_x = vt100->x;
+    vt100->saved_y = vt100->y;
+}
+```
+
+and soon we realized it was the missing piece! Saving the cursor position to restore it later!!
+
+It soon started to look like an ugly undocumented, but almost working, ... thing:
+
+```python
+print(f"\0337\033[0;{lines-1}r\0338")
+try:
+    for i in range(250):
+        time.sleep(0.2)
+        print("Hello world", i)
+        print(
+            f"\0337\033[{lines};0f", datetime.now().isoformat(), "\0338", sep="", end=""
+        )
+except KeyboardInterrupt:
+    pass
+finally:
+    print(f"\0337\033[0;{lines}r\033[{lines};0f\033[0K\0338")
+```
+
+
+But wow, `f"\0337\033[0;{lines}r\033[{lines};0f\033[0K\0338"` should
+really be made more readable, it start to hurt my eyes. I had to go
+and let it as is for a night.
+
+Today I'm back at it again, and tried to add some comments, and delay
+to actually see how it behave step by step:
+
+```python
+import time
+import os
+from datetime import datetime
+
+columns, lines = os.get_terminal_size()
+
+
+def write(s):
+    print(s, end="")
+    time.sleep(1)
+
+
+write("\n")  # Ensure the last line is available.
+write("\0337")  # Save cursor position
+write(f"\033[0;{lines-1}r")  # Reserve the bottom line
+write("\0338")  # Restore the cursor position
+write("\033[1A")  # Move up one line
+
+try:
+    for i in range(250):
+        time.sleep(0.2)
+        write(f"Hello {i}")
+        write("\0337")  # Save cursor position
+        write(f"\033[{lines};0f")  # Move cursor to the bottom margin
+        write(datetime.now().isoformat())  # Write the date
+        write("\0338")  # Restore cursor position
+        write("\n")
+except KeyboardInterrupt:
+    pass
+finally:
+    write("\0337")  # Save cursor position
+    write(f"\033[0;{lines}r")  # Drop margin reservation
+    write(f"\033[{lines};0f")  # Move the cursor to the bottom line
+    write("\033[0K")  # Clean that line
+    write("\0338")  # Restore cursor position
+```
+
+Don't forget the `-u` (unbuffered) Python flag if you want to see it step by step:
+
+![progress bar started at the top]({static}/images/hello-1-2-3.gif)
+
+Started from the bottom so we see it scroll:
+
+![progress bar started at the bottom]({static}/images/ctrl-c.gif)
+
+Notice how on interruption (or normal exit) it cleans the progress bar
+before exiting, restoring the cursor at the right place.
+
+But hey, we could have read the apt code!
+
+Yes, it was less challenging, but now that it works, we have to take a look at it!
+
+So I `apt-get source apt` and found, in `install-progress.cc`:
+
+```cpp
+// scroll down a bit to avoid visual glitch when the screen
+// area shrinks by one row
+std::cout << "\n";
+
+// save cursor
+std::cout << "\0337";
+
+// set scroll region (this will place the cursor in the top left)
+std::cout << "\033[0;" << std::to_string(nr_rows - 1) << "r";
+
+// restore cursor but ensure its inside the scrolling area
+std::cout << "\0338";
+static const char *move_cursor_up = "\033[1A";
+std::cout << move_cursor_up;
+```
+
+Already looks familiar to you?