Significant rewrite after model modification: introducing `*Sources`
objects that encapsulate metadata and fetch information (urls,
protocols). The API (#20) is organized as pipe elements with sources
being what flows through the pipe.
1. fetch program sources
2. fetch rendition sources
3. fetch variant sources
4. fetch targets
5. process (download+mux) targets
Some user selection filter or modifiers could then be applied at any
step of the pipe. Our __main__.py is an implementation of that scheme.
Implied modifications include:
- Later failure on unsupported protocols, used to be in `api`, now in
`hls`. This offers the possibility to filter and/or support them
later.
- Give up honoring the http ranges for mp4 download, stream-download
them by fixed chunk instead.
- Cleaning up of the `hls` module moving the main download function to
__init__ and specific (mp4/vtt) download functions to a new
`download` module.
On the side modifications include:
- The progress handler showing downloading rates.
- The naming utilities providing rendition and variant code insertion.
- Download parts to working directories and skip unnecessary
re-downloads on failure.
This was a big change for a single commit... too big of a change maybe.
Change/add/rename model's data structures in order to provide a more
useful API #20, introducing new structures:
- `Sources`: summarizing program, renditions and variants found
at a given ArteTV page URL
- `Target`: summarizing all required data for a download
And new functions:
- `fetch_sources()` to build the `Sources` from a URL
- `iter_[renditions|variants]()` describe the available options for the
`Sources`
- `select_[renditions|variants]()` to narrow down the desired options
for the `Sources`
- `compile_sources` to compute such a `Target` from `Sources`
- `download_target` to download such a `Target`
Finally, this should make the playlist handling #7 easier (I know, I've
said that before)
- Rename variables and function to reflect model names.
- Convert infrastructure data (JSON, M3U8) to model types.
- Change algorithms to produce/consume `Source` model, in particular
using generator functions to build a list of `Source`s rather than the
opaque `rendition => variant => urls` mapping (this will make #7 very
straight forward).
- Download all master playlists after API call before selecting
rendition/variants.
Motivation for the last point:
We use to offer rendition choosing right after the API call, before we
download the appropriate master playlist to figure out the available
variants.
The problem with that is that ArteTV's codes for the renditions (given
by the API) do not necessarily include complete languages information
(if it is not French or German), for instance a original audio track in
Portuguese would show as `VOEU-` (as in "EUropean"). The actual mention
of the Portuguese would only show up in the master playlist.
So, the new implementation actually downloads all master playlists
straight after the API call. This is a bit wasteful, but I figured it
was necessary to provide quality interaction with the user.
Bonus? Now when we first prompt the user for rendition choice, we
actually already know the available variants available, maybe we make
use of that fact in the future...
Implemented modules:
- api: deals with ArteTV JSON API
- hls: deals with HLS protocol
- muxing: deals with the stream multiplexing
- naming: deals with output file naming
- www: deals with ArteTV web interface
