Rust roadmap¶

Where a worker still spends its time in Python, which way the remaining hot path moves to Rust, and in which order. The rule this follows: heavy work runs natively; native code we write ourselves is Rust (ready-made components may be C, C++ or Rust); Python is glue: business rules, wiring, orchestration and scripts, nothing on the hot path.

Two things do not change on the way: the REST /v1 responses and the GraphQL schema (both pinned by tests/snapshots/, REST byte for byte), and the deployment (Docker, compose, deploy/remote-deploy.sh, 1 GB VMs).

Audit¶

Measured with scripts/audit.py on Apple Silicon (8 cores, Python 3.13), in process, with t.me replaced by the recorded pages and the in-process cache (the L1 of the two-level cache) as the cache; memory on Linux (arm64, Python 3.14). The baseline is main after the two-level cache (PR #201). "Allocated" is the peak of Python allocations while serving one request (tracemalloc). Treat absolute numbers as relative.

uv run python scripts/audit.py --stages        # per endpoint and per stage
RAYON_NUM_THREADS=1 uv run python scripts/audit.py  # CPU cost on one core
uv run python scripts/audit.py --memory 200    # RSS of a feed loop (Linux)

Cost per request¶

The 100-channel feeds use aliases of the six recorded channels, each with its own username: 2 000 posts with many duplicate stories, a heavy case for the duplicate search.

Cold page pipeline, per recorded channel page (ms, single thread):

Parsing releases the GIL; everything after it (from_json to Python objects, pydantic validation and serialization, the cache encoding) holds it: 0.5–2.6 ms per page. A /v1/body miss also validates and renders the page a second time on the event loop (body_json → body()), although the parsing thread already rendered it.

Feed build, 100 cached pages, 2 000 posts (2.0 code, ms):

Event-loop stall. Cached /v1/body/durov from two clients at 50 req/s each, latency counted from when each request was due, next to one client building 100-channel feeds in a loop:

(Alone, that request takes 0.11 ms.)

By component¶

Memory per worker¶

Linux, arm64, Python 3.14 (scripts/memory_profile.py, scripts/audit.py --memory):

Python's own live objects stay at ~13 MiB through the mixed workload (tracemalloc); the rest is the interpreter and its imports, the in-process cache, native buffers and what malloc keeps between bursts.

A worker's baseline is the interpreter and the imported application (FastAPI, pydantic, strawberry, redis-py, the native extension), plus up to MEMORY_CACHE_MAX_MB of L1. WEB_CONCURRENCY workers duplicate all of it, the L1 included.

What it shows¶

• Nearly all CPU-bound work outside the parser and the ranker was in feed assembly: 41 ms of Python per 100-channel feed with the GIL held, ~20 MiB of short-lived Python objects, and a stall of every other request on the worker for that long. That is the biggest single item.
• Next is the page pipeline after the parser (0.5–2.6 ms of GIL-held pydantic per page, twice for a /v1/body miss). It matters for refresher ticks and cold traffic, not for cache hits.
• Cache hits are already cheap (0.07–0.33 ms). What remains there is the per-request floor of granian's ASGI bridge and FastAPI, and pydantic for /v1/post and previews.
• GraphQL costs ~0.85 ms of Python per query regardless of the data.
• The I/O-bound parts (HTTP client, Redis) are already native or C; what Python adds there is per-command overhead, not throughput.
• The ranker itself (Rust) is now the largest CPU item of a feed: ~80 ms per 2 000 posts on one core. Optimising it is independent of where the glue runs.

Options¶

(a) Keep the granian/FastAPI shell, move whole pipelines native¶

Each endpoint's work becomes one native call: fetch, parse, validate and render bytes, with the GIL released; native cache, single-flight and rate limiter. Python keeps routing, settings, feature flags, the error mapping, the feed's freshness rules and the refresher's scheduling.

• For: incremental, every step ships alone and is checked against the byte snapshots; OpenAPI (from the pydantic models), GraphQL and the CLI stay as they are; business rules stay in Python; Docker and the deploy scripts do not change. It is the path the parser, the ranker and the L1 cache already took.
• Against: the per-request floor stays (granian's ASGI bridge, FastAPI, ~50 µs per request, ~20 k req/s per worker), and so does a full interpreter per worker. GraphQL stays in Python.

(b) A Rust HTTP server (axum/hyper on tokio) serving /v1¶

The server is Rust; Python is embedded (PyO3) or called only for business rules.

• For: the lowest floor (tens of µs and several times the throughput per core), one multi-threaded process instead of one process per worker, so one L1 cache and one HTTP pool per server instead of one per worker: the biggest memory saving.
• Against: a big-bang switch of the server, router, middleware, error mapping and OpenAPI generation; an embedded interpreter brings the GIL back for every rule it runs, so rules must become data or Rust to get the benefit; GraphQL would need a Rust port (async-graphql) with an identical schema and errors, or stay a second, Python service. The rules stop being "easy to change in Python".

(c) A Rust core service and a thin Python control plane¶

A Rust binary serves /v1 (and later GraphQL) and runs the refresher; Python keeps configuration, the CLI, fixtures and scripts, and possibly GraphQL as a separate process calling the core.

• For: the same runtime gains as (b) without embedding; a clean boundary.
• Against: two processes per VM to deploy, configure and monitor; the control plane needs an interface (config file, admin API); the same big-bang server switch as (b), and everything that today is Python glue (feature flags, freshness rules) has to be re-expressed in Rust or in config.

Recommendation: (a), staged¶

Option (a) gets the CPU and GIL wins where they are measured (feed assembly, the page pipeline, GraphQL resolution) without a risky switch. Each stage moves a whole pipeline into telegram-me-native, ships on its own, is verified by the byte snapshots and has its own measured win. When the pipelines are native, the process is already a Rust core behind a Python shell: what is left in Python is the ~50 µs ASGI/FastAPI floor and one interpreter per worker. Whether that is worth replacing with an axum front (moving towards (c)) is decided then, on numbers (stage 6), not now.

Stages¶

Each stage: snapshots (make snapshots, raw digests) unchanged, its own before/after numbers, its own PR.

1. Native feed assembly — done in this release. FeedService hands the cached page bytes to telegram_me_native.build_feed and gets the response bytes back (one call, GIL released): reading pages, ranking signals, ranking, rendering. Python keeps the stale-while-revalidate rules and popularity. See the results.
2. Native page pipeline. Parse, validate and render in one call: parse_channel_page returns the cache encoding and the rendered /v1/body (and /v1/more) bytes directly, with the schema checks pydantic does today (URL normalisation — pydantic-core uses the same url crate —, literal values, defaults, the field validators, dropping posts that do not validate). The pydantic models stay as the documented schema (OpenAPI, GraphQL types) and become the oracle of a differential test over the fixtures and generated posts. Removes 0.5–2.6 ms of GIL-held Python per page and the second render of a /v1/body miss; makes refresher ticks GIL-free.
3. Native cached reads. /v1/post, /v1/preview and /v1/previews rendered natively from cached bytes (extract one post, render previews), so no cache hit builds Python objects. Builds on stage 2's renderer.
4. Native Redis level and popularity. A Rust Redis client (redis-rs or fred, on the tokio runtime) behind the existing Cache protocol and the two-level cache, plus popularity and the leader lock; one round trip per feed for all its pages (MGET). Coordinated with the cross-worker request coalescing work, which also lives on Redis.
5. GraphQL resolution. Resolvers read native-rendered bytes and the heaviest selections (body.posts) are projected natively; strawberry keeps the schema, so the SDL snapshot does not change. Removes most of the ~0.85 ms per query.
6. The shell, decided on numbers. With 1–5 done, measure what the ASGI bridge, FastAPI and the middlewares cost and what one interpreter per worker costs in RAM. If it is worth it, serve /v1 from an axum front in the same image (Python keeping GraphQL, docs and the CLI) or run one worker with native threads instead of several processes. The rate limiter and proof of work are already native (ratelimit.rs, pow.rs, PR #202).

Not planned: the ranker's algorithm. Its speed is separate work: at ~80 ms per 2 000 posts on one core it is now the largest CPU item of a feed.

Stage 1 results¶

Same machine, same script, main (with the two-level cache) against this release. Responses are byte-identical (tests/snapshots/raw_digests.json).

Resident memory does not change measurably: on Linux a worker serving 200 100-channel feeds peaked at 220 MiB before and 218 MiB after, and the mixed 20 000-request workload at 220 and 223 MiB (run-to-run noise is about ±10 MiB). The Python objects a feed no longer builds were short-lived, and a worker's RSS is dominated by the interpreter with its imports (85 MiB) and the in-process cache. The memory win of this roadmap comes from stage 6 (fewer interpreters, one cache per server), not from moving individual pipelines.

What is left of a feed in Python is resolving the pages (the cache lookups, whose L1 reads still hold the GIL for tens of µs each) and the popularity update. The rest of the CPU is the ranker.