Performance¶

All numbers below were measured on the same machine (Apple Silicon laptop, Python 3.14) while preparing 2.0, with t.me replaced by recorded pages so that only the service itself is measured. Treat them as relative, not absolute.

1.x → 2.0, in-process¶

Median latency of one request, upstream served from fixtures:

Two-level cache¶

A repeated request used to cost one Redis round trip when REDIS_URL was set: production's Upstash is ~90 ms away over TLS, so almost all of a cached response's time was the GET. Each worker now answers from its in-process cache first (Rust, moka) and goes to Redis only on a miss (see Caching).

Median / p95 of 200 repeated requests for @durov (one worker, ASGI called directly, upstream served from fixtures, Apple M-series, Python 3.13). The 90 ms Redis is a local Valkey 9.1 behind a proxy that delays each direction by 45 ms; "cache" is the time spent in cache.get per request:

• The first read of a key on a worker still costs one Redis round trip (GET and PTTL in one pipeline), as before; so does a write. The mixed workload is mostly first reads (every request names a new channel alias), which is why it gains less.
• Without Redis the native cache replaces the Python LRU: a hit no longer decompresses in Python (/v1/body 0.15 → 0.10 ms), and a 95 KB page is stored in 83 µs instead of 461 µs (zlib on 3.13; Python 3.14's zstd is closer).
• With Redis each worker now also holds what it read or wrote in the last MEMORY_CACHE_MAX_TTL (10) seconds, bounded by MEMORY_CACHE_MAX_MB: +19–31 MiB of peak RSS in this run, where everything was read within 10 s.
• Releasing the GIL around reads and writes was measured and rejected: next to two CPU-bound Python threads (pydantic on the parsing threads) a 95 KB read took 6.3 ms (p50) with the GIL released, 38 µs with it held.

Native feed assembly¶

A feed used to decode every cached page into Python objects, build a ranker input per post and serialise the result with pydantic, on the event loop with the GIL held: ~41 ms of a 100-channel feed. It is now one call to telegram_me_native.build_feed with the GIL released (cached page bytes in, response bytes out). scripts/audit.py, in process, same machine, 100-channel feeds of 2 000 posts:

Details, per-endpoint costs and what moves next: Rust roadmap.

Request coalescing¶

scripts/coalescing_load.py starts 6 worker processes (think two servers of three granian workers) on one Valkey, each with 32 clients calling its app directly. Every client picks requests at random from the same 37: each recorded channel through /v1/body (latest and around a post), /v1/post, /v1/more both ways, three previews, a 6-channel feed and a GraphQL query with a body and the feed. t.me is replaced by recorded pages answered after 150 ms; CACHE_MODE=normal (10 s), so the 27 distinct upstream requests expire twice in a 25 s run. Two runs each, Apple M-series with 8 cores:

• The lease is what removes the duplicates across processes: 75 upstream requests is the minimum for this run (27 distinct, renewed at most twice), and it holds with Redis 90 ms away. Without it every process fetches each page itself, as before.
• Sharing the rendering is what cuts the CPU: identical feeds are built once (and, for GraphQL, decoded once) instead of once per request, /v1/post is cached rendered, and concurrent /v1/body misses render once. The freed CPU went into serving 4–5× as many requests in the same time, with shorter event-loop stalls.
• A Redis command that times out opens the worker's circuit breaker; that worker then fetches on its own (one fetch per key, as before the lease) until it closes. Seen in development when a worker's event loop stalled for over 2 s (before GraphQL feeds were decoded once), not in the runs above.

The cached hot path is unchanged (one worker, median of 2 500 hits, ASGI called directly):

Admission is a few native calls on a miss only; a hit adds one native call that compares the entry's age with its renewal point.

Page parsing: Python + lexbor → Rust¶

Median time to parse one recorded channel page (scripts/benchmark.py):

The bigger difference is under load: the Rust parser releases the GIL, so concurrent requests parse pages in parallel instead of queueing on one core (24 pages on 24 threads: 111 ms → 67 ms on 8 cores, before the optimisations above).

Event loop under a refresher tick¶

Event-loop lag of one worker with Redis (Valkey) and the embedded refresher, in-process workload of 8 clients × 3 000 requests twice, with a refresher tick in between that refreshes the ~200 channels the first pass made popular:

Each finished fetch used to parse its page (~3 ms native) and validate and render it (~0.5–2 ms pydantic) right on the event loop. When a tick's fetches finished together they ran back to back for about a second, longer than REDIS_TIMEOUT, so a Redis command in flight timed out and opened the circuit breaker for 30 s. Pages are now parsed on two dedicated threads: the default executor (up to cpu + 4 threads) was no faster and, competing with the event loop for the GIL, tripled its p99 lag (~21 ms instead of ~7 ms). The refresher refreshes at most 16 channels at a time, which also leaves upstream slots for user requests.

Outgoing HTTP client¶

2 000 HTTPS requests for a 148 KB page, 32 concurrent, against a local TLS server:

1.x additionally created a new client — and a new TLS handshake — for every request to t.me.

HTTP server¶

One worker, 64 connections, 10 s (wrk), same application:

Pre-rendering cached responses raised the cached /v1/body throughput from ~850 to ~4 350 requests per second on one worker.

Shutdown and memory¶

Memory¶

Measured on arm64 with the Docker image (docker stats, one worker unless noted), before and after the memory work that went into 2.0. "Light" is a few hundred live requests over six channels; "heavy" is 2 × 20 000 requests from 8 concurrent clients over 1 200 distinct channels, with t.me replaced by the recorded pages.

What changed and why:

• The memory cache counted characters, not bytes. Values were Python str; one emoji makes CPython store a whole string with four bytes per character, so a rendered page of 89 KiB of UTF-8 took 353 KiB, and the 64 MB limit let a worker hold ~260 MiB of cache. Values are now UTF-8 bytes, compressed with zstd level 1 (zlib on Python 3.13), and the limit counts the compressed value, the key and the bookkeeping (~230 bytes per entry, measured):

  Page	As str	UTF-8	In the cache now
  @durov, rendered	353 KiB	89 KiB	21 KiB
  @tginfo, rendered	700 KiB	233 KiB	43 KiB
  @stranaua, rendered	93 KiB	59 KiB	13 KiB

  Decompressing a page (25–120 µs) is faster than encoding the same str to UTF-8 on every hit was (27–306 µs), and Redis replies are read as bytes as well.

• glibc kept freed memory. Responses are built from large short-lived objects, and after a burst glibc's heap stayed at its peak: with Redis, a worker held 97 MiB of heap of which 15 MiB were in use. Workers now call malloc_trim(0) every MEMORY_TRIM_INTERVAL seconds (about 1 ms per call), which returned ~80 MiB in that case.

• The granian main process imported the application (FastAPI, GraphQL, Redis, the HTTP client, the native extension) only to supervise workers. It now imports it in the worker factory.
• FastAPI ran the dependencies in a thread pool (plain def), which kept an extra thread — with its own malloc arena — in every worker. They are async def now.

Evaluated and not adopted (heavy traffic, measured while the parser was still Python; compare the rows with each other):

Parsing does not keep memory: pages are parsed natively and returned as one JSON string, and the memory regression tests show no growth over hundreds of uncached parses. WEB_CONCURRENCY, MEMORY_CACHE_MAX_MB and WORKERS_MAX_RSS_MB keep their defaults: with real accounting, 64 MB of cache now holds 4–16 times more pages than before within a fixed budget.

Guarding against regressions¶

tests/test_memory.py runs thousands of requests through the app and fails when memory keeps growing after a warm-up or the cache exceeds its limit; it catches both a leaked parsed page per request and leaked native memory. rust/telegram-me-native/tests/cache_memory.rs counts every allocation of the in-process cache and fails when its accounting no longer covers them. To see the numbers:

uv run python scripts/memory_profile.py --requests 20000 --channels 200

Reproducing¶

uv run python scripts/benchmark.py   # in-process, recorded pages, no network
uv run python scripts/audit.py --stages   # per endpoint (wall, CPU, allocations) and per stage
docker run -d --name tme-valkey -p 6390:6379 valkey/valkey:8
uv run python scripts/coalescing_load.py --redis redis://127.0.0.1:6390/0

For server throughput, start an instance and point a load generator at it:

make up                              # API + Valkey on :8000
wrk -t2 -c64 -d10s --latency http://127.0.0.1:8000/healthz