niklas/spielab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: update README with enhanced description and features; add image asset

Browse Source
This commit is contained in:
Niklas Haiden
2026-09-11 13:18:17 +02:00
parent e66e15fe21
commit d77ce2220a
2 changed files with 35 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+35 -71
View File
@@ -1,91 +1,55 @@
# Spielab # Spielab
A Rust + GPUI AirPlay sender for a Linux Wayland desktop. Discover Apple TV, pair with its on-screen code, choose a monitor or window using the desktop's screen-sharing dialog, and stream it with system audio. A native Rust + GPUI app for sharing a Linux Wayland desktop with Apple TV, including system audio.
The media mode is H.264 with selectable **720p/30**, **1080p/30** (default), or **1080p/60**, with stereo **44.1 kHz ALAC** audio. **Interactive** uses a 50 ms playback lead by default; **Responsive** offers 100 ms and **Buffered** restores the previous 500 ms lead for connections that stutter. This is a requested playback buffer, not a measurement of total input-to-display latency. It targets modern Apple TVs with HomeKit pairing and PTP screen-stream timing. Legacy FairPlay-only receivers and NTP-only receivers are not supported yet. ![Spielab sharing options in dark mode](docs/images/spielab.png)
## Features
- Share a screen or window, or create an extended display on supported desktops.
- H.264 video at 720p/30, 1080p/30, or 1080p/60.
- Automatic VA-API hardware encoding with software fallback when hardware is unavailable.
- Selectable 50, 100, or 500 ms playback buffers. These are buffer targets, not total display latency.
- Saved Apple TV pairing and light/dark themes.
Targets modern Apple TVs with HomeKit pairing and PTP timing. HEVC, NVENC, and legacy FairPlay/NTP receivers are not supported yet.
## Requirements
- Linux with Wayland, Vulkan graphics drivers, PipeWire, and a working ScreenCast desktop portal.
- Rust, a C linker, pkg-config, FreeType, Fontconfig, libxkbcommon, and libxkbcommon-x11.
- GStreamer (`gst-launch-1.0`) with `pipewiresrc`, `queue`, `videoconvert`, `videoscale`, `videorate`, `y4menc`, and `fdsink`. `pipewiresrc` must support `keepalive-time`.
- FFmpeg with `libx264`, plus PipeWire's `pw-cat`. GPU encoding also requires `h264_vaapi`, a compatible VA-API driver, and access to `/dev/dri/renderD*`.
- Apple TV on the same network with AirPlay enabled.
## Run ## Run
```sh ```sh
./scripts/run.sh cargo run --locked
``` ```
Use **Connection** to select your Apple TV and enter its code if prompted. Once authorization succeeds, Spielab opens **Sharing**, where you choose a screen/window or extended display, adjust stream quality, and click **Start sharing**. Sharing controls remain unavailable before authorization. The prominent **Disconnect** / **Cancel** button in the top-right corner, or Escape, stops the session. The initial pairing is saved for subsequent connections. 1. In **Connection**, select your Apple TV and enter its code if prompted.
2. Once authorized, **Sharing** opens. Choose a source, quality, and playback buffer.
3. In the desktop picker, **single-click one source, then press Share**. Avoid double-clicking on KDE to avoid a known portal crash.
4. Click **Start sharing**. System audio is included automatically.
**Settings → Appearance → Dark mode** switches between light and dark colors immediately. The preference is saved in `preferences.json` alongside the application's state and restored on the next launch. Settings remain accessible while connected or sharing. Use **Disconnect / Cancel** at the top right, or Escape, to stop. **Settings → Appearance** controls dark mode. Disconnecting also removes any temporary extended display.
In the desktop picker, **single-click exactly one source, then press Share**. Avoid double-clicking: KDE portal 6.7.4 can abort when a source card accepts the dialog from its own click handler. Spielab requests checkbox selection to avoid that path; the workaround still needs interactive verification on this host. Pairing and preferences live in `$XDG_STATE_HOME/spielab` or `~/.local/state/spielab`. Override with `SPIELAB_STATE_DIR`. The optional `./scripts/run.sh` uses the checkout's `.cargo-home` cache by default and reuses `.state` when present.
Choose quality before starting. The presets cap video bitrate at 8, 20, and 30 Mbps respectively. The 1080p presets preserve more detail; 60 fps needs more CPU and network capacity. Capture is scaled with the original aspect ratio preserved. For native Linux GUI builds, `build.rs` supplies a local linker symlink if the xkbcommon-x11 runtime is installed without its development symlink. No system files are modified.
GPU H.264 encoding is selected automatically through [FFmpeg's VA-API encoder](https://ffmpeg.org/ffmpeg-codecs.html#VAAPI-encoders) when available. Before sharing, Spielab tests actual encoded frames at the selected resolution and frame rate on accessible DRM render devices. If none pass, it uses software H.264. The command-line mirror example reports the selected encoder. AMD Radeon 890M hardware encoding has been verified on this host; other VA-API drivers are selected only if their probe passes. NVENC and other hardware APIs are not implemented. ## Development
The GPU path uses VBR at a target of 75% of the selected bitrate cap, no B-frames, and an async depth of one. Software uses the existing CRF presets. Encoding runs on the GPU; desktop capture, scaling and transfer to GPU memory still involve the CPU. A device failure after the startup probe stops sharing with an error; it does not switch codecs or encoders in a running session.
For responsiveness, start with **Interactive** and **1080p/60** if the host can encode it in real time. Try **720p/30** if encoding cannot keep up. Latency mode updates video SETUP, video presentation timestamps, audio SETUP, and audio synchronization together. Capture starts after receiver setup, the raw capture queue retains only the newest waiting frame, and encoded packets flush immediately. A separate packet-length pipe identifies completed encoded frames immediately, avoiding the previous wait for the next frame. Control feedback runs concurrently with media delivery. Network transport, encoding, decoding, and the TV's own display processing still add delay. Actual end-to-end latency requires measurement on the receiver.
**Create extended display** requests a separate virtual monitor when the desktop portal advertises support. Select it and press Share, then start sharing. Move windows onto it using your desktop's usual display controls. Disconnecting closes the portal session and removes the temporary monitor. This feature depends on the compositor and portal; creating the monitor has not yet been verified after the KDE crash workaround.
Required on the Linux host:
- A Wayland session, Vulkan-capable GPU drivers, and a working ScreenCast desktop portal (KDE, GNOME, or an appropriate compositor backend).
- Rust and the GPUI native build dependencies: a C linker, pkg-config, FreeType, Fontconfig, libxkbcommon and libxkbcommon-x11.
- `gst-launch-1.0` with `pipewiresrc`, `queue`, `videoconvert`, `videoscale`, `videorate`, `y4menc`, and `fdsink`. The PipeWire plugin must support `keepalive-time`.
- `ffmpeg` with the `libx264` encoder, and PipeWire's `pw-cat` utility. Hardware encoding additionally needs `h264_vaapi`, a VA-API driver with H.264 encoding support, and access to `/dev/dri/renderD*`.
- Apple TV on the same reachable network with AirPlay enabled.
Plain `cargo run` and `cargo build --locked` work on this host. For native Linux GUI builds, `build.rs` supplies a build-local xkbcommon-x11 linker symlink when the compiler can find the runtime library but its development symlink is missing. No system files are changed. If neither library is installed, install the distribution’s xkbcommon-x11 development package. Cross-compilation requires the target development libraries.
The development profile optimizes Spielab's packet handling and ChaCha20-Poly1305 dependencies because the development runner is used for live sharing. On this host the isolated 64 KiB video-packet encryption diagnostic fell from approximately 7.8 ms to 0.057 ms per packet. This microbenchmark does not measure display latency.
## How it works
- GPUI renders directly through its Wayland backend; the X11 backend is disabled.
- A separate Tokio worker handles discovery, pairing, network requests, and cancellation.
- The XDG ScreenCast portal grants a PipeWire node and remote FD. Both the portal session and FD remain owned until sharing ends; cancelling closes the session.
- GStreamer reads the authorized PipeWire source. FFmpeg encodes H.264 with [immediate packet flushing](https://ffmpeg.org/ffmpeg-formats.html); Rust assembles access units, sends the decoder configuration, and encrypts each video packet with ChaCha20-Poly1305 using the complete header as authenticated data.
- `pw-cat` captures the default output's monitor, not the microphone. Rust encodes PCM into ALAC, encrypts RTP audio, answers retransmission requests from a bounded history, and sends timing announcements on the negotiated control port.
- Video and audio share the receiver's PTP timeline, anchored using Apple TV's response timestamps. Encrypted event-channel commands, feedback, and video heartbeats keep the session alive.
Pairing identities are saved with mode 0600 in a directory with mode 0700. The default is `$XDG_STATE_HOME/spielab` or `~/.local/state/spielab`; `SPIELAB_STATE_DIR` overrides it. The development runner reuses this checkout's `.state` directory if it exists. Identity files are excluded from version control.
## Validation
```sh ```sh
cargo test --no-default-features cargo test --locked --no-default-features
cargo test --no-default-features --test media_runtime -- --ignored cargo clippy --locked --all-targets -- -D warnings
cargo test --no-default-features --lib -- --ignored
cargo check --all-targets
``` ```
Tests cover authenticated video framing and nonce separation, signed 64-bit clock IDs, audio/video playback lead agreement, PIN validation, private identity replacement, ALAC decoding through an independent decoder, and real H.264 encode/parse/decode round trips for every quality preset, including decoded dimensions and frame rate. An additional timed test checks that tiny, static frames arrive continuously without waiting for the output buffer to fill. The ignored runtime tests require FFmpeg. Additional ignored tests exercise real encoding and decoding. Run them with `cargo test --locked --no-default-features -- --ignored`; they require FFmpeg, and the hardware test requires VA-API H.264 on `/dev/dri/renderD128`.
The ignored hardware test requires VA-API H.264 on `/dev/dri/renderD128`. It checks automatic GPU selection and independently decodes more than one GOP at every quality preset. The ignored library tests also verify that an unavailable render device falls back to a working software encoder. On hosts without VA-API, run the software integration tests individually instead of all ignored integration tests. CLI examples for discovery, pairing, and mirroring are in [`examples/`](examples/).
Hardware smoke tests: ## License and acknowledgments
```sh [GPL-2.0-only](LICENSE). Built using [airplay2-rs](https://github.com/lmcgartland/airplay2-rs), with protocol references from [Doubletake](https://github.com/omarroth/doubletake) and [UxPlay](https://github.com/FDH2/UxPlay).
cargo run --no-default-features --example discover
SPIELAB_STATE_DIR=.state cargo run --no-default-features --example pair -- 'Living Room'
SPIELAB_STATE_DIR=.state cargo run --no-default-features --example mirror -- 'Living Room' --test
SPIELAB_STATE_DIR=.state cargo run --no-default-features --example mirror -- 'Living Room'
SPIELAB_STATE_DIR=.state cargo run --no-default-features --example mirror -- 'Living Room' --extend --quality=1080p60 --latency=50
```
The mirror example stops after 30 seconds; set `SPIELAB_TEST_SECONDS` to change that. Its `--test` mode sends a moving test card with the same audio/session path. Without that flag it opens the screen picker.
During development, an AppleTV14,1 accepted saved-identity pair verification, encrypted control/video/audio setup, and a 20-second, 600-frame test stream. Live Wayland capture and output-monitor audio have also been connected through the GPUI app. Visible picture, audible sound, and A/V synchronization still need receiver-side confirmation; sending packets alone does not prove playback.
The responsive-mode smoke test also completed: Apple TV accepted the 100 ms setup and the sender transmitted 600 frames at 1080p/60 over ten seconds. This validates negotiation and transmission, not measured input latency or receiver-side A/V synchronization.
The same ten-second, 600-frame Apple TV test passed with Radeon 890M VA-API hardware H.264 encoding. All quality presets also passed independent decoding through FFmpeg, and the unavailable-device fallback test passed. End-to-end latency has not been measured.
The 50 ms mode completed a 15-second, 900-frame 1080p/60 GPU test. Packet-length framing also passes a regression test delivering one complete frame with no subsequent frame. The concurrent-feedback implementation kept measured encoded-frame queue wait below 1 ms after startup during this test, while control replies reached 17 ms. These results validate individual sender improvements, not receiver-side presentation timing.
After enabling compiler optimization for the packet path, a further 600-frame test passed. Steady-state one-second intervals reported 60 frames with maximum measured queue wait and encrypt/write duration both below 1 ms.
## Protocol references
[airplay2-rs](https://github.com/lmcgartland/airplay2-rs/tree/a7f019fe6246ebd9701201a8a5c31e9a15243956) supplies the pinned Rust discovery, cryptographic pairing, and RTSP libraries. The mirroring wire layout and stream negotiation were researched against [Doubletake](https://github.com/omarroth/doubletake) and [UxPlay](https://github.com/FDH2/UxPlay). Spielab's session, capture, packetization, and UI code are in this repository.
Sender diagnostics log completed frames per interval, maximum encoded-frame queue time, maximum encrypt/write time, and control-response time. These exclude capture, GPU encoding, network delivery after the socket write, and receiver display time; they are not end-to-end latency measurements.
docs/images/spielab.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 350 KiB