Kataglyphis-Inference-Engine

An inference engine with Flutter/Dart frontend and Rust/C++ backend, showcasing Gstreamer capabilities enhanced with AI. Read further if you are interested in cross platform AI inference.

Official homepage

Overview

Kataglyphis-Inference-Engine bundles a Flutter/Dart frontend, a Rust/C++ inference core, and a rich set of camera streaming pipelines powered by GStreamer. The repository acts as an end-to-end reference for building cross-platform inference products that target desktop, web, and embedded devices.

Highlights & Key Features – Kataglyphis-Inference-Engine

🌟 Highlights

• 🎨 GStreamer native GTK integration – Leveraging users to write beautiful Linux AI inference apps.
• 📹 GStreamer WebRTC livestreaming with ready-to-use pipelines for USB, Raspberry Pi, and Orange Pi cameras.
• 🌉 flutter_rust_bridge integration – Ensures a seamless API boundary between Dart UI and Rust logic.
• 🐳 Containerized development flow plus native instructions for Windows, Linux, web. For details in my build environment look into Kataglyphis-ContainerHub. On Windows the container engine is Stevedore and build containers run with --isolation process (full host CPU count) — see docs/source/platforms.md.
• 🐍 Python inference demos for rapid experimentation alongside the Rust core.

📊 Feature Status Matrix

Core Features

Infrastructure & Build

Testing & Quality Assurance

Frontend Platforms

Platform Summary

Legend:

• ✔️ Completed - Feature fully implemented and tested
• 🔶 In Progress - Active development underway
• ❌ Not Started - Planned but not yet begun
• N/A - Not applicable for this platform

Quick Start

1. Clone the repository with submodules:

NOTE: On Windows I use Git Bash instead of
Powershell or cmd

git clone --recurse-submodules --branch develop git@github.com:Kataglyphis/Kataglyphis-Inference-Engine.git
cd Kataglyphis-Inference-Engine

2. Initialize submodules if needed.
   If u used --recurse-submodules while cloning you are already good.
   Otherwise you can use this :smile:

   git submodule update --init --recursive
   

Refer to the detailed docs below for platform-specific requirements, camera streaming pipelines, and deployment workflows.

3. Build the app. Every lane runs containerized, and local runs invoke the same script CI invokes — the exact commands, presets and switches are in AGENTS.md § 4:

   Lane	What CI runs	Locally
   Windows (amd64)	scripts/windows/Build-Windows.ps1	the same script, in the same image
   Linux native (amd64 / arm64)	scripts/linux/ci/ci-container-run-native-linux.sh	Invoke-LinuxLane.ps1
   Android	scripts/linux/ci/ci-container-run-android.sh	Invoke-LinuxLane.ps1 -Lane android
   Web	scripts/linux/ci/ci-container-run-web-linux.sh	Invoke-LinuxLane.ps1 -Lane web

   The Linux entries go through ContainerHub's run-in-linux-container action in CI and Rancher Desktop's nerdctl locally; inside the container the two are identical, down to the argument list. Reproduce a CI failure locally before pushing — that is the whole point of the arrangement.

   Note that the Linux build_linux stage runs a full CodeQL analysis on x64, not just a build — see AGENTS.md before starting one.

   Building the Linux lane locally on a Windows host has host-side prerequisites — Rancher Desktop's engine, the drive the repo lives on being visible to containerd's own mount namespace, and QEMU binfmt registered for an arm64 run. The concrete commands are in AGENTS.md § 4, "The Linux lane, locally"; the reasoning behind them is ContainerHub's, in rancher-desktop-linux-containers.md. Both prerequisites are lost on a VM restart, and skipping either is silent: you get a bind mount that resolves and is empty, or an arm64 container running x86-64 binaries.

Browse the API docs locally

Generate the site into doc/api, then serve it. Use the pub-activated dartdoc, not the SDK-bundled dart doc: dartdoc 9.0.4 (bundled with several Flutter SDKs, including the Windows build image) crashes on any Flutter app with a _stripDocImports RangeError; ≥ 9.0.9 fixes it.

scripts/windows/Build-Windows.ps1 already does this for you as its "Generate API Docs" step (skip it with -SkipDocs); the commands below are for generating and serving the site by hand.

dart pub global activate dartdoc      # pulls >= 9.0.9
dart pub global run dartdoc --output doc/api
dart pub global activate dhttpd
export PATH="$PATH:$HOME/.pub-cache/bin"
dhttpd --path doc/api --host 127.0.0.1 --port 8080

On PowerShell the pub-cache bin goes on PATH differently:

$env:Path += ";$env:USERPROFILE\AppData\Local\Pub\Cache\bin"

Then open 127.0.0.1:8080.

Documentation

Build the full documentation website with dart pub global run dartdoc (see the note above — not the SDK-bundled dart doc). The generated site in doc/api now includes the guides from docs/source.

Tests

Testing infrastructure is under active development. Track progress on the roadmap or contribute test plans via pull requests.

Roadmap

Upcoming features and improvements will be documented in this repository.
Please have a look docs/source/roadmap.md for more deetails.

Contributing

Contributions are what make the open-source community amazing. Any contributions are greatly appreciated.

1. Fork the project.
2. Create your feature branch (git checkout -b feature/AmazingFeature).
3. Commit your changes (git commit -m 'Add some AmazingFeature').
4. Push to the branch (git push origin feature/AmazingFeature).
5. Open a Pull Request.

License

MIT (see here)

Acknowledgements

Thanks to the open-source community and all contributors!

Literature

Helpful tutorials, documentation, and resources:

Multimedia

• GStreamer

Rust

• GStreamer-rs tutorial
• gst-plugins-rs
• GStreamer WebRTC

Raspberry Pi

• GStreamer on Raspberry Pi
• libcamera
• libcamera on Raspberry Pi

CMake/C++

• clang-cl

Flutter/Dart

• Linux Native Textures
• flutter_rust_bridge
• Flutter on RISCV

Protocols

• WebRTC

Tooling

• tmux
• zellij
• psmux

Android

• Gstreamer+flutter+android

Contact

Jonas Heinle
Twitter: @Cataglyphis_
Email: cataglyphis@jonasheinle.de

Project Links:

• GitHub: Kataglyphis-Inference-Engine
• Homepage: Official Site