Skip to content

QML-FFmpeg Project Overview and Technical Architecture

🔗 GitLab repository: Utilities/QML-FFmpeg

QML-FFmpeg is a high-performance, multi-threaded video player solution built with Qt Quick/QML, FFmpeg, and SDL2. The project focuses on a smooth migration from the traditional OpenGL/FBO offscreen rendering path to a modern Qt Quick Scene Graph / RHI (Rendering Hardware Interface) compatible rendering pipeline. This reduces the dependency on a specific graphics backend and provides cross-platform hardware-accelerated playback capabilities.

🌟 Core Features

  1. RHI-compatible Scene Graph rendering
  2. Replaces the outdated QQuickFramebufferObject path with Qt Quick Scene Graph nodes for video frame rendering, making the player compatible with the Qt 6 RHI rendering architecture.
  3. Decoupled multi-threaded pipeline
  4. Separates demuxing, video decoding, and audio playback into independent threads, keeping the UI responsive while preserving smooth playback.
  5. Uses SDL2 audio playback as the master clock for AV sync, providing stable audio-video synchronization.
  6. Three-stage smooth hardware acceleration fallback
  7. Stage 1 (zero-copy GPU rendering): zero-copy rendering based on Direct3D11 / VA-API, mapping GPU textures directly.
  8. Stage 2 (GPU-to-CPU copy rendering): uses av_hwframe_transfer_data() to copy NV12 and other hardware-decoded frames back to system memory, then submits them to the Scene Graph after high-performance color-space conversion. This path has resolved hardware frame pool deadlocks and avfilter filtering issues.
  9. Stage 3 (pure CPU software decoding): when hardware acceleration initialization or transfer fails repeatedly, the player reloads within 5 frames and smoothly falls back to CPU decoding, such as YUV420P, without interrupting playback.
  10. Dual integration architecture
  11. Supports both direct static integration of the player core in a standalone application and dynamic injection as an independent QML extension plugin.
  12. Ready-to-use deployment and validation
  13. Provides CMake presets and automation scripts. On Windows, required FFmpeg/SDL2 DLLs and test videos are copied after build; Linux (Ubuntu) deployment is also designed to remain portable.

📁 Module Architecture

QML-FFmpeg (project root)
 ├── QML-FFmpeg.pro         # Main subdirs project file, directly openable in Qt Creator
 ├── docs/                  # Hardware/software validation checklists and technical design notes
 ├── QMLVideoPlayer-OpenGL/  # Standalone app mode: statically compiled player core and QML HUD
 ├── VideoPlugin-OpenGL/    # QML plugin module: dynamically loaded C++ plugin with shader resources
 ├── Video-QML/             # Plugin test shell: pure QML test app importing VideoPlugin 1.0
 └── ThirdParty/            # Third-party dependencies: prebuilt FFmpeg & SDL2 binary SDKs via Git LFS

📊 Platform Validation Status

According to the upstream platform_hwa_checklist.md, the current validation status is:

  • Windows (NVIDIA physical machine): Stage 2 (GPU-to-CPU NV12 copy rendering) and Stage 3 software decoding have been fully validated, providing strong stability and fault tolerance. D3D11 zero-copy rendering is still pending final closure.
  • Linux (Ubuntu virtual machine): because there is no GPU passthrough, VA-API hardware decoding is unavailable. The application successfully falls back to Stage 3 pure software decoding, and playback plus the control UI work normally.
  • ARM / embedded platforms: planned work includes Rockchip MPP and Raspberry Pi V4L2 M2M hardware decoding, plus DRM / GLES zero-copy rendering.

REF

[1]. about/index.md

[2]. platform_hwa_checklist.md