Welcome to Firefly
English
Firefly Docsss
Last Updated: 2026-10-12 10:28:18

FFMedia General Usage

FFMedia provides a C++ API, Python bindings, and the ffmedia command-line tool. The recommended order is:

  1. C++: for production applications and media workflows that require full control.
  2. Python: for rapid development, functional validation, and script integration.
  3. CLI: for rapid validation, debugging, general-purpose tasks, and script automation.

Regardless of the interface, the basic pipeline lifecycle is:

Create modules -> set parameters -> initialize the input module -> connect downstream modules
-> initialize downstream modules -> start the input module -> run -> stop the pipeline

Initialize the producer before connecting it so that it can publish its output channels. connectProducer() matches the upstream and downstream modules using media type, codec, and image or audio parameters. Check the return value of every connection and initialization call before starting the pipeline.

C++ Integration

C++ is FFMedia's primary application interface. It provides direct access to module APIs, the parameter system, channel matching, and MediaBuffer. Production applications normally use C++ for device management, business loops, error handling, and custom module integration.

Project Configuration

Use the CMake target exported by the SDK:

find_package(FFMedia REQUIRED CONFIG COMPONENTS core)
target_link_libraries(my_app PRIVATE FFMedia::FFMedia)

Include the stable aggregate header:

#include <ffmedia/ffmedia.hpp>

The headers, CMake configuration, and shared library must come from the same SDK package. Do not mix files from different packages.

Create a Basic Pipeline

The following example reads a file, decodes it with MPP, and writes the decoded data to a file. In a real application, the output module can be replaced with ModuleDrmDisplay, ModuleRendererVideo, or a network output module.

#include <cstdio>
#include <memory>

#include <ffmedia/ffmedia.hpp>

using namespace FFMedia;

int main()
{
    auto source = std::make_shared<ModuleFileReader>("input.mp4", false);
    int ret = source->init();
    if (ret < 0)
        return ret;

    auto decoder = std::make_shared<ModuleMppDec>();
    ret = decoder->connectProducer(source);
    if (ret < 0)
        return ret;
    ret = decoder->init();
    if (ret < 0)
        return ret;

    auto output = std::make_shared<ModuleFileWriter>("output.nv12");
    ret = output->connectProducer(decoder);
    if (ret < 0)
        return ret;
    ret = output->init();
    if (ret < 0)
        return ret;

    source->start();

    // A real application runs its business loop or waits for EOS here.
    std::getchar();

    source->stop();
    return 0;
}

The connection is:

ModuleFileReader -> ModuleMppDec -> ModuleFileWriter

Calling start() on the source module starts its connected downstream modules. Calling stop() on the source stops the pipeline. To display and save at the same time, connect the same decoder to multiple output modules.

Set Parameters

Parameters can be set directly through module interfaces or through parameter paths managed uniformly by MediaParameter. Check the return values before running the pipeline:

auto encoder = std::make_shared<ModuleMppEnc>(MEDIA_CODEC_VIDEO_H264);

if (encoder->setParameter("encode/fps", 30) < 0)
    return -1;

if (encoder->setParameter("encode/bitrate", 4096) < 0)
    return -1;

Use the current module's queryParameter() or params output as the source of truth for parameter names, types, defaults, ranges, and writable states. See the FFMedia parameter system for complete rules.

Python Integration

The Python binding module is named ff_pymedia. It exposes the main module, parameter, channel-connection, and lifecycle features. It is suitable for rapid development, automated validation, and script integration; use C++ when the application needs full control over threads, devices, or custom modules.

Install and Import

Install a wheel that matches the Python interpreter and system architecture:

python3 -m pip install python/ff_pymedia-<wheel matching the current Python version>.whl
import ff_pymedia as ff

The Python version, wheel architecture, and shared libraries must match. The exact wheel filename depends on the contents of the current package.

Create a Python Pipeline

The following example reads a file, decodes it with MPP, and receives decoded buffers through a produce hook:

import ff_pymedia as ff


def on_frame(name, value, buffer):
    # Keep an independent copy if the frame is needed after the callback.
    frame = buffer.clone()
    if frame is not None:
        print(name, frame.getActiveSize())


source = ff.ModuleFileReader("input.mp4", False)
if source.init() < 0:
    raise RuntimeError("failed to initialize source")

decoder = ff.ModuleMppDec()
if decoder.connectProducer(source) < 0:
    raise RuntimeError("failed to connect decoder")
if decoder.init() < 0:
    raise RuntimeError("failed to initialize decoder")
if not decoder.setMediaBufferProduceHooker(on_frame):
    raise RuntimeError("failed to set output hook")

source.start()
try:
    input("Press Enter to stop...\n")
finally:
    source.stop()

Use the current binding exports as the source of truth for Python constructors, parameter paths, and callback signatures. The current ff_pymedia binding does not export ModuleAppSource, ModuleAppProcessor, or application-processing callback types. Use the exported ModuleMemReader for application input, or use the C++ API. See the Python section of the FFMedia API for other binding differences.

CLI Quick Validation

The ffmedia CLI is a generic pipeline validation tool. Use it before writing an application to check:

  • Whether the current SDK provides the required modules.
  • Whether parameter paths, types, and values are valid.
  • Whether producer output channels match consumer input channels.
  • Whether the pipeline connection graph is correct.

Query Modules and Parameters

./bin/ffmedia modules
./bin/ffmedia params mpp-dec
./bin/ffmedia params rga
./bin/ffmedia run --help

Describe a Pipeline with the CLI

The CLI uses three kinds of arguments:

ArgumentPurposeExample
-m ID=TYPEDeclare a module instance-m decoder=mpp-dec
-p ID:PARAMETERSSet module parameters-p 'decoder:output{format=NV12}'
-c PRODUCER[@CHANNELS]=CONSUMERConnect modules-c source@0=decoder

Example 1: Read a media file, decode it, and display it in a system window.

./bin/ffmedia run \
    -m source=ffmpeg-demux \
    -m decoder=mpp-dec \
    -m dis=renderer-video \
    -p 'source:source{uri=input.mp4;loop=0}' \
    -c source=decoder \
    -c decoder=dis \
    --sync dis=video

Example 2: Read four media files, decode them, compose them into a 2x2 layout on a 1080p canvas, and display the result in a 720p system window.

./bin/ffmedia run \
  -m src1=file-reader -m src2=file-reader -m src3=file-reader -m src4=file-reader \
  -m dec1=mpp-dec -m dec2=mpp-dec -m dec3=mpp-dec -m dec4=mpp-dec \
  -m stack=video-stack \
  -m vo=renderer-video \
  -p 'src1:source{path=/path/to/input1.mp4;loop=true}' \
  -p 'src2:source{path=/path/to/input2.mp4;loop=true}' \
  -p 'src3:source{path=/path/to/input3.mp4;loop=true}' \
  -p 'src4:source{path=/path/to/input4.mp4;loop=true}' \
  -p 'stack:output{width=1920;height=1080;format=NV12};frame-rate=30' \
  -p 'stack:input-layout{input-id=0;crop{x=0;y=0;width=956;height=536}}' \
  -p 'stack:input-layout{input-id=1;crop{x=964;y=0;width=956;height=536}}' \
  -p 'stack:input-layout{input-id=2;crop{x=0;y=544;width=956;height=536}}' \
  -p 'stack:input-layout{input-id=3;crop{x=964;y=544;width=956;height=536}}' \
  -p 'vo:window{x=100;y=100;width=1280;height=720}' \
  -c src1=dec1 \
  -c src2=dec2 \
  -c src3=dec3 \
  -c src4=dec4 \
  -c dec1=stack \
  -c dec2=stack \
  -c dec3=stack \
  -c dec4=stack \
  -c stack=vo

Example 3: Decode a 1080P RTSP video stream and convert it to an RGB24 canvas. Blend three locally decoded video frames onto this canvas with transparency levels of 0.3, 0.6, and 1.0 respectively, then render the canvas to the system window for display.

./ffmedia run \
  -m rtsp=rtsp-client \
  -m file1=file-reader \
  -m file2=file-reader \
  -m file3=file-reader \
  -m netdec=mpp-dec \
  -m dec1=mpp-dec \
  -m dec2=mpp-dec \
  -m dec3=mpp-dec \
  -m mix=image-processor \
  -m dis=renderer-video \
  -p 'rtsp:source{url=rtsp://admin:firefly123@172.16.2.100:554/av_stream}' \
  -p 'file1:source{path=input.mp4;loop=true}' \
  -p 'file2:source{path=input.mp4;loop=true}' \
  -p 'file3:source{path=input.mp4;loop=true}' \
  -p 'mix:output{width=1920;height=1080;format=RGB24}' \
  -p 'mix:blend{input-id=1;destination-rect{x=40;y=40;width=640;height=360};opacity=0.3}' \
  -p 'mix:blend{input-id=2;destination-rect{x=1200;y=40;width=640;height=360};opacity=0.6}' \
  -p 'mix:blend{input-id=3;destination-rect{x=40;y=670;width=640;height=360};opacity=1.0}' \
  -p 'dis:title="RTSP + 3 local videos"' \
  -c rtsp=netdec \
  -c file1=dec1 \
  -c file2=dec2 \
  -c file3=dec3 \
  -c netdec=mix \
  -c dec1=mix \
  -c dec2=mix \
  -c dec3=mix \
  -c mix=dis

For the complete CLI syntax and options, see the upstream ffmedia command-line guide.

Common Problems

SymptomFirst checks
Shared library not foundCheck whether LD_LIBRARY_PATH contains the SDK's lib/aarch64-linux-gnu or lib directory
Module not foundCheck whether modules lists the module and whether the required optional component is enabled
Parameter setting failedCheck the path, type, range, and current state against the params output
Connection failedCheck the channel ID, media type, and consumer input requirements
DRM or Camera initialization failedCheck device permissions, device formats, connector, and plane settings
Python import failedCheck the Python version, wheel architecture, and shared library path

On this page