Terminal Command Guide

llamapi-cli is the LlamaPi command-line component. Its command entry point is llamapi, which provides hardware inspection, model discovery and download, chat execution, model instance management, and automatic-load configuration.

For task-oriented workflows, read Download and Manage Models, Run and Deploy Models, and Persistent Deployment first. This chapter documents the syntax, options, prerequisites, behavior, and output of every command.

Command Format

llamapi [OPTIONS] <COMMAND>

Available commands:

Global Options

--config is a top-level option and must appear before the subcommand:

llamapi --config /path/to/config.toml load qwen3:4b

Check the version:

llamapi --version

Current output is similar to:

✦ Firefly AI · LlamaPi v0.2.0

firefly-llamapi-cli and firefly-llamapi-server always have the same version.

llamapi-cli Configuration

Default path:

~/.config/llamapi-cli/config.toml

When the file is absent, the llamapi-cli uses defaults. A complete example is:

[model_store]
path = "/var/lib/llamapi/models"

[download]
source = "auto"

[server]
config_path = "/etc/llamapi-server/config.toml"

The llamapi-cli reads host and port from the llamapi-server configuration. A bind address such as 0.0.0.0 is converted to 127.0.0.1 for local access. All examples use port 9265.

Model Argument Conventions

Model Names

Supported forms include:

qwen3:4b
bge-m3

Model variants are displayed as:

model(platform/chip)

For example:

qwen3:4b(rkllm/rk3588)

Platform Option

Use the following option when a variant must be selected explicitly:

-p, --platform <platform>/<chip>

Platform and chip names are case-sensitive. Use values reported by llamapi platform and llamapi list --online.

Runtime IDs

The default runtime ID is:

name@platform-chip

A custom ID:

• May contain Chinese characters.
• Cannot contain whitespace.
• Cannot contain /.
• Cannot have a display width greater than 20.
• Cannot conflict with a loaded model group or automatic-load entry.

Environment and Status

platform

Shows hardware, inference platforms, and llamapi-server status.

llamapi platform

The command has no options.

Example with a running service:

SoC:                    RK3588
Coprocessor:            null
Supported platforms:    rkllm, rknn2
Unsupported platforms:  rknn3
Service:                running  (http://127.0.0.1:9265/v1)

Example when the service is stopped:

SoC:                    unknown
Coprocessor:            unknown
Supported platforms:    unknown (service not running)
Unsupported platforms:  unknown (service not running)
Service:                not running  (http://127.0.0.1:9265/v1)

Run 'llamapi run <model>' to start the service.

Notes:

• SoC, coprocessor, and platform data come from GET /v1/platforms.
• Multiple identical chips are displayed as model ×count.
• An undetected coprocessor is displayed as null.
• A running service with no available platform reports none.
• A stopped service cannot provide hardware data, so related values are unknown.

list

Lists models. Without options, it lists downloaded local models.

llamapi list [OPTIONS]

Examples:

llamapi list
llamapi list --online
llamapi list --all

Output columns:

list --online depends on /v1/platforms and does not start the llamapi-server. It only shows variants with available: true and a matching detected chip.

list --all does not filter by hardware and does not depend on the platform API.

Remote lookup behavior:

• auto queries Hugging Face and ModelScope concurrently and uses the first valid result.
• Success from one source cancels the unfinished request to the other source.
• Each source is attempted up to three times, with 500 ms and 1000 ms delays before retries.
• Connection timeout is 3 seconds, each request timeout is 6 seconds, and the total lookup limit is 20 seconds.
• Connection, send, read, parse, HTTP 429, and HTTP 5xx failures are retried.
• Other 4xx responses and empty repository lists are not retried.

Sorting is stable and natural: first by MODEL, then by PLATFORM. Models without a size suffix precede sized variants, and sizes are ordered numerically.

ps

Shows loaded models and automatic-load configuration.

llamapi ps

The command has no options.

Example:

ID       MODEL       TYPE        PLATFORM          STATUS        INSTANCES    ENABLE
abc      qwen3:4b    chat        rkllm/rk3588      ● active      2            yes(1)
demo     qwen3:4b    chat        rkllm/rk3588      ○ inactive    -            yes(1)

A model that is only downloaded, with no loaded runtime ID and no automatic-load entry, is not shown. When there are no matching models, only the table header is printed.

Local Model Management

pull

Downloads a model into the local model store.

llamapi pull <MODEL> [OPTIONS]

Examples:

llamapi pull qwen3:4b
llamapi pull qwen3:4b --platform rknn3/rk1828
llamapi pull qwen3:4b --source modelscope

Execution flow:

1. Query /v1/platforms for available platforms and chips.
2. Select a remote repository from the model name and platform.
3. Prompt in an interactive terminal when multiple compatible candidates exist; require --platform otherwise.
4. Ask whether to redownload an existing model.
5. Download files with progress output.
6. Validate model.toml.
7. Display total model size.

pull does not start the llamapi-server and does not load the model after downloading.

Successful output is similar to:

✦ Firefly AI · LlamaPi
✔ Download complete: qwen3:4b(rkllm/rk3588)
  Platform  rkllm/rk3588
  Size      3.8 GB

rm

Removes local model files.

llamapi rm <MODEL> [OPTIONS]

Examples:

llamapi rm qwen3:4b
llamapi rm qwen3:4b --platform rkllm/rk3588

If matching runtime IDs are active, the llamapi-cli asks whether to unload them. After confirmation, it removes the model directory and displays the freed disk space.

Multiple local variants prompt interactively; non-interactive use requires an explicit platform.

Model Execution

run

Starts the llamapi-server as needed, downloads and loads a chat model, and then opens an interactive session or sends a single prompt.

llamapi run <MODEL_OR_ID> [PROMPT] [OPTIONS]

Examples:

llamapi run qwen3:4b
llamapi run qwen3:4b "Hello"
llamapi run qwen3:4b --system "You are a Python assistant."
llamapi run qwen3:4b --platform rknn3/rk1828
llamapi run abc
llamapi run qwen3:4b "Write a poem" --temperature 0.9 --max-tokens 512
llamapi run qwen3:4b "Hello" --stats

Key behavior:

1. Starts the llamapi-server if it is not running.
2. Uses an exact loaded runtime ID directly.
3. Strictly uses an explicit platform without automatic performance selection.
4. Prefers compatible local chat variants when no platform is specified.
5. Queries remote repositories only when no local variant is available.
6. Rejects chat when only an Embedding variant exists and suggests load.
7. Downloads a missing model automatically.
8. Uses the default runtime ID for model-name execution.
9. Reuses an existing default ID that points to the same model directory without resizing it.
10. Reports a conflict if the default ID points to another directory.
11. Unloads only a model newly loaded by the current command.
12. Does not stop a llamapi-server that it started automatically.

With --stats, the following statistics are displayed after the response:

--- stats ---
prompt_tokens:     13
completion_tokens: 14
first_token:       380 ms
speed:             18.7 token/s

Concurrency behavior

Multiple run processes share one runtime model group, but version 0.2.0 has no session lease or reference count. The process that loaded the model may unload it while another process is still using it.

For stable concurrent serving:

1. Preload the model with llamapi load.
2. Have clients call the HTTP API directly.
3. Configure instance count and request queue for the available hardware.

Interactive Mode Commands

llamapi run <MODEL> enters interactive mode.

/set supports:

Enter """ to start multiline input and enter it again to finish. Press Ctrl+C during generation to interrupt the current response. Press Ctrl+C at the input prompt or Ctrl+D to exit.

Model Instance Management

load

Loads a local model or resizes an existing runtime ID.

llamapi load <MODEL_OR_ID> [OPTIONS]

Examples:

llamapi load qwen3:4b
llamapi load qwen3:4b --instance 2
llamapi load qwen3:4b --platform rkllm/rk3588 --id abc --instance 2
llamapi load abc --instance 3

Behavior:

• An exact active ID is resized to --instance.
• A local model name loads the requested number of instances.
• An existing default ID for the same model is resized.
• Matching instance counts result in an already-loaded message and no operation.
• Partial instance creation can still succeed and reports both requested and actual counts.
• --id cannot be used when the positional argument already matches an existing ID.
• Multiple local variants prompt interactively; non-interactive use requires --platform.

The llamapi-server must be running. A model-name load requires the model to be downloaded locally.

Coprocessor limitation: Using --instance to load multiple model instances on a coprocessor may cause loading failure, chip communication failure, and abnormal service state. Keep --instance 1. See Coprocessor Communication Failure After Loading Multiple Model Instances for recovery.

unload

Unloads a model group and all of its instances.

llamapi unload <MODEL_OR_ID> [OPTIONS]

Examples:

llamapi unload qwen3:4b
llamapi unload abc
llamapi unload qwen3:4b --platform rkllm/rk3588

Matching rules:

1. Match an exact ID first.
2. If no ID matches, match by model name.
3. Unload directly when one model group matches.
4. Prompt interactively when multiple groups match.
5. Require an explicit ID in non-interactive use.

IDs, model names, and platforms are case-sensitive. The llamapi-server must be running.

Automatic Loading

enable

Adds a model to the llamapi-server configuration so it loads whenever the service starts.

llamapi enable <MODEL> [OPTIONS]

Examples:

llamapi enable qwen3:4b
llamapi enable qwen3:4b --instance 2 --now
llamapi enable qwen3:4b --platform rkllm/rk3588 --id abc --instance 2

If an ID already points to the same model directory, the command updates its instance count and defaults. If the ID points to another directory, the command reports a conflict.

Multiple local variants prompt interactively; non-interactive use requires an explicit platform.

disable

Removes a model automatic-load entry from the llamapi-server configuration.

llamapi disable <MODEL_OR_ID> [OPTIONS]

Examples:

llamapi disable qwen3:4b
llamapi disable qwen3:4b --now
llamapi disable abc

An exact configured ID removes that ID. Otherwise, the llamapi-cli derives the default ID from the model name and platform. A custom ID must be specified explicitly and is not removed indirectly by disable qwen3:4b.