Demo Applications

FireflyChat

FireflyChat is a graphical application platform for LLM developed by the Firefly team. Users can quickly experience the actual use of various large models on the visual WebUI provided by FireflyChat.

Installation

You need to obtain the required installation package from the Download Center to install FireflyChat. There are two installation packages, which are fireflychat*.deb and libfirefly-ai*.deb.

After obtaining the installation package, transfer it to AIBOX-1684X and run the following command to install it:

# Install venv
sudo apt update
sudo apt install python3-venv

# Install FireflyChat
sudo dpkg -i libfirefly-ai*.deb
sudo dpkg -i fireflychat*.deb

After installing FireflyChat, you need to import the model file to FireflyChat to use it properly.

Model import

LLM

Users can obtain the model files (.bmodel) provided by the Firefly team from the LLM model resource in the Download Center.

After obtaining the model file, transfer it to AIBOX-1684X and run the following command to import it:

# Move the model file location, where <model_path> replace with the actual model file path
sudo mv <model_path> /models/LLM/

Embedding model

Users can obtain the model files (.bmodel) provided by the Firefly team from the embedding model resource in the Download Center.

After obtaining the model file, transfer it to AIBOX-1684X and run the following command to import it:

# Move the model file location, where <model_path> replace with the actual model file path
sudo mv <model_path> /models/embedding/

Restart FireflyChat

After adding or modifying the required model files, you need to restart FireflyChat for the new model to take effect:

sudo systemctl restart FireflyChat

Software usage

WebUI usage (Main usage methods)

FireflyChat will run automatically after it is installed successfully. You can directly access FireflyChat by entering <ip_address>:7860 in the browser.

Note: <ip_address> must be replaced with the actual IP address, for example: 172.16.11.18:7860.

Software background management

FireflyChat is an application registered as the systemd service. You can control the running status of FireflyChat by running the following commands:

# Stop FireflyChat
sudo systemctl stop FireflyChat

# Restart FireflyChat
sudo systemctl restart FireflyChat

# Disable the automatic startup of FireflyChat
sudo systemctl disable FireflyChat

# Viewing run logs
sudo journalctl -u FireflyChat

Quickly deploy Llama3

To facilitate a quick experience with Llama3, Firefly team has provided an out-of-the-box rapid deployment package, which can be obtained from Download Center.

Deployment process

Transfer the deployment package to the AIBOX-1684X machine, using the scp command is recommended:

Run the ifconfig command on AIBOX-1684X to view the IP address.
Run the scp command to transmit the deployment package, for example: scp llama3.tar linar@ <aibox-ip-addr>:~/, where <aibox-ip-addr> is the IP address of AIBOX-1684X.

Decompress the deployment package: tar -xvf ./llama3.tar.

One command to deploy Llama3: ./talk_to_llama3.sh.

Note: Networking is not required during the deployment. The first deployment of Llama3 requires a long period of time (3-5 minutes) for software installation.

On this page