Back to articles
AI & Cloud

Build an AI Automated Video Editor System on Ubuntu VPS with ComfyUI and FFmpeg API

A detailed guide on building an automated AI video editing system on Ubuntu VPS using ComfyUI and FFmpeg API, covering installation, configuration, API integration, and testing.

DPTCloud Engineering4 min read
Build an AI Automated Video Editor System on Ubuntu VPS with ComfyUI and FFmpeg API

Image credit: terydanphiri · Pixabay Content License

Build an AI Automated Video Editor System on Ubuntu VPS with ComfyUI and FFmpeg API

In the digital era, automating video processing workflows has become essential for businesses and individuals alike. With the rapid advancement of AI, building an AI Automated Video Editor system on an Ubuntu VPS offers a flexible, cost-effective, and scalable solution. This guide walks you through setting up such a system using ComfyUI—a GUI for Stable Diffusion—and FFmpeg API, a powerful video processing tool.

Requirements and Prerequisites

Before starting, ensure you have:

  • A Ubuntu 20.04/22.04 VPS with at least 8GB RAM (16GB recommended for heavy AI models)
  • SSH access to the VPS
  • Python 3.8+ and pip installed
  • Git for cloning repositories
  • GPU (optional but recommended) for AI processing acceleration

Install and Configure ComfyUI

ComfyUI is a graphical interface for Stable Diffusion that enables you to build image and video processing pipelines by connecting nodes. To install ComfyUI on your Ubuntu VPS:

git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

After installation, start ComfyUI with the following command:

python main.py --listen 0.0.0.0:8188 --listen 127.0.0.1:8188 --listen 192.168.1.100:8188

This starts ComfyUI on port 8188. You can access the interface via a browser at http://your-vps-ip:8188.

Integrate FFmpeg API for Video Processing

FFmpeg is a widely used video processing tool supporting formats and functions like trimming, merging, format conversion, and effects. To integrate FFmpeg with ComfyUI, install FFmpeg and create a Python API wrapper.

Install FFmpeg on the VPS:

sudo apt update
sudo apt install ffmpeg -y

Create a Python script, e.g., ffmpeg_api.py:

import subprocess
import os

def run_ffmpeg_command(input_path, output_path, command):
    try:
        result = subprocess.run(
            ["ffmpeg", "-i", input_path] + command.split() + [output_path],
            check=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE
        )
        return {"status": "success", "output": result.stdout.decode()}
    except subprocess.CalledProcessError as e:
        return {"status": "error", "error": e.stderr.decode()}

# Example: Trim video from 10s to 30s
run_ffmpeg_command("input.mp4", "output.mp4", "-ss 10 -t 20 -c copy")

Integrate ComfyUI and FFmpeg API

To create an automated video editing pipeline, use ComfyUI to process images (e.g., generate AI-enhanced frames), then pass the output to FFmpeg for video processing. The pipeline works as follows:

  1. User uploads a video via API
  2. ComfyUI processes images (e.g., generates AI background or effects)
  3. FFmpeg processes the video: merges AI frames, trims, converts formats
  4. Final video is saved and returned

Automate this using FastAPI to create an API endpoint:

from fastapi import FastAPI, File, UploadFile
from fastapi.responses import JSONResponse
import os

app = FastAPI()

@app.post("/edit-video")
async def edit_video(file: UploadFile = File(...)):
    # Save temporary file
    temp_path = f"/tmp/{file.filename}"
    with open(temp_path, "wb") as f:
        f.write(await file.read())

    # Call ComfyUI for image processing (via API or script)
    # Call FFmpeg to process video
    ffmpeg_command = f"ffmpeg -i {temp_path} -vf \"scale=1280:720\" -c:v libx264 -crf 23 output.mp4"
    os.system(ffmpeg_command)

    return JSONResponse(content={"status": "success", "output": "Video processed successfully"})

Testing and Monitoring

After deployment, test the system by:

  • Uploading a video via the API
  • Verifying ComfyUI processes images correctly
  • Ensuring FFmpeg outputs the video without errors
  • Monitoring VPS resources (RAM, CPU) using htop or top

Troubleshooting

  • ComfyUI fails to start: Check GPU CUDA support or switch to CPU mode.
  • FFmpeg errors: Verify input video format; use ffmpeg -formats to check supported formats.
  • API not responding: Confirm ports 8188 (ComfyUI) and 8000 (FastAPI) are open.

References

Building an AI Automated Video Editor on an Ubuntu VPS with ComfyUI and FFmpeg API gives you full control over video processing workflows, enabling high customization for both business and personal use.

DPTCloud Tier III Infrastructure

High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache

Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.

View 22 VPS Plans
ECO · vn-02Recommended

VPS-4E

297.000 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe
ECO · vn-02

VPS-5E

614.900 ₫ / mo

10% VAT included

  • CPU: 8 vCPU
  • RAM: 16 GB
  • SSD: 100 GB NVMe
ADVANCED · vn-03Recommended

VPS-4T

548.900 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe

Related Articles

View all
ZaloZalo SupportDiscord0384 753 478