Back to articles
AI & Cloud

Using Dagger and Gopython to Eliminate All YAML Files in CI/CD Pipeline

A technical guide on using Dagger and Gopython to completely remove YAML files from CI/CD pipelines, replacing them with pure Python code for increased flexibility and security.

DPTCloud Engineering4 min read
Using Dagger and Gopython to Eliminate All YAML Files in CI/CD Pipeline

Image credit: Zezya · Pixabay Content License

Introduction

In modern CI/CD systems, YAML files are commonly used to define pipelines, but this approach brings several limitations in flexibility, security, and control. This guide introduces how to use Dagger and Gopython to completely replace YAML files with pure Python code, enabling more flexible, secure, and maintainable CI/CD automation.

Why Remove YAML from CI/CD?

YAML files in CI/CD (e.g., in GitHub Actions, GitLab CI, or Drone CI) are often criticized for:

  • Lack of programmability: No support for conditions, loops, functions, or complex logic.
  • Security risks: YAML code can be compromised or altered if access is not strictly controlled.
  • Hard to test: No unit testing capability for pipeline logic.
  • Hard to scale: Complex pipelines become difficult to read and maintain in YAML.

Using Dagger to Build Pipelines with Python

Dagger is a programming tool for building CI/CD pipelines using code—typically Python—allowing you to define workflow steps as Python functions. You can run Dagger on a High-RAM Cloud VPS for optimal performance.

Here is a basic example of using Dagger to build and test a Python application:

import dagger

async def main():
    # Initialize Dagger
    client = dagger.Client()

    # Get Python image
    python = client.container().from_("python:3.11")

    # Mount source code
    src = client.host().directory("./src")

    # Build and test
    result = (
        python
        .with_directory("/src", src)
        .with_workdir("/src")
        .with_exec(["pip", "install", "-r", "requirements.txt"])
        .with_exec(["python", "-m", "pytest"])
    )

    # Get output
    output = await result.stdout()
    print(output)

# Run the program
import asyncio
asyncio.run(main())

Using Gopython to Check and Process Python Code in the Pipeline

Gopython is a Python code checker that detects syntax errors, type issues, and optimizes code. When building pipelines in Python, code validation is essential to ensure quality.

Run Gopython to check Python code in your project:

# Install Gopython
pip install gopython

# Run checks
gopython check ./src --strict

Building a Fully Python-Based Pipeline

By combining Dagger and Gopython, you can build a CI/CD pipeline entirely in Python, eliminating the need for YAML files. Example:

import dagger
import subprocess

async def build_and_test():
    client = dagger.Client()

    # Build image
    image = (
        client.container()
        .from_("python:3.11")
        .with_directory("/src", client.host().directory("./src"))
        .with_workdir("/src")
        .with_exec(["pip", "install", "-r", "requirements.txt"])
        .with_exec(["python", "-m", "pytest"])
    )

    # Check code with Gopython
    subprocess.run(["gopython", "check", "./src", "--strict"])

    # Build and push image
    result = await image.publish("myapp:latest")
    print(f"Image published: {result}")

# Run the pipeline
import asyncio
asyncio.run(build_and_test())

Execution and Verification

To verify the pipeline:

  1. Create a Python project with a src folder and requirements.txt.
  2. Create a pipeline.py file with the Python code above.
  3. Run: python pipeline.py on a High-RAM Cloud VPS.
  4. Ensure Gopython is installed and runs code checks.

Troubleshooting

  • Dagger fails to run: Verify Docker is running and has proper access permissions.
  • Gopython not found: Reinstall with pip install gopython.
  • Access issues: Ensure the VPS has access to the source directory.

References

DPTCloud Tier III Infrastructure

High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache

Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.

View 22 VPS Plans
ECO · vn-02Recommended

VPS-4E

297.000 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe
ECO · vn-02

VPS-5E

614.900 ₫ / mo

10% VAT included

  • CPU: 8 vCPU
  • RAM: 16 GB
  • SSD: 100 GB NVMe
ADVANCED · vn-03Recommended

VPS-4T

548.900 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe

Related Articles

View all
ZaloZalo SupportDiscord0384 753 478