DeOldify服务CI/CD流水线：GitHub Actions自动构建镜像+部署验证

1. 项目概述

DeOldify是一款基于深度学习技术的图像上色工具，能够将黑白照片自动转换为彩色照片。本文将详细介绍如何通过GitHub Actions构建完整的CI/CD流水线，实现DeOldify服务的自动化构建、测试和部署。

1.1 技术架构

  • 核心模型：基于U-Net架构的DeOldify深度学习模型
  • 服务框架：Flask + Gunicorn
  • 容器化：Docker镜像
  • CI/CD工具：GitHub Actions
  • 部署平台：支持Kubernetes、ECS等主流容器平台

2. CI/CD流水线设计

2.1 整体流程

  1. 代码提交：开发者推送代码到GitHub仓库
  2. 自动触发：GitHub Actions检测到变更并启动流水线
  3. 构建阶段：构建Docker镜像并推送到镜像仓库
  4. 测试阶段：运行自动化测试验证服务功能
  5. 部署阶段：将镜像部署到测试环境
  6. 验证阶段：执行端到端测试验证部署结果
  7. 生产发布：人工审核后发布到生产环境

2.2 关键组件

  • workflow文件：.github/workflows/deoldify-ci-cd.yml
  • Dockerfile：定义容器构建规范
  • 测试脚本：包含单元测试和集成测试
  • 部署脚本：Kubernetes/ECS部署配置

3. 实现步骤详解

3.1 准备工作

3.1.1 项目结构
deoldify-service/
├── app/
│   ├── main.py          # Flask应用入口
│   ├── model.py         # 模型加载与推理
│   └── utils.py         # 工具函数
├── tests/
│   ├── unit/            # 单元测试
│   └── integration/     # 集成测试
├── Dockerfile           # 容器构建文件
├── requirements.txt     # Python依赖
└── .github/
    └── workflows/
        └── deoldify-ci-cd.yml  # CI/CD工作流
3.1.2 Dockerfile配置
FROM python:3.9-slim

WORKDIR /app

# 安装系统依赖
RUN apt-get update && apt-get install -y \
    libgl1-mesa-glx \
    libglib2.0-0 \
    && rm -rf /var/lib/apt/lists/*

# 复制项目文件
COPY requirements.txt .
COPY . .

# 安装Python依赖
RUN pip install --no-cache-dir -r requirements.txt

# 下载模型
RUN python -c "from app.model import load_model; load_model()"

# 暴露端口
EXPOSE 7860

# 启动命令
CMD ["gunicorn", "--bind", "0.0.0.0:7860", "--workers", "4", "app.main:app"]

3.2 GitHub Actions配置

3.2.1 基础工作流
name: DeOldify CI/CD Pipeline

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

env:
  IMAGE_NAME: deoldify-service
  REGISTRY: ghcr.io
  REPOSITORY: ${{ github.repository }}

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'
          
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install pytest
          
      - name: Run unit tests
        run: |
          pytest tests/unit -v
          
      - name: Build Docker image
        run: docker build -t $IMAGE_NAME .
        
      - name: Run integration tests
        run: |
          docker run -d -p 7860:7860 --name deoldify-test $IMAGE_NAME
          sleep 10  # 等待服务启动
          pytest tests/integration -v
          docker stop deoldify-test
          docker rm deoldify-test
3.2.2 镜像构建与推送
  build-and-push:
    needs: build-and-test
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Log in to GitHub Container Registry
        uses: docker/login-action@v2
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
          
      - name: Build and push Docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: |
            ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:latest
            ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:${{ github.sha }}
3.2.3 部署到测试环境
  deploy-to-test:
    needs: build-and-push
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
          
      - name: Deploy to ECS
        run: |
          # 更新ECS任务定义
          aws ecs register-task-definition \
            --family deoldify-test \
            --network-mode awsvpc \
            --execution-role-arn ${{ secrets.ECS_TASK_ROLE }} \
            --container-definitions '[
              {
                "name": "deoldify",
                "image": "${{ env.REGISTRY }}/${{ env.REPOSITORY }}:${{ github.sha }}",
                "essential": true,
                "portMappings": [
                  {
                    "containerPort": 7860,
                    "hostPort": 7860
                  }
                ],
                "environment": [
                  {
                    "name": "ENVIRONMENT",
                    "value": "test"
                  }
                ]
              }
            ]'
            
          # 更新服务
          aws ecs update-service \
            --cluster deoldify-cluster \
            --service deoldify-test \
            --task-definition deoldify-test \
            --force-new-deployment

3.3 测试验证

3.3.1 单元测试示例
# tests/unit/test_model.py
import pytest
from app.model import colorize_image
from PIL import Image
import numpy as np

def test_colorize_image():
    # 创建测试用的黑白图像
    test_img = Image.new('L', (100, 100), color=128)
    
    # 调用上色函数
    colored_img = colorize_image(test_img)
    
    # 验证结果
    assert colored_img.mode == 'RGB'
    assert colored_img.size == (100, 100)
    assert np.array(colored_img).shape == (100, 100, 3)
3.3.2 集成测试示例
# tests/integration/test_api.py
import requests
import base64
from io import BytesIO
from PIL import Image

BASE_URL = "http://localhost:7860"

def test_health_check():
    response = requests.get(f"{BASE_URL}/health")
    assert response.status_code == 200
    assert response.json()["status"] == "healthy"

def test_colorize_api():
    # 创建测试图像
    test_img = Image.new('L', (100, 100), color=128)
    img_byte_arr = BytesIO()
    test_img.save(img_byte_arr, format='JPEG')
    img_byte_arr.seek(0)
    
    # 调用API
    files = {'image': ('test.jpg', img_byte_arr, 'image/jpeg')}
    response = requests.post(f"{BASE_URL}/colorize", files=files)
    
    # 验证响应
    assert response.status_code == 200
    result = response.json()
    assert result["success"] is True
    assert "output_img_base64" in result
    
    # 验证图像
    img_data = base64.b64decode(result["output_img_base64"])
    img = Image.open(BytesIO(img_data))
    assert img.mode == 'RGB'
    assert img.size == (100, 100)

4. 高级配置与优化

4.1 多阶段构建优化

# 构建阶段
FROM python:3.9-slim as builder

WORKDIR /app
COPY requirements.txt .

RUN apt-get update && apt-get install -y gcc python3-dev
RUN pip install --user -r requirements.txt

# 运行时阶段
FROM python:3.9-slim

WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .

ENV PATH=/root/.local/bin:$PATH
ENV PYTHONPATH=/app

# 其余配置与之前相同...

4.2 缓存优化

# 在GitHub Actions中添加缓存步骤
- name: Cache Python dependencies
  uses: actions/cache@v3
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

4.3 安全扫描

- name: Scan for vulnerabilities
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: ${{ env.IMAGE_NAME }}
    format: 'table'
    exit-code: '1'
    ignore-unfixed: true
    severity: 'CRITICAL,HIGH'

5. 部署验证

5.1 自动化验证脚本

#!/bin/bash

# 验证服务健康状态
health_check() {
    local retries=5
    local delay=5
    
    for ((i=1; i<=retries; i++)); do
        response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:7860/health)
        
        if [ "$response" -eq 200 ]; then
            echo "Service is healthy"
            return 0
        fi
        
        echo "Attempt $i: Service not ready, waiting $delay seconds..."
        sleep $delay
    done
    
    echo "Health check failed after $retries attempts"
    return 1
}

# 验证图像上色功能
test_colorize() {
    local test_image="test.jpg"
    local output_file="output.jpg"
    
    # 创建测试图像
    convert -size 100x100 xc:gray -quality 90 "$test_image"
    
    # 调用API
    response=$(curl -s -X POST -F "image=@$test_image" http://localhost:7860/colorize)
    
    # 检查响应
    if echo "$response" | jq -e '.success == true' >/dev/null; then
        echo "Colorize API responded successfully"
        
        # 保存结果图像
        echo "$response" | jq -r '.output_img_base64' | base64 -d > "$output_file"
        
        if [ -s "$output_file" ]; then
            echo "Colorized image saved to $output_file"
            return 0
        else
            echo "Failed to save colorized image"
            return 1
        fi
    else
        echo "Colorize API failed: $response"
        return 1
    fi
}

# 主验证流程
main() {
    if health_check; then
        if test_colorize; then
            echo "Deployment validation PASSED"
            exit 0
        fi
    fi
    
    echo "Deployment validation FAILED"
    exit 1
}

main

5.2 监控与告警

- name: Setup monitoring
  run: |
    # 安装Prometheus exporter
    pip install prometheus-flask-exporter
    
    # 修改应用代码添加监控端点
    echo "from prometheus_flask_exporter import PrometheusMetrics" >> app/main.py
    echo "metrics = PrometheusMetrics(app)" >> app/main.py

6. 总结

通过本文介绍的CI/CD流水线，我们实现了DeOldify服务的自动化构建、测试和部署流程。这套方案具有以下优势：

  1. 自动化程度高：从代码提交到部署验证全流程自动化
  2. 质量保障：包含多层次的测试验证
  3. 可扩展性强：易于扩展到其他深度学习服务
  4. 安全可靠：包含安全扫描和健康检查
  5. 高效部署：支持快速迭代和回滚

实际部署时，可以根据具体需求调整各阶段配置，如增加性能测试、蓝绿部署等高级特性，进一步提升交付质量和效率。

获取更多AI镜像

想探索更多AI镜像和应用场景？访问 CSDN星图镜像广场，提供丰富的预置镜像，覆盖大模型推理、图像生成、视频生成、模型微调等多个领域，支持一键部署。

# 深度学习
Logo
九章云极普惠算力

更多推荐

vLLM-v0.17.1效果展示：vLLM在国产海光DCU平台上的ROCm兼容性验证

本文介绍了如何在星图GPU平台上自动化部署vLLM-v0.17.1镜像，实现高效的大语言模型推理服务。该镜像在海光DCU平台上展现出优异的ROCm兼容性，适用于构建高并发的AI问答系统，支持50+用户同时访问且响应时间低于500ms，显著提升服务效率。

avatar 九章云极普惠算力

VideoAgentTrek-ScreenFilter代码实例：Supervisor自启服务管理实战

本文介绍了如何在星图GPU平台上自动化部署VideoAgentTrek-ScreenFilter镜像，实现基于YOLO的视频/图片屏幕内容检测服务。通过配置Supervisor守护进程，该应用可升级为具备自动重启和状态监控能力的生产级服务，确保检测任务稳定运行。

avatar 九章云极普惠算力

DeepSeek-OCR-2效果展示：印章覆盖文字、朱砂批注干扰下的鲁棒性识别能力

本文介绍了如何在星图GPU平台自动化部署🖋️ 深求·墨鉴 (DeepSeek-OCR-2)镜像，实现复杂场景下的文字识别。该镜像特别适用于处理带有印章覆盖和朱砂批注干扰的文档数字化，如古籍保护、法律合同等场景，展现出色的鲁棒性和高精度识别能力。

avatar 九章云极普惠算力