Runway Video Generation Guide

This guide explains how to use the Runway AI video generation feature that has been integrated into the AI Content Processing system.

Overview

The Runway integration allows you to generate videos from static images using AI. It uses Runway's Gen-4 Turbo model to create short videos with smooth motion and cinematic quality.

Prerequisites

1. Runway API Key

You need a Runway API key to use this feature:

Sign up at runwayml.com
Get your API key from the Runway dashboard
Add it to your .env file:

RUNWAY_API_KEY=your_runway_api_key_here

2. Install Dependencies

Make sure the Runway SDK is installed:

pip install runwayml>=0.16.0

Or install from the updated requirements.txt:

pip install -r requirements.txt

Usage Methods

1. API Endpoint (Recommended)

Start the API Server

python run_api.py
# or
uvicorn api_server:app --host 0.0.0.0 --port 8000 --reload

Generate Video via HTTP POST

curl -X POST "http://localhost:8000/generate-video" \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://example.com/your-image.jpg",
    "prompt_text": "A serene mountain landscape with clouds slowly drifting",
    "ratio": "1280:720",
    "duration": 5
  }'

Python API Client

import requests

response = requests.post("http://localhost:8000/generate-video", json={
    "image_url": "https://example.com/image.jpg",
    "prompt_text": "Beautiful ocean waves with seabirds flying",
    "ratio": "1280:720",
    "duration": 8,
    "model": "gen4_turbo"
})

result = response.json()
print(f"Video URL: {result['video_url']}")

2. Command Line Interface

Use the provided CLI script:

# Basic usage
python runway_video_cli.py --image "https://example.com/image.jpg"

# Custom prompt and settings
python runway_video_cli.py \
  --image "https://example.com/mountain.jpg" \
  --prompt "Clouds slowly drifting across mountain peaks" \
  --duration 8 \
  --ratio "1024:1024" \
  --output "mountain_video.json"

3. Direct Python Usage

from src.text_extractor import TextExtractor

# Initialize extractor
extractor = TextExtractor()

# Generate video
result_json = extractor.generate_video_from_image(
    image_path="https://example.com/image.jpg",
    prompt_text="A peaceful forest scene with gentle wind",
    ratio="1280:720",
    duration=5
)

import json
result = json.loads(result_json)
print(f"Status: {result['status']}")
if result['status'] == 'success':
    print(f"Video URL: {result['video_url']}")

Parameters

Required Parameters

image_url (string): URL of the source image
- Must be a publicly accessible image URL
- Supported formats: JPG, PNG, GIF, BMP, TIFF, WEBP

Optional Parameters

prompt_text (string): Text describing the desired video content
- Default: "A cinematic video with smooth motion"
- Examples: "Gentle waves on a beach", "Clouds moving across the sky"
ratio (string): Video aspect ratio
- Default: "1280:720" (16:9)
- Options: "1280:720", "1024:1024", "720:1280", etc.
duration (integer): Video length in seconds
- Default: 5
- Range: 1-10 seconds
model (string): Runway model to use
- Default: "gen4_turbo"
- Available: "gen4_turbo" (currently the main model)

Response Format

The API returns a JSON response with the following structure:

{
  "request_id": "uuid-string",
  "status": "success|failed|timeout|error",
  "task_id": "runway-task-id",
  "video_url": "https://runway-generated-video.mp4",
  "source_image": "https://your-source-image.jpg",
  "prompt_text": "Your prompt text",
  "ratio": "1280:720",
  "duration": 5,
  "model": "gen4_turbo",
  "processing_time_seconds": 45.2,
  "error": null,
  "timestamp": "2025-09-26T10:30:00Z"
}

Status Values

success: Video generated successfully, video_url contains the result
failed: Generation failed, check error field for details
timeout: Generation took too long (>5 minutes), may still be processing
error: System error occurred, check error field

Examples

Example 1: Landscape Animation

curl -X POST "http://localhost:8000/generate-video" \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://images.unsplash.com/photo-1506905925346-21bda4d32df4",
    "prompt_text": "Majestic mountain peaks with clouds slowly drifting across the sky",
    "ratio": "1280:720",
    "duration": 8
  }'

Example 2: Ocean Scene

curl -X POST "http://localhost:8000/generate-video" \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://images.unsplash.com/photo-1439066615861-d1af74d74000",
    "prompt_text": "Gentle ocean waves with seabirds flying overhead",
    "ratio": "1280:720",
    "duration": 6
  }'

Example 3: Square Format for Social Media

curl -X POST "http://localhost:8000/generate-video" \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://images.unsplash.com/photo-1441974231531-c6227db76b6e",
    "prompt_text": "Peaceful forest with leaves gently swaying in the breeze",
    "ratio": "1024:1024",
    "duration": 5
  }'

Testing

Run the example script to test the integration:

python runway_video_examples.py

This script will:

Check API health
Test video generation via API endpoint
Test direct processor usage
Generate CURL examples
Save results to JSON files

Troubleshooting

Common Issues

"Runway processor not available"
- Check that RUNWAY_API_KEY is set in your .env file
- Verify the key is valid and active
"Import error: No module named 'runwayml'"
- Install the Runway SDK: pip install runwayml>=0.16.0
"Video generation timed out"
- Video generation can take 1-5 minutes depending on complexity
- The video might still be processing on Runway's servers
- Check the task ID in Runway's dashboard
"Failed to fetch asset" or "Image URL not accessible"
- Ensure the image URL is publicly accessible
- Try the URL in a browser to verify it works
- Use direct image URLs without complex query parameters
- For Unsplash images, use format: https://images.unsplash.com/photo-ID?w=800&q=80
- Avoid URLs with auto=format, dpr=, or ixlib= parameters
- Some CDNs block external access - try downloading and hosting elsewhere

Rate Limits

Runway has usage limits based on your subscription plan
The system includes automatic retry logic for temporary failures
Monitor your Runway dashboard for usage statistics

Integration Details

File Structure

The Runway integration consists of:

src/file_processors/runway_processor.py - Main processor class
src/config.py - Configuration (API key, model settings)
api_server.py - FastAPI endpoint /generate-video
runway_video_cli.py - Command-line interface
runway_video_examples.py - Testing and examples

Architecture

The Runway processor follows the same pattern as other processors in the system:

Inherits from BaseProcessor
Implements can_process() and extract_text() methods
Integrates with the TextExtractor orchestrator
Provides both direct usage and API endpoints

Performance

Video generation typically takes 30 seconds to 3 minutes
Processing time depends on video complexity and Runway server load
The system includes a 5-minute timeout with polling every 10 seconds
Results are returned as soon as generation completes

API Documentation

The video generation endpoint is automatically included in the FastAPI documentation:

Swagger UI: http://localhost:8000/docs
ReDoc: http://localhost:8000/redoc

Look for the /generate-video endpoint in the documentation for interactive testing.

Best Practices

Use descriptive prompts: Better prompts lead to better videos
Choose appropriate ratios: Match your intended use case (social media, web, etc.)
Start with shorter durations: 3-5 seconds often work best
Use high-quality source images: Better input leads to better output
Monitor processing time: Video generation is resource-intensive

Support

For issues specific to this integration:

Check the logs for detailed error messages
Verify your Runway API key and quota
Test with the provided examples first

For Runway-specific issues:

Check Runway's documentation: https://docs.dev.runwayml.com/
Monitor your Runway dashboard for usage and limits

Navigation Menu

Uh oh!

Runway Video Generation Guide

Overview

Prerequisites

1. Runway API Key

2. Install Dependencies

Usage Methods

1. API Endpoint (Recommended)

Start the API Server

Generate Video via HTTP POST

Python API Client

2. Command Line Interface

3. Direct Python Usage

Parameters

Required Parameters

Optional Parameters

Response Format

Status Values

Examples

Example 1: Landscape Animation

Example 2: Ocean Scene

Example 3: Square Format for Social Media

Testing

Troubleshooting

Common Issues

Rate Limits

Integration Details

File Structure

Architecture

Performance

API Documentation

Best Practices

Support

Uh oh!

FilesExpand file tree

RUNWAY_VIDEO_GUIDE.md

Latest commit

History

RUNWAY_VIDEO_GUIDE.md

File metadata and controls

Runway Video Generation Guide

Overview

Prerequisites

1. Runway API Key

2. Install Dependencies

Usage Methods

1. API Endpoint (Recommended)

Start the API Server

Generate Video via HTTP POST

Python API Client

2. Command Line Interface

3. Direct Python Usage

Parameters

Required Parameters

Optional Parameters

Response Format

Status Values

Examples

Example 1: Landscape Animation

Example 2: Ocean Scene

Example 3: Square Format for Social Media

Testing

Troubleshooting

Common Issues

Rate Limits

Integration Details

File Structure

Architecture

Performance

API Documentation

Best Practices

Support