Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,15 @@
}
]
},
{
"group": "Hybrid Cloud (Beta)",
"pages": [
"hybrid-cloud/overview",
"hybrid-cloud/setup",
"hybrid-cloud/deploy",
"hybrid-cloud/troubleshooting"
]
},
{
"group": "Accounts and billing",
"pages": [
Expand Down
39 changes: 39 additions & 0 deletions hybrid-cloud/deploy.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
title: "Deploy Pods on your external compute"
sidebarTitle: "Deploy to your compute"
description: "Deploy Pods onto the machines your organization has connected to Runpod, using the same console workflow you use for Runpod cloud."
---

After a machine reaches **Online**, anyone in your organization can deploy Pods on it. You use the same deploy flow you already use for Runpod cloud. The only difference is where you pick the GPU.

## Requirements

- At least one machine with the **Online** status. See [Set up external compute](/hybrid-cloud/setup).
- Membership in the organization that owns the machine, with permission to deploy Pods. See [Roles](/organizations/org-roles).

## Deploy a Pod

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also supported in serverless

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The completeness of the setup workflows for serverless are mixed at the moment. For example the quick deploy and deploy from hub options currently lack the External compute option.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prestwick Thanks for the context. This page only covers Pods for now, and Serverless is out of scope. Happy to add a note if you think readers will look for it.


<Steps>
<Step title="Open the Pods page">
In the Runpod console, make sure your organization is selected, then go to **Pods** and click **Deploy**.
</Step>
<Step title="Choose your external compute">
In the GPU picker, find the **External compute** section. It lists the GPU types available on your organization's online machines. Select the GPU you want.
</Step>
<Step title="Configure and deploy">
Configure the rest of the Pod the way you normally would, including the template, storage, and environment variables. Then click **Deploy**. For details on each Pod setting, see [Manage Pods](/pods/manage-pods).
</Step>
</Steps>

## How placement works

When you deploy on external compute, Runpod places the Pod on one of your organization's online machines that matches the GPU you selected. Pods deployed on your machines stay within your organization. Other Runpod customers can't deploy to them.

## Track usage

The **External compute** page shows how many GPUs on your machines are in use, so you can see available capacity before you deploy.

## Next steps

- [Troubleshoot external compute](/hybrid-cloud/troubleshooting).
- [Learn more about Pods](/pods/overview).
68 changes: 68 additions & 0 deletions hybrid-cloud/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: "Runpod Hybrid Cloud"
sidebarTitle: "Overview"
description: "Run AI workloads on the GPUs you own and on Runpod cloud from one console. Your hardware runs first, and Runpod cloud handles overflow."
---

Runpod Hybrid Cloud lets you bring hardware you already own under the same console, CLI, and APIs your team uses on Runpod. Your compute runs first, and Runpod cloud handles overflow when you need it.
Comment thread
prestwick marked this conversation as resolved.

## Requirements

Runpod Hybrid Cloud is available to organizations only. The Runpod team must enable External Compute for your org account before the feature appears in your console. [Contact us](https://www.runpod.io/hybrid-cloud) to get set up.

## How it works

<Steps>
<Step title="Add your machine">

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'machine' vs 'compute

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use compute. Lines up better with the planned terminology changes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment below for the usage of "machine" and "compute"

In the console, go to **Resources → External compute** and click **Add compute**. Give your machine a name and click **Next**.
</Step>
<Step title="Install the host connector">
Copy the install command that appears and paste it into an SSH session on your machine. This installs the Runpod host connector, a small program that connects your machine to Runpod over an encrypted connection. Open the required firewall ports, then reboot. For full instructions, see [Set up external compute](/hybrid-cloud/setup).
</Step>
<Step title="Wait for verification">
Runpod runs automated hardware checks in the background. Verification usually takes several minutes, and you don't need to do anything. When verification is complete, the machine's status changes to **Online** and its GPU type appears in the console.
</Step>
<Step title="Deploy from the same console">
Your team deploys Pods exactly as they would for any Runpod machine. An **External Compute** section appears in the GPU picker showing your available hardware. The deploy experience is identical to standard Runpod Pods. See [Deploy Pods on your external compute](/hybrid-cloud/deploy). External compute usage is tracked in the console so you can see how many GPUs are in use at any time.
</Step>
</Steps>

## What you get

**One control plane across owned and cloud compute**

Manage the hardware you own and Runpod cloud as a single estate. No scheduling scripts, access tickets, or per-silo tooling.

**Consistent developer experience**

Your team gets the same Runpod console, CLI, and APIs on the hardware you own. SSH still works.

**Your hardware, your data, your boundary**

You own the hardware and the data on it. Runpod orchestrates it without taking ownership. Everything you onboard is visible only to your organization.

**No lock-in**

The host connector uninstalls cleanly. Your workloads are standard containers.

## Who it's for

Runpod Hybrid Cloud is built for teams that own GPU infrastructure and want to put it to work without replatforming.

- **Research institutions and universities**: Give researchers self-service access to the institution's GPUs, with Runpod cloud absorbing overflow when the queue is full.
- **Enterprise teams**: Unify on-prem clusters, hyperscaler reservations, and Runpod cloud under one control plane with org-scoped governance.
- **AI-native startups**: Focus on your core product instead of building your own abstraction across compute providers. Runpod gives you a provider-agnostic API layer out of the box, so you can run on your own hardware first and burst to Runpod cloud without changing how your team deploys.

## Security

The host connector connects your machine to Runpod over an encrypted overlay network. Everything you onboard is scoped to your organization and isn't visible to other customers. For network requirements, see [Set up external compute](/hybrid-cloud/setup#requirements). For security reviews, you get a direct line to Runpod engineers to walk the architecture.

## Get started

Runpod Hybrid Cloud is in alpha and available to select customers, and the Runpod team works with you directly to set up your machines. [Request a demo](https://www.runpod.io/hybrid-cloud) to get started.

## Next steps

- [Set up external compute](/hybrid-cloud/setup).
- [Deploy Pods on your external compute](/hybrid-cloud/deploy).
- [Troubleshoot external compute](/hybrid-cloud/troubleshooting).
74 changes: 74 additions & 0 deletions hybrid-cloud/setup.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: "Set up external compute"
sidebarTitle: "Set up external compute"
description: "Connect a machine you own to Runpod Hybrid Cloud by installing the host connector, then track it through verification until it's online."
---

This guide walks you through connecting a machine you own to Runpod. After you add it and install the host connector, Runpod verifies the hardware automatically. The machine is then online and available to your organization.

## Requirements
Comment thread
lavanya-gunreddi marked this conversation as resolved.

- An [organization](/organizations/orgs-overview) with external compute enabled. During alpha, the Runpod team enables this for you.
- An organization role that allows you to add compute. See [Roles](/organizations/org-roles).
- A single machine with at least one GPU. Multi-node machines connected by InfiniBand or a similar fabric aren't supported yet.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maltmanRP you good with this language? thinking we could leave it more open ended like "talk to runpod" if they need multinode usecase.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this phrasing. Customer feedback on this will help with prioritization.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Promtpless - change it to "A single machine with at least one GPU. If you need multi-node support, such as machines connected by InfiniBand or a similar fabric, contact the Runpod team to discuss your use case."

- SSH access to the machine, with permission to install software.

Your machine must also meet these hardware and network requirements:

| Requirement | Value |
|---|---|
| Operating system | Ubuntu 24.04, with a kernel from 6.8 through 7.0.x. |
| GPU drivers | None preinstalled. The Runpod installer manages GPU drivers. |
| Root disk | At least 200 GB. |
| Docker disk | A dedicated disk of at least 600 GB. |
| Disk throughput | At least 150 MB/s. |
| Outbound network | All TCP, UDP 4242 (Runpod control plane), UDP 53 (DNS), UDP 123 (time sync), and ICMP echo. |
| Inbound network | TCP 10000 and 20000. |

## Add your machine

<Steps>
<Step title="Start the add compute flow">
In the Runpod console, go to **Resources → External compute** and click **Add compute**.
</Step>
<Step title="Describe the machine">
Enter a name for the machine and click **Next**.

<Note>
Use only lowercase letters, numbers, and hyphens in the machine name. The name can't start or end with a hyphen, and can't contain underscores.
</Note>
</Step>
<Step title="Install the host connector">
Copy the install command shown in the console. Open an SSH session on your machine, paste the command, and run it.

The host connector is a small program that connects your machine to Runpod over an encrypted connection that the machine opens outbound.
</Step>
<Step title="Configure networking and reboot">
In the same onboarding window that shows the install command, copy the networking command and paste it into your SSH session. The command sets your machine's public and private IP addresses. Make sure the ports listed in [Requirements](#requirements) are open, then reboot the machine.
</Step>
<Step title="Wait for verification">
Once the host connector reports in, Runpod configures the machine and runs automated hardware checks. Verification usually takes several minutes, and you don't need to do anything during this step. Follow progress on the **External compute** page.
</Step>
</Steps>

## Machine statuses

Each machine on the **External compute** page shows one of these statuses:

| Status | What it means |
|---|---|
Comment thread
lavanya-gunreddi marked this conversation as resolved.
| Pending install | The machine has been added, but the host connector hasn't reported in yet. |
| Pending verification | The host connector is installed and the machine is registered. Runpod is preparing it for verification. |
| Verification in progress | Runpod is running automated hardware checks. |
| Online | The machine passed verification. Your team can deploy Pods on it. |
| Verification failed | The hardware checks found a problem. See [Troubleshooting](/hybrid-cloud/troubleshooting). |
| Verification timed out | Runpod couldn't connect to the machine to run the hardware checks. See [Troubleshooting](/hybrid-cloud/troubleshooting#verification-timed-out). |
| Offline | Runpod can't reach the machine's host connector. |
| Under maintenance | The machine is temporarily not accepting new Pods. |

Machines you add are visible only to your organization. They aren't listed on the Runpod marketplace or made available to other customers.

## Next steps

- [Deploy a Pod on your external compute](/hybrid-cloud/deploy).
- [Fix verification problems](/hybrid-cloud/troubleshooting).
90 changes: 90 additions & 0 deletions hybrid-cloud/troubleshooting.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
---
title: "Troubleshoot external compute"
sidebarTitle: "Troubleshooting"
description: "Fix common Runpod Hybrid Cloud problems, including failed hardware verification, machines stuck in a status, and missing GPUs in the picker."
---

This page covers the most common problems when you connect and use external compute, and what to do about each.

## Verification failed

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also a "verification timed out" state. When I was testing this was because sshd wasn't listening on all interfaces and therefore Nebula the Runpod control plane couldn't make an SSH connection to the machine. This likely could be generalized as the machine couldn't complete the network connection to the control plane.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a “Verification timed out” section, written generally as Runpod not being able to reach the machine, with the sshd fix as the first step. Also added it to the status table on the setup page.


When a machine's status is **Verification failed**, the automated hardware checks found a problem with the machine or its configuration. Running verification again without changing anything gives the same result. Fix the underlying issue first, then retry verification.

To see which checks failed, open the machine on the **External compute** page and go to the **Verifications** tab.

### DockerFS

The disk mounted at `/var/lib/docker` is too small. The failure message shows the size it found and the size it requires.

To fix this, mount a dedicated disk that meets the Docker disk size in [Requirements](/hybrid-cloud/setup#requirements) at `/var/lib/docker`, then retry verification.

### DockerFstabUUID

The Docker disk is mounted in `/etc/fstab` by its device name, such as `/dev/sdb` or `/dev/md0`, instead of by its UUID. Device names can change when the machine reboots, which can stop the Docker disk from mounting.

To fix this:

1. Find the UUID of the Docker disk:

```bash
sudo blkid
```

2. In `/etc/fstab`, replace the device name on the `/var/lib/docker` line with the UUID. For example:

```text
UUID=DISK_UUID /var/lib/docker xfs defaults 0 0
```

3. Check that the disk mounts correctly, then retry verification:

```bash
sudo mount -a
```

### CUDA Open Variant

The machine has the wrong variant of the NVIDIA driver installed, usually the server variant packages. Runpod requires the open kernel module variant, and the Runpod installer installs the driver for you.

To fix this, remove any NVIDIA driver packages you installed before onboarding, reboot the machine, and retry verification. To see which NVIDIA packages are installed, run:

```bash
dpkg -l | grep nvidia
```

## Verification timed out

**Verification timed out** means Runpod couldn't connect to your machine to run the hardware checks. To fix this:

1. Make sure `sshd` is listening on all network interfaces, not only on localhost.
2. Check that your firewall allows the inbound and outbound traffic listed in [Requirements](/hybrid-cloud/setup#requirements).
3. Retry verification.

## Machine stuck in pending install

**Pending install** means Runpod hasn't heard from the host connector yet. To fix this:

1. Make sure the install command finished without errors. If you aren't sure, run it again.
2. Check that the machine can make outbound connections to the internet.
3. Confirm that you rebooted the machine after you configured networking.

## Machine stuck in verification in progress

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @prestwick it will timeout after 15 min and retry once, then if that fails again, will end in Verification failed state. don't know if we need to expose all the nitty gritty but we have sweepers and retries that will make sure these land in some terminal state after ~30 minutes.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know. Maybe we state here that verification should complete within 30 minutes. Otherwise contact support.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Promptless Verification usually completes within 30 minutes. If a machine stays in Verification in progress for longer than that, contact Runpod support with the machine name, and they'll help you complete verification.


Verification usually completes within 30 minutes. If a machine stays in **Verification in progress** for longer than that, [contact Runpod support](#get-help) with the machine name, and they'll help you complete verification.

## Machine doesn't appear in the GPU picker

If you can't find your machine's GPU when you deploy a Pod:

- Check that the machine's status is **Online** on the **External compute** page. Machines in any other status can't accept Pods.
- Make sure you've selected the organization that owns the machine in the console.
- Check whether every GPU on your machines is already in use. The **External compute** page shows current usage.

## Get help

Runpod Hybrid Cloud is in alpha, and each customer works with the Runpod team during onboarding. If these steps don't fix the problem, contact your Runpod onboarding contact and include the machine name and its current status.

## Next steps

- [Set up external compute](/hybrid-cloud/setup).
- [Deploy Pods on your external compute](/hybrid-cloud/deploy).
Loading