diff --git a/docs.json b/docs.json index 7d76631cf..4176678a4 100644 --- a/docs.json +++ b/docs.json @@ -313,6 +313,15 @@ } ] }, + { + "group": "Hybrid Cloud (Beta)", + "pages": [ + "hybrid-cloud/overview", + "hybrid-cloud/setup", + "hybrid-cloud/deploy", + "hybrid-cloud/troubleshooting" + ] + }, { "group": "Accounts and billing", "pages": [ diff --git a/hybrid-cloud/deploy.mdx b/hybrid-cloud/deploy.mdx new file mode 100644 index 000000000..809b7e9fd --- /dev/null +++ b/hybrid-cloud/deploy.mdx @@ -0,0 +1,39 @@ +--- +title: "Deploy Pods on your external compute" +sidebarTitle: "Deploy to your compute" +description: "Deploy Pods onto the machines your organization has connected to Runpod, using the same console workflow you use for Runpod cloud." +--- + +After a machine reaches **Online**, anyone in your organization can deploy Pods on it. You use the same deploy flow you already use for Runpod cloud. The only difference is where you pick the GPU. + +## Requirements + +- At least one machine with the **Online** status. See [Set up external compute](/hybrid-cloud/setup). +- Membership in the organization that owns the machine, with permission to deploy Pods. See [Roles](/organizations/org-roles). + +## Deploy a Pod + + + + In the Runpod console, make sure your organization is selected, then go to **Pods** and click **Deploy**. + + + In the GPU picker, find the **External compute** section. It lists the GPU types available on your organization's online machines. Select the GPU you want. + + + Configure the rest of the Pod the way you normally would, including the template, storage, and environment variables. Then click **Deploy**. For details on each Pod setting, see [Manage Pods](/pods/manage-pods). + + + +## How placement works + +When you deploy on external compute, Runpod places the Pod on one of your organization's online machines that matches the GPU you selected. Pods deployed on your machines stay within your organization. Other Runpod customers can't deploy to them. + +## Track usage + +The **External compute** page shows how many GPUs on your machines are in use, so you can see available capacity before you deploy. + +## Next steps + +- [Troubleshoot external compute](/hybrid-cloud/troubleshooting). +- [Learn more about Pods](/pods/overview). diff --git a/hybrid-cloud/overview.mdx b/hybrid-cloud/overview.mdx new file mode 100644 index 000000000..6b1d94445 --- /dev/null +++ b/hybrid-cloud/overview.mdx @@ -0,0 +1,68 @@ +--- +title: "Runpod Hybrid Cloud" +sidebarTitle: "Overview" +description: "Run AI workloads on the GPUs you own and on Runpod cloud from one console. Your hardware runs first, and Runpod cloud handles overflow." +--- + +Runpod Hybrid Cloud lets you bring hardware you already own under the same console, CLI, and APIs your team uses on Runpod. Your compute runs first, and Runpod cloud handles overflow when you need it. + +## Requirements + +Runpod Hybrid Cloud is available to organizations only. The Runpod team must enable External Compute for your org account before the feature appears in your console. [Contact us](https://www.runpod.io/hybrid-cloud) to get set up. + +## How it works + + + + In the console, go to **Resources → External compute** and click **Add compute**. Give your machine a name and click **Next**. + + + Copy the install command that appears and paste it into an SSH session on your machine. This installs the Runpod host connector, a small program that connects your machine to Runpod over an encrypted connection. Open the required firewall ports, then reboot. For full instructions, see [Set up external compute](/hybrid-cloud/setup). + + + Runpod runs automated hardware checks in the background. Verification usually takes several minutes, and you don't need to do anything. When verification is complete, the machine's status changes to **Online** and its GPU type appears in the console. + + + Your team deploys Pods exactly as they would for any Runpod machine. An **External Compute** section appears in the GPU picker showing your available hardware. The deploy experience is identical to standard Runpod Pods. See [Deploy Pods on your external compute](/hybrid-cloud/deploy). External compute usage is tracked in the console so you can see how many GPUs are in use at any time. + + + +## What you get + +**One control plane across owned and cloud compute** + +Manage the hardware you own and Runpod cloud as a single estate. No scheduling scripts, access tickets, or per-silo tooling. + +**Consistent developer experience** + +Your team gets the same Runpod console, CLI, and APIs on the hardware you own. SSH still works. + +**Your hardware, your data, your boundary** + +You own the hardware and the data on it. Runpod orchestrates it without taking ownership. Everything you onboard is visible only to your organization. + +**No lock-in** + +The host connector uninstalls cleanly. Your workloads are standard containers. + +## Who it's for + +Runpod Hybrid Cloud is built for teams that own GPU infrastructure and want to put it to work without replatforming. + +- **Research institutions and universities**: Give researchers self-service access to the institution's GPUs, with Runpod cloud absorbing overflow when the queue is full. +- **Enterprise teams**: Unify on-prem clusters, hyperscaler reservations, and Runpod cloud under one control plane with org-scoped governance. +- **AI-native startups**: Focus on your core product instead of building your own abstraction across compute providers. Runpod gives you a provider-agnostic API layer out of the box, so you can run on your own hardware first and burst to Runpod cloud without changing how your team deploys. + +## Security + +The host connector connects your machine to Runpod over an encrypted overlay network. Everything you onboard is scoped to your organization and isn't visible to other customers. For network requirements, see [Set up external compute](/hybrid-cloud/setup#requirements). For security reviews, you get a direct line to Runpod engineers to walk the architecture. + +## Get started + +Runpod Hybrid Cloud is in alpha and available to select customers, and the Runpod team works with you directly to set up your machines. [Request a demo](https://www.runpod.io/hybrid-cloud) to get started. + +## Next steps + +- [Set up external compute](/hybrid-cloud/setup). +- [Deploy Pods on your external compute](/hybrid-cloud/deploy). +- [Troubleshoot external compute](/hybrid-cloud/troubleshooting). diff --git a/hybrid-cloud/setup.mdx b/hybrid-cloud/setup.mdx new file mode 100644 index 000000000..278428d63 --- /dev/null +++ b/hybrid-cloud/setup.mdx @@ -0,0 +1,74 @@ +--- +title: "Set up external compute" +sidebarTitle: "Set up external compute" +description: "Connect a machine you own to Runpod Hybrid Cloud by installing the host connector, then track it through verification until it's online." +--- + +This guide walks you through connecting a machine you own to Runpod. After you add it and install the host connector, Runpod verifies the hardware automatically. The machine is then online and available to your organization. + +## Requirements + +- An [organization](/organizations/orgs-overview) with external compute enabled. During alpha, the Runpod team enables this for you. +- An organization role that allows you to add compute. See [Roles](/organizations/org-roles). +- A single machine with at least one GPU. Multi-node machines connected by InfiniBand or a similar fabric aren't supported yet. +- SSH access to the machine, with permission to install software. + +Your machine must also meet these hardware and network requirements: + +| Requirement | Value | +|---|---| +| Operating system | Ubuntu 24.04, with a kernel from 6.8 through 7.0.x. | +| GPU drivers | None preinstalled. The Runpod installer manages GPU drivers. | +| Root disk | At least 200 GB. | +| Docker disk | A dedicated disk of at least 600 GB. | +| Disk throughput | At least 150 MB/s. | +| Outbound network | All TCP, UDP 4242 (Runpod control plane), UDP 53 (DNS), UDP 123 (time sync), and ICMP echo. | +| Inbound network | TCP 10000 and 20000. | + +## Add your machine + + + + In the Runpod console, go to **Resources → External compute** and click **Add compute**. + + + Enter a name for the machine and click **Next**. + + + Use only lowercase letters, numbers, and hyphens in the machine name. The name can't start or end with a hyphen, and can't contain underscores. + + + + Copy the install command shown in the console. Open an SSH session on your machine, paste the command, and run it. + + The host connector is a small program that connects your machine to Runpod over an encrypted connection that the machine opens outbound. + + + In the same onboarding window that shows the install command, copy the networking command and paste it into your SSH session. The command sets your machine's public and private IP addresses. Make sure the ports listed in [Requirements](#requirements) are open, then reboot the machine. + + + Once the host connector reports in, Runpod configures the machine and runs automated hardware checks. Verification usually takes several minutes, and you don't need to do anything during this step. Follow progress on the **External compute** page. + + + +## Machine statuses + +Each machine on the **External compute** page shows one of these statuses: + +| Status | What it means | +|---|---| +| Pending install | The machine has been added, but the host connector hasn't reported in yet. | +| Pending verification | The host connector is installed and the machine is registered. Runpod is preparing it for verification. | +| Verification in progress | Runpod is running automated hardware checks. | +| Online | The machine passed verification. Your team can deploy Pods on it. | +| Verification failed | The hardware checks found a problem. See [Troubleshooting](/hybrid-cloud/troubleshooting). | +| Verification timed out | Runpod couldn't connect to the machine to run the hardware checks. See [Troubleshooting](/hybrid-cloud/troubleshooting#verification-timed-out). | +| Offline | Runpod can't reach the machine's host connector. | +| Under maintenance | The machine is temporarily not accepting new Pods. | + +Machines you add are visible only to your organization. They aren't listed on the Runpod marketplace or made available to other customers. + +## Next steps + +- [Deploy a Pod on your external compute](/hybrid-cloud/deploy). +- [Fix verification problems](/hybrid-cloud/troubleshooting). diff --git a/hybrid-cloud/troubleshooting.mdx b/hybrid-cloud/troubleshooting.mdx new file mode 100644 index 000000000..cac420766 --- /dev/null +++ b/hybrid-cloud/troubleshooting.mdx @@ -0,0 +1,90 @@ +--- +title: "Troubleshoot external compute" +sidebarTitle: "Troubleshooting" +description: "Fix common Runpod Hybrid Cloud problems, including failed hardware verification, machines stuck in a status, and missing GPUs in the picker." +--- + +This page covers the most common problems when you connect and use external compute, and what to do about each. + +## Verification failed + +When a machine's status is **Verification failed**, the automated hardware checks found a problem with the machine or its configuration. Running verification again without changing anything gives the same result. Fix the underlying issue first, then retry verification. + +To see which checks failed, open the machine on the **External compute** page and go to the **Verifications** tab. + +### DockerFS + +The disk mounted at `/var/lib/docker` is too small. The failure message shows the size it found and the size it requires. + +To fix this, mount a dedicated disk that meets the Docker disk size in [Requirements](/hybrid-cloud/setup#requirements) at `/var/lib/docker`, then retry verification. + +### DockerFstabUUID + +The Docker disk is mounted in `/etc/fstab` by its device name, such as `/dev/sdb` or `/dev/md0`, instead of by its UUID. Device names can change when the machine reboots, which can stop the Docker disk from mounting. + +To fix this: + +1. Find the UUID of the Docker disk: + + ```bash + sudo blkid + ``` + +2. In `/etc/fstab`, replace the device name on the `/var/lib/docker` line with the UUID. For example: + + ```text + UUID=DISK_UUID /var/lib/docker xfs defaults 0 0 + ``` + +3. Check that the disk mounts correctly, then retry verification: + + ```bash + sudo mount -a + ``` + +### CUDA Open Variant + +The machine has the wrong variant of the NVIDIA driver installed, usually the server variant packages. Runpod requires the open kernel module variant, and the Runpod installer installs the driver for you. + +To fix this, remove any NVIDIA driver packages you installed before onboarding, reboot the machine, and retry verification. To see which NVIDIA packages are installed, run: + +```bash +dpkg -l | grep nvidia +``` + +## Verification timed out + +**Verification timed out** means Runpod couldn't connect to your machine to run the hardware checks. To fix this: + +1. Make sure `sshd` is listening on all network interfaces, not only on localhost. +2. Check that your firewall allows the inbound and outbound traffic listed in [Requirements](/hybrid-cloud/setup#requirements). +3. Retry verification. + +## Machine stuck in pending install + +**Pending install** means Runpod hasn't heard from the host connector yet. To fix this: + +1. Make sure the install command finished without errors. If you aren't sure, run it again. +2. Check that the machine can make outbound connections to the internet. +3. Confirm that you rebooted the machine after you configured networking. + +## Machine stuck in verification in progress + +Verification usually completes within 30 minutes. If a machine stays in **Verification in progress** for longer than that, [contact Runpod support](#get-help) with the machine name, and they'll help you complete verification. + +## Machine doesn't appear in the GPU picker + +If you can't find your machine's GPU when you deploy a Pod: + +- Check that the machine's status is **Online** on the **External compute** page. Machines in any other status can't accept Pods. +- Make sure you've selected the organization that owns the machine in the console. +- Check whether every GPU on your machines is already in use. The **External compute** page shows current usage. + +## Get help + +Runpod Hybrid Cloud is in alpha, and each customer works with the Runpod team during onboarding. If these steps don't fix the problem, contact your Runpod onboarding contact and include the machine name and its current status. + +## Next steps + +- [Set up external compute](/hybrid-cloud/setup). +- [Deploy Pods on your external compute](/hybrid-cloud/deploy).