Run locally / Your cloud account
Run models in your cloud account
Create a GPU VM, connect over SSH, run your model, then delete the machine.
Commands have not been run by us end-to-end. You need a cloud account, billing, its CLI and GPU quota in your chosen region. GPU quota requests and capacity checks may be needed before launch. You pay the provider directly.
GPU memory needed: 70 GB. Memory follows the recipe’s recorded runtime requirements; see the model setup page for verification status.
Model setup and licence →Your cloud account
AWS
Choose by GPU memory first: one GPU must fit the model. If none fits, use a multi-GPU instance with tensor parallelism. Also check system RAM, disk and runtime compatibility. Figures below describe the hardware, not a tested model setup.
| Instance type | GPU | Memory per GPU | Total GPU memory |
|---|---|---|---|
| g6.xlarge | 1 × L4 | 24 GB | 24 GB |
| g6e.xlarge | 1 × L40S | 48 GB | 48 GB |
| p5.4xlarge | 1 × H100 | 80 GB | 80 GB |
| p5.48xlarge | 8 × H100 (80 GB each) | 80 GB | 640 GB |
| p4de.24xlarge | 8 × A100 (80 GB each) | 80 GB | 640 GB |
Provider specifications (checked 2 November 2026): GPU specifications 1 · GPU specifications 2 · GPU specifications 3 · GPU specifications 4
For basal-omni-1.0-ultra: 70 GB needed → p5.4xlarge (1 × H100; 80 GB per GPU).
Use a subnet with an internet gateway. The security group permits SSH only from your IP. Multi-GPU P instances need tensor parallelism, a compatible attention-head count and enough disk space.
1. Create a VM and connect
Replace the example IP, key and network values first. The example creates a 200 GB boot disk; increase it if your model needs more space.
INSTANCE_TYPE=p5.4xlarge
: "${INSTANCE_TYPE:?Choose an instance by GPU memory first}"
[ "$INSTANCE_TYPE" != REPLACE_WITH_INSTANCE_FROM_TABLE ] || { echo 'Choose INSTANCE_TYPE from the table first'; exit 1; }
# On your computer: AWS CLI configured; choose a region with GPU capacity.
export AWS_DEFAULT_REGION=eu-central-1
DM_KEY=your-existing-ec2-key-name
DM_KEY_FILE="$HOME/.ssh/your-key.pem"
DM_SUBNET=subnet-REPLACE_ME
DM_VPC=vpc-REPLACE_ME
DM_MY_IP=203.0.113.1/32 # Replace with your public IP /32.
DM_AMI=$(aws ec2 describe-images --owners amazon --filters 'Name=name,Values=Deep Learning Base OSS Nvidia Driver GPU AMI (Ubuntu 24.04)*' 'Name=architecture,Values=x86_64' 'Name=state,Values=available' --query 'sort_by(Images,&CreationDate)[-1].ImageId' --output text)
[ "$DM_AMI" != None ] || { echo 'No matching AMI in this region'; exit 1; }
DM_SG=$(aws ec2 create-security-group --group-name dm-local-guide --description 'SSH from my IP only' --vpc-id "$DM_VPC" --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id "$DM_SG" --protocol tcp --port 22 --cidr "$DM_MY_IP"
DM_VM=$(aws ec2 run-instances --image-id "$DM_AMI" --instance-type "$INSTANCE_TYPE" --key-name "$DM_KEY" --subnet-id "$DM_SUBNET" --security-group-ids "$DM_SG" --associate-public-ip-address --count 1 --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":200,"VolumeType":"gp3","DeleteOnTermination":true}}]' --query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$DM_VM"
DM_IP=$(aws ec2 describe-instances --instance-ids "$DM_VM" --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
ssh -i "$DM_KEY_FILE" ubuntu@"$DM_IP"Provider image and setup documentation
2. Install and run inside the SSH session
Check the driver with nvidia-smi. Cloud GPU images can ship an older driver: recipes using CUDA 13 need an R580 or newer compatible NVIDIA driver. Check the NVIDIA driver requirements and your provider’s image instructions before installing. The CUDA version shown by nvidia-smi is the driver’s supported version.
For a model with a supported installer recipe, plan checks hardware fit; verify the driver separately, then install:
curl -fsSL https://decisionmodels.io/local/install.sh | sh && export PATH="$HOME/.local/bin:$PATH" # Replace MODEL_SLUG with a supported model from /local. dm-local plan MODEL_SLUG dm-local install MODEL_SLUG
This model needs the author’s serving runtime for typed decisions. Follow its local setup page and model card; a stock vLLM command is not offered.
Do not open ports 8000 or 8484 to the internet. Review any custom model code before running it.
3. Reach localhost through a tunnel
In a second terminal on your computer (keep the model running):
ssh -i "$DM_KEY_FILE" -N -L 8000:127.0.0.1:8000 ubuntu@"$DM_IP"
Then connect to http://127.0.0.1:8000. For the installer endpoint, replace both occurrences of 8000 with 8484.
4. Delete the resources
Exit SSH, then run the following on your computer. Check the provider console for remaining disks or addresses; stopped VMs can still incur storage charges.
# On your computer, after leaving SSH: aws ec2 terminate-instances --instance-ids "$DM_VM" aws ec2 wait instance-terminated --instance-ids "$DM_VM" aws ec2 delete-security-group --group-id "$DM_SG"
Your cloud account
Azure
Choose by GPU memory first: one GPU must fit the model. If none fits, use a multi-GPU instance with tensor parallelism. Also check system RAM, disk and runtime compatibility. Figures below describe the hardware, not a tested model setup.
| Instance type | GPU | Memory per GPU | Total GPU memory |
|---|---|---|---|
| Standard_NV36ads_A10_v5 | 1 × A10 | 24 GB | 24 GB |
| Standard_NC24ads_A100_v4 | 1 × A100 | 80 GB | 80 GB |
| Standard_NC40ads_H100_v5 | 1 × H100 NVL | 94 GB | 94 GB |
| Standard_NC80adis_H100_v5 | 2 × H100 NVL (94 GB each) | 94 GB | 188 GB |
Provider specifications (checked 2 November 2026): GPU specifications 1 · GPU specifications 2 · GPU specifications 3
For basal-omni-1.0-ultra: 70 GB needed → Standard_NC24ads_A100_v4 (1 × A100; 80 GB per GPU).
Check image availability and NVIDIA driver compatibility for your chosen size. NVads A10 sizes below NV36 have fractional GPUs (4–12 GB), not a full 24 GB. New A100 capacity is limited; H100 may be available instead.
1. Create a VM and connect
Replace the example IP, key and network values first. The example creates a 200 GB boot disk; increase it if your model needs more space.
INSTANCE_TYPE=Standard_NC24ads_A100_v4
: "${INSTANCE_TYPE:?Choose an instance by GPU memory first}"
[ "$INSTANCE_TYPE" != REPLACE_WITH_INSTANCE_FROM_TABLE ] || { echo 'Choose INSTANCE_TYPE from the table first'; exit 1; }
# On your computer: az login first. Pick a region with GPU quota.
DM_REGION=westeurope
DM_GROUP=dm-local-guide # New resource group dedicated to this guide only.
DM_MY_IP=203.0.113.1/32 # Replace with your public IP /32.
az group create --name "$DM_GROUP" --location "$DM_REGION"
# Review available NVIDIA-enabled Ubuntu HPC images in your region.
az vm image list --location "$DM_REGION" --publisher microsoft-dsvm --offer ubuntu-hpc --sku 2204 --all --output table
az network nsg create --resource-group "$DM_GROUP" --name dm-ssh
az network nsg rule create --resource-group "$DM_GROUP" --nsg-name dm-ssh --name MySSH --priority 100 --source-address-prefixes "$DM_MY_IP" --destination-port-ranges 22 --protocol Tcp --access Allow
az vm create --resource-group "$DM_GROUP" --name dm-model --image microsoft-dsvm:ubuntu-hpc:2204:latest --size "$INSTANCE_TYPE" --admin-username dmuser --generate-ssh-keys --nsg dm-ssh --nsg-rule NONE --os-disk-size-gb 200 --public-ip-sku Standard
DM_IP=$(az vm show -g "$DM_GROUP" -n dm-model -d --query publicIps -o tsv)
ssh dmuser@"$DM_IP"Provider image and setup documentation
2. Install and run inside the SSH session
Check the driver with nvidia-smi. Cloud GPU images can ship an older driver: recipes using CUDA 13 need an R580 or newer compatible NVIDIA driver. Check the NVIDIA driver requirements and your provider’s image instructions before installing. The CUDA version shown by nvidia-smi is the driver’s supported version.
For a model with a supported installer recipe, plan checks hardware fit; verify the driver separately, then install:
curl -fsSL https://decisionmodels.io/local/install.sh | sh && export PATH="$HOME/.local/bin:$PATH" # Replace MODEL_SLUG with a supported model from /local. dm-local plan MODEL_SLUG dm-local install MODEL_SLUG
This model needs the author’s serving runtime for typed decisions. Follow its local setup page and model card; a stock vLLM command is not offered.
Do not open ports 8000 or 8484 to the internet. Review any custom model code before running it.
3. Reach localhost through a tunnel
In a second terminal on your computer (keep the model running):
ssh -N -L 8000:127.0.0.1:8000 dmuser@"$DM_IP"
Then connect to http://127.0.0.1:8000. For the installer endpoint, replace both occurrences of 8000 with 8484.
4. Delete the resources
Exit SSH, then run the following on your computer. Check the provider console for remaining disks or addresses; stopped VMs can still incur storage charges.
# Deletes the dedicated group and all its VM, disk and network resources. az group delete --name "$DM_GROUP" --yes
Your cloud account
Google Cloud
Choose by GPU memory first: one GPU must fit the model. If none fits, use a multi-GPU instance with tensor parallelism. Also check system RAM, disk and runtime compatibility. Figures below describe the hardware, not a tested model setup.
| Instance type | GPU | Memory per GPU | Total GPU memory |
|---|---|---|---|
| g2-standard-8 | 1 × L4 | 24 GB | 24 GB |
| a2-ultragpu-1g | 1 × A100 | 80 GB | 80 GB |
| a2-ultragpu-2g | 2 × A100 (80 GB each) | 80 GB | 160 GB |
| a3-highgpu-1g | 1 × H100 (Spot or Flex-start only) | 80 GB | 80 GB |
Provider specifications (checked 2 November 2026): GPU specifications 1
For basal-omni-1.0-ultra: 70 GB needed → a2-ultragpu-1g (1 × A100; 80 GB per GPU).
A3 single-GPU H100 requires Spot or Flex-start; it is not an ordinary on-demand VM. A2 Ultra includes automatically attached local SSD storage, which is lost when the VM is deleted. IAP access requires IAM permissions and the IAP API. The example creates a private network and Cloud NAT for outbound downloads; NAT and its public IP also incur charges.
1. Create a VM and connect
Replace the example IP, key and network values first. The example creates a 200 GB boot disk; increase it if your model needs more space.
INSTANCE_TYPE=a2-ultragpu-1g
: "${INSTANCE_TYPE:?Choose an instance by GPU memory first}"
[ "$INSTANCE_TYPE" != REPLACE_WITH_INSTANCE_FROM_TABLE ] || { echo 'Choose INSTANCE_TYPE from the table first'; exit 1; }
# On your computer: gcloud auth login and select your project first.
DM_REGION=us-central1
DM_ZONE=us-central1-a # Check the chosen GPU is available in this zone.
# Dedicated private network; Cloud NAT allows package and weights downloads.
gcloud compute networks create dm-guide-net --subnet-mode=custom
gcloud compute networks subnets create dm-guide-subnet --network dm-guide-net --region "$DM_REGION" --range 10.42.0.0/24
gcloud compute routers create dm-guide-router --network dm-guide-net --region "$DM_REGION"
gcloud compute routers nats create dm-guide-nat --router dm-guide-router --region "$DM_REGION" --nat-all-subnet-ip-ranges --auto-allocate-nat-external-ips
# Uses an IAP SSH tunnel; grant yourself IAP tunnel + SSH permissions first.
gcloud compute firewall-rules create dm-guide-iap-ssh --network dm-guide-net --allow tcp:22 --source-ranges 35.235.240.0/20 --target-tags dm-guide
gcloud compute instances create dm-model --zone "$DM_ZONE" --machine-type "$INSTANCE_TYPE" --image-project deeplearning-platform-release --image-family common-cu129-ubuntu-2204-nvidia-580 --boot-disk-size 200GB --maintenance-policy TERMINATE --metadata install-nvidia-driver=True --tags dm-guide --subnet dm-guide-subnet --no-address
gcloud compute ssh dm-model --zone "$DM_ZONE" --tunnel-through-iapProvider image and setup documentation
2. Install and run inside the SSH session
Check the driver with nvidia-smi. Cloud GPU images can ship an older driver: recipes using CUDA 13 need an R580 or newer compatible NVIDIA driver. Check the NVIDIA driver requirements and your provider’s image instructions before installing. The CUDA version shown by nvidia-smi is the driver’s supported version.
For a model with a supported installer recipe, plan checks hardware fit; verify the driver separately, then install:
curl -fsSL https://decisionmodels.io/local/install.sh | sh && export PATH="$HOME/.local/bin:$PATH" # Replace MODEL_SLUG with a supported model from /local. dm-local plan MODEL_SLUG dm-local install MODEL_SLUG
This model needs the author’s serving runtime for typed decisions. Follow its local setup page and model card; a stock vLLM command is not offered.
Do not open ports 8000 or 8484 to the internet. Review any custom model code before running it.
3. Reach localhost through a tunnel
In a second terminal on your computer (keep the model running):
gcloud compute ssh dm-model --zone "$DM_ZONE" --tunnel-through-iap -- -N -L 8000:127.0.0.1:8000
Then connect to http://127.0.0.1:8000. For the installer endpoint, replace both occurrences of 8000 with 8484.
4. Delete the resources
Exit SSH, then run the following on your computer. Check the provider console for remaining disks or addresses; stopped VMs can still incur storage charges.
# On your computer; also deletes the auto-delete boot disk. gcloud compute instances delete dm-model --zone "$DM_ZONE" --quiet gcloud compute firewall-rules delete dm-guide-iap-ssh --quiet gcloud compute routers delete dm-guide-router --region "$DM_REGION" --quiet gcloud compute networks subnets delete dm-guide-subnet --region "$DM_REGION" --quiet gcloud compute networks delete dm-guide-net --quiet