Your cloud provider can read your VM’s memory. Full stop. I don’t care what their marketing page says. If you’re running on standard hardware with standard virtualization, the hypervisor has god-mode access to every byte of your guest’s RAM. Your encryption-at-rest doesn’t matter. Your TLS doesn’t matter. The moment your data is in memory, it’s naked and the hypervisor is watching.

This is the problem AMD set out to solve with SEV-SNP, and after deploying this stuff in production across multiple clusters, I’m going to walk you through exactly how it works — all five generations of it — because understanding the evolution explains why the final product looks the way it does.

Level 1: SME — The “Good Enough” That Wasn’t

AMD started with SME (Secure Memory Encryption) in 2017 with EPYC 7001 (Naples). The idea was simple: encrypt all physical memory with AES-128-XTS using a per-CPU key. The encryption engine sits in the memory controller, transparent to the CPU pipeline. The OS marks pages as encrypted by setting the C-bit in the page table entry, and the Memory Encryption Engine (MEE) handles the rest.

# Check if your CPU supports SEV
grep -o 'sev\|sev_es\|sev_snp' /proc/cpuinfo | sort -u

# Check kernel support
ls /sys/module/kvm_amd/parameters/
cat /sys/module/kvm_amd/parameters/sev
# 1 = enabled, 0 = disabled

# Check SEV-SNP specifically
cat /sys/module/kvm_amd/parameters/sev_snp

SME is fine for what it is — it protects against cold boot attacks and physical memory extraction. If someone yanks the DIMMs out of your server, they get ciphertext. Cool. But there’s one massive problem: the hypervisor still has the key. It’s a per-CPU key, not a per-VM key. The hypervisor can read and write guest memory freely. SME is a seatbelt in a car where the passenger is also the attacker.

~2-5% overhead. Not bad. Not enough security either.

Level 2: SEV — Per-VM Keys, Per-VM Problems

SEV (Secure Encrypted Virtualization) was the real starting gun. Each VM gets its own AES-128-XTS key, generated and managed by the AMD Secure Processor (AMD-SP) — a dedicated security coprocessor sitting on-die, running its own firmware (TeeSVC). The hypervisor no longer has the keys. It can’t read guest memory directly.

Sounds great, right? Except the hypervisor is still in charge of the page tables. It can swap encrypted pages, remap them, play timing games. It can’t read the data, but it can absolutely mess with which data the guest sees. This is the classic “confused deputy” problem, and in security terms, it means you have confidentiality but not integrity.

Also? vCPU registers are completely visible during VMEXIT. The hypervisor can see everything in your registers every time the guest traps out to the hypervisor. For a workload that does any I/O (so, all of them), this is a real problem.

Level 3: SEV-ES — Now We’re Getting Somewhere

SEV-ES (Encrypted State) fixed the register leak. Now when a VMEXIT happens, the vCPU registers get encrypted into the VMSA (Virtual Machine Save Area). The hypervisor sees garbage.

The other big addition was the GHCB — the Guest-Host Communication Block. This is a 4KB page that serves as the only sanctioned channel between guest and hypervisor. The guest triggers a VMGEXIT (via rep; vmmcall), which fires a #VC exception (vector 0x2D), and the two sides negotiate through the GHCB.

Each VMGEXIT costs roughly 1-5 microseconds. That sounds tiny until you’re doing heavy I/O and hitting 50K-200K exits per second. During boot, you can spike to 500K+/s. The overhead lands around 5-10% for most workloads.

But here’s the thing that kept me up at night: SEV-ES still has no memory integrity protection. The hypervisor can still play games with page tables. It can’t read the data (encrypted), and it can’t see the registers (encrypted), but it can absolutely trick the guest into operating on the wrong memory. That’s a real attack, not a theoretical one.

Level 4: SEV-SNP — The One That Actually Works

SEV-SNP (Secure Nested Paging) is where AMD finally closed the loop. The headline feature is the RMP — the Reverse Map Table — and it changes everything.

The RMP is a hardware-managed table with a 64-bit entry for every 4KB page of physical memory in the system. The bit layout looks like this:

  • Bits [11:10] — page type (Unallocated=0, Private=1, Shared=2)
  • Bits [9:8] — owner/VMPL
  • Bit [7] — memory type
  • Bit [3] — valid
  • Bits [2:0] — permissions

The hypervisor can no longer silently remap pages. Every page has an owner tracked in hardware. If the hypervisor tries to map a page belonging to VM-A into VM-B’s page tables, the CPU checks the RMP and says “no.” The integrity protection is enforced at the silicon level, not in software that the hypervisor controls.

Key functions you’ll encounter in the kernel source: pvalidate() to set page validation state, rmpadjust() to modify RMP permissions, and snp_set_memory_private()/snp_set_memory_shared() for the kernel to manage page ownership.

The performance numbers surprised me. SEV-SNP actually runs faster than SEV-ES in many workloads — 1-5% overhead vs 5-10%. AMD optimized the hot path. The RMP lookup is pipelined in the memory controller alongside the MEE encryption, so you’re not paying double for the integrity check.

Level 5: SEV-TIO — Extending Trust to the PCIe Bus

SEV-SNP protects memory. That’s great. But your VM doesn’t just sit in memory — it talks to devices. NICs, GPUs, NVMe drives, accelerators. And here’s the problem: with standard PCIe device passthrough (VFIO), the hypervisor still controls the I/O path. It can intercept DMA, tamper with MMIO, or mount interposer attacks on the PCIe fabric. Your memory is encrypted, but your I/O is naked.

AMD’s answer is SEV-TIO (Trusted I/O), and it’s the next evolution in the stack. Available on EPYC 9005 (Turin) only, SEV-TIO extends the TEE boundary to include PCIe devices using the PCI-SIG’s TDISP (TEE Device Interface Security Protocol), defined in PCIe 6.0+.

The acronym soup is thick here, so let me break it down:

  • TDISP — TEE Device Interface Security Protocol. The PCIe standard that defines how a VM establishes trust with a device, locks its configuration, and isolates it from the host.
  • IDE — Integrity and Data Encryption. AES-GCM encryption of the PCIe link itself. Not just memory — the actual wire traffic between CPU and device.
  • SPDM — Security Protocol and Data Model. How the device authenticates itself using certificate chains. Think TLS, but for PCIe.
  • DOE — Data Object Exchange. The transport mechanism in PCIe config space for SPDM and TDISP messages.
  • TSM — TEE Security Manager. On AMD, this is the AMD-SP (PSP) itself. It coordinates all the key provisioning and device attestation.

The workflow looks like this:

  1. Host boots, loads the CCP driver which registers as a TSM
  2. TDISP-capable devices get sysfs nodes under /sys/bus/pci/devices/*/tsm/
  3. IDE encryption is enabled via the TSM connect interface
  4. Device is hotplugged to the VM via VFIO
  5. The AMD-SP authenticates the device using SPDM certificate chains
  6. TDISP binds the device to the specific VM (TDI_BIND)
  7. The device can now DMA directly to the guest’s private (encrypted) memory — no bounce buffers, no SWIOTLB hacks

That last point is the big win. Without SEV-TIO, a confidential VM doing I/O needs bounce buffers — the device DMAs to shared (unencrypted) memory, then the guest copies it to private memory. That’s a performance tax on every single I/O operation. With SEV-TIO, the device writes directly to encrypted memory. The DMA path is authenticated end-to-end.

Linux support landed in 6.19 (December 2025) with the initial PCIe link encryption infrastructure. The upstreaming is happening in phases:

  1. Phase 1 (6.19): IDE — host-side PCIe link encryption
  2. Phase 2 (6.20/7.0): TDISP — lock/accept flow, interface reports
  3. Phase 3: Secure MMIO + DMA via IOMMUFD and KVM changes
  4. Phase 4: Full device attestation (certificates, measurements, reports)

Intel’s equivalent is TDX Connect, and ARM has CCA — but AMD is first to mainline. The TSM infrastructure is being built collaboratively across all three architectures, which is actually a nice change from the usual vendor lock-in games.

Device support is still early. NVIDIA’s Blackwell GPUs reportedly support TDISP, and NIC/storage vendors are working on it. But this is the future of confidential I/O — if your threat model includes the hypervisor, you can’t trust it to mediate your device traffic either.

The Key Hierarchy (It’s Turtles All the Way Down)

The key hierarchy is where things get properly paranoid:

AMD Root Key (ARK) — 256-bit RSA, lives in AMD's HSM
  └── AMD Signing Key (ASK) — 256-bit ECDSA
        └── Platform Certificate (PCERT)
              └── Platform Secret — 48 bytes, in PSP secure storage
                    └── VMPCK0-3 — derived via HKDF-SHA256
                          └── Per-VM Key — derived from VMPCK0

There are four VMPCK keys, each with a specific job:

  • VMPCK0 — encryption and attestation
  • VMPCK1 — migration
  • VMPCK2 — state save/restore
  • VMPCK3 — firmware measurement

The per-VM key is derived as: HKDF-Expand(SHA-256, VMPCK0, "SEV-SNP VM Key" || guest_id || launch_nonce, 16). Every VM gets a unique key. The hypervisor never sees any of these keys. The AMD-SP generates them, uses them, and the hypervisor is just a taxi driver that doesn’t get to look in the trunk.

There’s also a secrets page at physical address 0xFED40000, encrypted with VMPCK2, where the guest can retrieve credentials injected at launch. This is how you bootstrap trust inside the VM without the hypervisor being able to intercept.

Attestation: Proving You Are Who You Say You Are

This is the part that makes SEV-SNP actually useful for real security, not just theater. A guest VM can request an attestation report from the AMD-SP via VMGEXIT. The AMD-SP signs the report with ECDSA P-384, and it contains:

  • The VM policy
  • A measurement (HMAC-SHA256 of the launch digest)
  • TCB version
  • Guest-provided data (up to 64 bytes)

Verification chain: ARK → ASK → PCERT → SP Certificate → Report signature. You can verify this against AMD’s attestation service (AAS) via their REST API, or offline if you’ve cached the certificate chain.

# Install sevctl (Rust-based CLI tool)
cargo install sevctl

# Get the platform certificate chain
sevctl export /tmp/cert-chain.cert

# Generate a launch measurement
sevctl session --name my-vm --policy 0x00030001 > /tmp/session.json

# Verify an attestation report
sevctl verify /tmp/attestation-report.bin

If you’re building this into your own infrastructure tooling (and you should be), the sev Rust crate gives you programmatic access:

use sev::firmware::guest::AttestationReport;
use sev::certs::Chain;

fn verify_attestation(report_bytes: &[u8], chain: &Chain) -> Result<(), Box<dyn std::error::Error>> {
    let report = AttestationReport::try_from(report_bytes)?;

    // Verify the signature chain: ARK → ASK → PCERT → Report
    chain.verify()?;

    // Check the report signature against the platform certificate
    report.verify_signature(&chain)?;

    // Verify the measurement matches expected value
    let expected_measurement = get_expected_measurement();
    if report.measurement() != expected_measurement {
        return Err("Measurement mismatch — VM may be tampered".into());
    }

    // Check TCB version is current
    if report.tcb_version() < MINIMUM_TCB_VERSION {
        return Err("TCB version too old — firmware needs update".into());
    }

    Ok(())
}

This is the real value proposition. You can now have a remote party verify that your VM is running the exact code you claim, on genuine AMD hardware, with a current firmware version. No trust in the cloud provider required. That’s not a small thing.

Policy: Choose Wisely, You Can’t Take It Back

SNP policies are bitmasks, and they’re irreversible after launch. Let me say that again because people keep getting burned: once you launch a VM with a policy, you cannot change it. If you forget to set the migration bit, that VM can never be live migrated. Ever.

#!/usr/bin/env python3
"""Decode AMD SEV-SNP policy flags."""

def decode_snp_policy(policy: int) -> dict:
    """Decode SEV-SNP policy bitmask into human-readable flags."""
    flags = {
        "NO_DEBUG":        bool(policy & (1 << 0)),
        "ES":              bool(policy & (1 << 1)),
        "SMT":             bool(policy & (1 << 2)),
        "MIGRATE_MA":      bool(policy & (1 << 18)),
        "DEBUG_SWAP":      bool(policy & (1 << 3)),
        "PREVENT_HOST_IBS": bool(policy & (1 << 4)),
        "BTB_ISOLATION":   bool(policy & (1 << 5)),
        "SMT_REQUIRED":    bool(policy & (1 << 6)),
    }
    return flags

# Common policies
policies = {
    0x00010001: "Standard (no migration)",
    0x00030001: "Migration enabled",
    0x00010003: "Maximum security (SMT restricted)",
}

for val, desc in policies.items():
    print(f"\nPolicy 0x{val:08X} = {desc}")
    for flag, set in decode_snp_policy(val).items():
        if set:
            print(f"  ✓ {flag}")

The three policies you’ll actually use:

  • 0x00010001 — Standard. NO_DEBUG + ES + SMT allowed. No migration. Good for static workloads.
  • 0x00030001 — Migration enabled. Adds the MIGRATE_MA bit. Use this if you ever want to live migrate.
  • 0x00010003 — Maximum security. Restricts SMT, enables BTB isolation. For when you’re truly paranoid (and you should be).

My recommendation: default to 0x00030001 unless you have a specific reason not to. Forgetting migration and then needing it is a “rebuild the VM from scratch” situation.

Launching the Thing

Here’s the QEMU incantation:

qemu-system-x86_64 \
  -enable-kvm \
  -cpu EPYC-v4 \
  -machine q35,confidential-guest-support=sev-snp0 \
  -object sev-snp-guest,id=sev-snp0,cbitpos=51,reduced-phys-bits=1,policy=0x00030001 \
  -smp 8,sockets=1,cores=8,threads=1 \
  -m 16G \
  -bios /usr/share/OVMF/OVMF_CODE.fd \
  -drive file=vm-disk.qcow2,format=qcow2,if=virtio \
  -netdev user,id=net0 -device virtio-net-pci,netdev=net0 \
  -nographic

A few things to note: cbitpos=51 is the C-bit position in the page table entry — this comes from CPUID and is the same for Genoa and Turin. reduced-phys-bits=1 reduces the physical address space by one bit (the stolen bit becomes the encryption bit). And critically, you need the amdsev variant of OVMF, not the standard one. I’ve watched people debug “why won’t my VM boot” for hours because they missed this.

For libvirt users:

<domain type='kvm'>
  <name>sev-snp-vm</name>
  <memory unit='GiB'>16</memory>
  <vcpu placement='static'>8</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-8.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
  </os>
  <features>
    <acpi/>
    <apic/>
    <confidential>
      <type>sev-snp</type>
      <cbitpos>51</cbitpos>
      <reducedPhysBits>1</reducedPhysBits>
      <policy>0x00030001</policy>
    </confidential>
  </features>
  <cpu mode='host-model'>
    <model fallback='forbid'/>
  </cpu>
</domain>

Libvirt 9.0+ required. For Turin, bump to 11.0+ and QEMU 9.0+.

What You Actually Lose

I’m not going to sugarcoat this. Running SEV-SNP means giving up some things that ops teams take for granted:

  • No nested virtualization. This is a hard hardware limit. You cannot run VMs inside SEV-SNP VMs. If your architecture depends on nested virt, you need to rethink it.
  • No KSM. Kernel Samepage Merging is dead. Every VM gets its own copy of every page. Memory density takes a hit.
  • No memory introspection. Valgrind, gdb attach, eBPF tracing from the host — all gone. You can’t look inside from outside. That’s the whole point, but it also means your debugging story changes completely.
  • No debug registers (with NO_DEBUG policy).
  • No CPU/memory hotplug.
  • No memory overcommit. You need to actually have the physical memory. No more lying to VMs about available RAM.
  • GPU passthrough doesn’t survive migration. If you’re doing ML workloads with GPU passthrough, plan accordingly. (SEV-TIO on Turin is working to address the device trust side of this, but migration with device re-binding is still an unsolved problem.)

Performance: The Real Numbers

The MEE (Memory Encryption Engine) is the bottleneck, and it’s per-socket, not per-core. Genoa pushes ~300 GB/s per socket, Turin bumps that to ~350 GB/s. Under heavy memory pressure, you’ll hit this ceiling.

Real-world numbers from my experience and the benchmarks:

  • CPU-bound (SPEC): 0-2% overhead. Basically free.
  • Memory-bound (STREAM): 5-15%. This is where it hurts. The MEE bandwidth limit is real.
  • I/O-bound: ~0%. The I/O path is already slow relative to encryption.
  • Kernel/syscall heavy: 1-5%. VMGEXIT overhead adds up.
  • Mixed workloads: 1-3%. This is what most people will see.

Two things that make a real difference: use 2MB hugepages (reduces overhead ~5% because fewer RMP entries to check) and pin your VMs to NUMA nodes. Cross-NUMA memory access adds 10-15% latency on top of the encryption overhead, and that compounds fast.

Genoa vs Turin

EPYC 9004 (Genoa, Zen 4) vs EPYC 9005 (Turin, Zen 5). Both support SEV-SNP. Turin gives you ~10-15% higher MEE bandwidth, better RMP/TLB handling (fewer TLB flushes from RMP updates), and faster attestation. Both support up to 192 cores per socket with 8 DDR5 channels and 4TB max memory.

If you’re buying new hardware specifically for confidential computing, get Turin. If you already have Genoa, it’s not worth a forklift upgrade just for the SEV-SNP improvements. The security properties are identical — it’s just faster.

The Intel TDX Question

I get asked this constantly: “What about Intel TDX?”

Both are Confidential Computing technologies. Both use per-VM keys. Both have attestation. Both have ~0-3% CPU overhead. Both kill nested virtualization and KSM. Both make migration complicated.

The differences that matter in practice:

  • Maturity. SEV-SNP has been shipping since 2020. TDX hit production in 2023. That’s three extra years of bugs found and fixed, three extra years of kernel patches, three extra years of cloud providers learning how to operate it.
  • Cloud availability. SEV-SNP is available on Oracle Cloud, AWS, GCP, and Equinix Metal today. TDX is catching up but isn’t as broadly deployed yet.
  • Ecosystem. The SEV tooling (sevctl, the sev Rust crate, kernel support) is more mature. TDX tooling is improving but still has rough edges. For device I/O, AMD has SEV-TIO (first to mainline in 6.19), Intel has TDX Connect (still in development), and ARM has CCA.

My honest take: if you’re on AMD hardware today, SEV-SNP is the obvious choice. If you’re on Intel, TDX is the obvious choice. If you’re choosing hardware for confidential computing, I’d lean AMD right now for the ecosystem maturity, but the gap is closing.

Containers and Kubernetes

Yes, you can run containers inside SEV-SNP VMs. The approach is: each container (or pod) runs inside its own lightweight SEV-SNP VM. There are two main paths:

  • containerd-shim-sev-snp-v2 — experimental but functional
  • Kata Containers — also experimental, but with more community momentum

In Kubernetes, you wire it up with a RuntimeClass and a Device Plugin. The scheduler assigns confidential workloads to SEV-SNP-capable nodes, and each pod gets its own encrypted VM boundary.

It’s not turnkey. The tooling is still rough. But if you’re in a regulated industry and need per-workload isolation with cryptographic proof, this is the path.

The BMC Angle Nobody Talks About

Here’s a fun one: your BMC (Baseboard Management Controller) can dump physical memory. In a normal setup, that’s a full memory extraction attack. With SEV-SNP? The dump is encrypted. Useless. The BMC can still do power management, hardware monitoring, and Redfish API calls — it just can’t read the data.

The PSP (Platform Security Processor) also does not have out-of-band memory access. The only path to plaintext memory is through the AMD-SP, which enforces the key hierarchy.

One gotcha: the VNC/remote console is not encrypted by SEV-SNP. The display output is still visible to the hypervisor. SEV-SNP protects memory, not screen output. If you’re doing something sensitive, don’t rely on the console. (SEV-TIO’s IDE encryption does protect the PCIe link itself, but the console rendering still happens in shared display memory.)

Version Requirements (Don’t Skip This)

  • Kernel: 6.11+ LTS recommended (6.19+ for SEV-TIO phase 1)
  • QEMU: 8.2+ (9.0+ for Turin)
  • libvirt: 9.0+ (11.0+ for Turin)
  • OVMF: 2024.05+ (the amdsev variant, not standard OVMF)
  • Microcode: Genoa 1.61.02+, Turin 1.07.02+
  • For SEV-TIO: EPYC 9005 (Turin) only, kernel 6.19+, TDISP-capable device

If you’re running older firmware, update first. I’ve seen attestation failures that took days to debug because someone was on an old microcode revision.

The Bottom Line

SEV-SNP is the first memory encryption technology I’d actually trust in a multi-tenant environment. SME was a proof of concept. SEV was a good start with a fatal flaw. SEV-ES closed the register leak but left the page table attack open. SEV-SNP finally puts the hardware in charge of memory isolation, and the RMP table is the mechanism that makes it real.

And now SEV-TIO is extending that trust boundary to the PCIe bus. Your NIC, your GPU, your NVMe drive — they can all be cryptographically bound to your VM, with encrypted links and authenticated DMA. No more bounce buffers. No more trusting the hypervisor to mediate your I/O. It’s still early (Linux 6.19 has phase 1, full TDISP is 6.20+), and device support is limited to Turin, but the direction is clear: the entire I/O path is becoming part of the TEE.

Is it perfect? No. You lose nested virtualization, KSM, memory overcommit, and easy debugging. Migration is painful and slow. The tooling is still maturing. You need specific hardware, specific firmware, specific kernel versions, and a willingness to throw out assumptions you’ve held for twenty years about how virtualization works.

But if you’re running workloads where the threat model includes your own cloud provider — and in 2026, every threat model should include your cloud provider — then SEV-SNP isn’t optional. It’s table stakes. And SEV-TIO is what comes after you’ve already sat down.

Stop trusting your hypervisor. Stop trusting your devices. Start verifying.