VPN Technologies and Protocols
Virtual Private Network (VPN) technologies establish encrypted tunnels across public or shared network infrastructure, enabling private communications between endpoints that may be geographically dispersed. This page covers the major VPN protocol categories, their underlying mechanisms, applicable deployment scenarios, and the decision criteria that distinguish one approach from another. Regulatory frameworks from NIST, CISA, and federal network security standards shape how organizations select and configure VPN solutions. Understanding this service sector matters because protocol choice directly affects cryptographic strength, interoperability, performance, and compliance posture.
Definition and scope
A VPN, in technical terms, is a logical network overlay that uses encapsulation and encryption to carry private traffic across an untrusted transport layer — typically the public internet. The scope of VPN technologies spans three distinct deployment models:
- Remote-access VPN: Connects individual endpoints (laptops, mobile devices) to a central network gateway.
- Site-to-site VPN: Bridges two or more fixed network locations, such as branch offices to a corporate data center.
- Cloud VPN: Extends private connectivity into cloud environments such as AWS, Azure, or GCP using managed gateways.
NIST Special Publication 800-77 Rev. 1, Guide to IPsec VPNs, provides the federal baseline for IPsec deployments and is referenced by agencies subject to FISMA. The scope of VPN regulation in U.S. federal environments is also shaped by CISA's Zero Trust Maturity Model, which treats traditional perimeter-based VPN as a legacy control requiring phased replacement with identity-aware access mechanisms.
VPN technologies intersect closely with network encryption protocols and zero-trust network architecture, both of which define the security model within which VPN operates.
How it works
VPN operation proceeds across four discrete phases regardless of protocol:
- Authentication: The initiating endpoint proves identity to the receiving gateway using certificates, pre-shared keys (PSKs), or multi-factor authentication. IKEv2 (Internet Key Exchange version 2) is the dominant standard for IPsec-based authentication.
- Key exchange: Cryptographic session keys are negotiated using Diffie-Hellman (DH) groups or Elliptic Curve Diffie-Hellman (ECDH). NIST SP 800-56A specifies approved key-establishment schemes.
- Tunnel establishment: Traffic is encapsulated within a protocol-defined tunnel. Encapsulation headers add overhead (typically 20–100 bytes per packet depending on protocol), which affects throughput.
- Data transmission and termination: Encrypted packets traverse the public network and are decrypted at the remote endpoint or gateway. Session teardown follows defined timeout or explicit disconnect procedures.
Protocol classification
| Protocol | Layer | Primary Use | Cipher Support |
|---|---|---|---|
| IPsec (IKEv2) | Layer 3 | Site-to-site, remote access | AES-256, SHA-2 |
| SSL/TLS (OpenVPN) | Layer 4–7 | Remote access, firewall traversal | AES-256-GCM |
| WireGuard | Layer 3 | High-performance remote access | ChaCha20-Poly1305 |
| L2TP/IPsec | Layer 2 | Legacy remote access | AES-128 (with IPsec) |
| SSTP | Layer 4–7 | Windows-native remote access | AES-256 |
| MPLS VPN | Layer 2.5 | Carrier-grade site-to-site | Provider-managed |
IPsec vs. TLS-based VPNs: IPsec operates at the network layer and requires OS-level integration, offering lower latency and full traffic tunneling. TLS-based VPNs (OpenVPN, SSTP) operate at higher layers, traverse NAT and firewalls more readily, and support clientless browser-based access. The tradeoff is that TLS VPNs carry higher per-packet overhead and are subject to TLS vulnerabilities tracked in the CVE database maintained by NIST NVD.
WireGuard, standardized in the Linux kernel as of version 5.6 (2020), uses a fixed cryptographic suite (ChaCha20-Poly1305, Curve25519, BLAKE2s) that eliminates cipher negotiation complexity but reduces cryptographic agility — a consideration in environments requiring algorithm rotation under NIST post-quantum guidance.
Common scenarios
VPN technologies appear across four primary operational contexts in U.S. enterprise and government environments:
Remote workforce access: Organizations deploy remote-access VPNs to connect employees working outside the corporate perimeter. This scenario is governed by network security for remote workforces policies and, in federal contexts, OMB Memorandum M-22-09, which mandates zero-trust architecture adoption across civilian agencies.
Branch office interconnection: Site-to-site IPsec tunnels link geographically distributed offices over ISP connections. Retail, healthcare, and financial organizations with 10 or more branch locations frequently standardize on IKEv2/IPsec due to hardware appliance interoperability.
OT and ICS environments: Industrial control systems require isolated VPN access for remote maintenance. CISA's ICS-CERT advisories document repeated exploitation of unpatched VPN appliances in operational technology networks — a sector covered in detail at OT and ICS network security.
Cloud hybrid access: Enterprises extending infrastructure into cloud platforms use VPN gateways (AWS Site-to-Site VPN, Azure VPN Gateway) to maintain private routing. Bandwidth constraints apply: AWS Site-to-Site VPN throughput is capped at 1.25 Gbps per tunnel (per AWS documentation), which affects high-throughput design decisions.
Decision boundaries
Protocol selection depends on five criteria that define clear boundaries between appropriate and inappropriate VPN choices:
- Compliance requirements: Federal agencies bound by FISMA must use FIPS 140-3 validated cryptographic modules. WireGuard's current implementations lack universal FIPS validation, making IPsec IKEv2 with AES-256 the required choice in those environments (NIST FIPS 140-3).
- Throughput demands: Environments requiring multi-gigabit encrypted throughput should evaluate hardware-accelerated IPsec rather than software-based TLS VPNs, which consume more CPU per packet.
- NAT traversal: Deployments behind double-NAT (common in residential ISP environments) favor OpenVPN or WireGuard, which traverse NAT more reliably than raw IPsec ESP.
- Client platform diversity: Cross-platform remote access with minimal client installation points toward SSL/TLS VPN or SSTP. Enterprise managed devices typically support IPsec IKEv2 natively on Windows, macOS, iOS, and Android.
- Zero-trust migration posture: Organizations aligned with CISA's Zero Trust Maturity Model treat VPN as a transitional control. Secure access service edge and network access control architectures progressively replace VPN-centric perimeter models by enforcing identity and device posture at every session, not just at tunnel establishment.
Split tunneling — routing only specific destination traffic through the VPN while allowing direct internet access for other traffic — reduces bandwidth load but expands the attack surface. NIST SP 800-46 Rev. 2, Guide to Enterprise Telework, Remote Access, and BYOD Security, advises against full split tunneling in high-sensitivity environments due to lateral exposure risk.
References
- NIST SP 800-77 Rev. 1 — Guide to IPsec VPNs
- NIST SP 800-46 Rev. 2 — Guide to Enterprise Telework, Remote Access, and BYOD Security
- NIST FIPS 140-3 — Security Requirements for Cryptographic Modules
- NIST Post-Quantum Cryptography Project
- NIST National Vulnerability Database (NVD)
- CISA Zero Trust Maturity Model
- CISA ICS-CERT Advisories
- AWS Site-to-Site VPN Documentation — Tunnel Options
- OMB Memorandum M-22-09 — Moving the U.S. Government Toward Zero Trust Cybersecurity Principles