r/aws • u/keto_brain • 2h ago
article Lessons from building centralized ingress and egress inspection with GWLB and Palo Alto VM-Series (and what I'd change in 2026)
I wrote up a design I built a few years ago for a large regulated enterprise and figured this sub would have opinions. Short version below, full writeup with diagrams and Terraform at the end.
Starting point: every spoke VPC had no IGW and no NAT. All traffic in both directions went over TGW, down Direct Connect, and out the data center's internet edge. We moved the internet edge into a dedicated DMZ account with Palo Alto VM-Series behind a Gateway Load Balancer, inspecting everything inbound and outbound, and let app teams create their own public endpoints through a pipeline.
The things that actually mattered:
Appliance mode on every TGW attachment in front of the firewalls. GWLB keeps a flow on one firewall only if both directions reach the same GWLB endpoint. Without appliance mode, TGW can send the reply into a different AZ and the session dies.
Client IP preservation off on the DMZ NLB. Inbound chain is public NLB in the DMZ (IP targets) > TGW > internal NLB in the spoke on static IPs > internal ALB. AWS doesn't support client IP preservation when targets are reached through a TGW, and the packet walk shows why: the spoke would reply to the real client IP, the spoke TGW route table sends 0.0.0.0/0 to the egress inspection path, and that firewall has never seen the connection. With preservation off, replies go back to the NLB's private address and return through the same ingress firewall.
Edge routes on the IGW target each public subnet's exact CIDR, one per AZ, so inbound traffic is inspected by the same-AZ GWLB endpoint.
Static private IPs on the spoke NLB via subnet_mapping, since the DMZ NLB targets it by address.
GWLB target_failover rebalance helps less than you'd think. Flows get rehashed to a healthy firewall, but that firewall has no session, so long lived TCP connections still reset.
Don't use ELB health checks on the firewall ASG during bring up. The GWLB health check fails until Panorama pushes the interface management profile, and the ASG will terminate firewalls that are still bootstrapping.
Cost of the design: the app never sees the real client IP. The ALB can't take Proxy Protocol, so client IP based controls have to live on the firewall, not in a WAF on the ALB.
What I'd change today:
For inbound HTTP, cross account CloudFront VPC origins (late 2025) can replace the whole NLB chain and pass the real client IP, if the security team accepts WAF at the edge instead of an NGFW.
AWS Network Firewall now attaches natively to TGW and removes the inspection VPC. But I wouldn't swap out the Palos for a client whose network team centrally manages every firewall in Panorama across the data center, branches, and cloud. Moving the cloud side to Network Firewall splits that into two policy models, two sets of logs, and two consoles. For those shops the GWLB inspection VPC is the cost of a single view.
Full writeup: https://www.briancarpio.com/blog/centralized-inspection-gwlb-palo-alto
Curious how others are handling inbound inspection now that CloudFront VPC origins work cross account. Are your security teams still requiring an NGFW in front of everything internet facing?

