Running LLMs across Cloudflare’s network requires us to be smarter and more efficient about GPU memory bandwidth. That’s why we developed Unweight, a lossless inference-time compression system that achieves up to a 22% model footprint reduction, so that we can deliver faster and cheaper inference than ever before.
Many products at Cloudflare aren’t possible without pushing the limits of network hardware and software to deliver improved performance, increased efficiency, or novel capabilities such as soft-unicast, our method for sharing IP subnets across data centers. Happily, most people do not need to know the intricacies of how your operating system handles network and Internet access in general. Yes, even most people within Cloudflare. But sometimes we try to push well beyond the design intentions of Linux’s networking stack. This is a story about one of those attempts.
WARP’s initial implementation resembled a VPN that allows Internet access through it. Here’s how we built it – and how you can, too.
We split a proxy application into multiple services to improve development agility and reliability. This blog also shares some common patterns we are leveraging to design a system supporting zero-downtime restart
Deploying new versions of long-lived server software while maintaining a reliable experience is challenging. For oxy, we established several development and operational patterns to increase reliability and reduce friction in deployments
In an abstract sense Cloudflare Warp is similar; its connection strategy punches a hole through firewalls and NAT, and provides easy and secure passage for HTTP traffic to your origin.
