Engineering
TypeScript vs Python
Sarah Vance
//
Jul 22, 2025
Why we rewrote our entire inference engine in Rust, but kept our control plane in TypeScript. Python is great for research, but for production systems handling thousands of concurrents, type safety is non-negotiable.
The Interface Layer
We love Python for Machine Learning research. The ecosystem is unmatched. But for the API that handles thousands of concurrent connections, Node.js + TypeScript offered better async primitives and ecosystem support for our specific needs. The "Two Language Problem" is real, but necessary for scale.
Rust for the Metal
For the actual tensor operations, we dropped down to Rust (via Candle and Burn). The memory safety guarantees of Rust prevent the segfaults common in C++ backends, while offering zero-cost abstractions that Python simply cannot match. This allows us to run models with near-metal performance while maintaining a safe codebase.
Conclusion
This hybrid stack TypeScript for the IO-bound control plane and Rust for the CPU/GPU-bound data plane allows us to move fast on the UI while remaining rock-solid on the infrastructure.
