Skip to main content

3 posts tagged with "Infrastructure"

Infrastructure

View All Tags

Why Node.js and Python Don't Use Linux's Native Async I/O API

· 11 min read
Ritesh Kadmawala
Founder, Vertexcover Labs - AI-native engineering studio
TL;DR:

Linux provides a native Asynchronous I/O (AIO) API that should theoretically allow true background file operations. Yet popular frameworks like Node.js and Python's asyncio avoid it entirely, relying instead on thread pools with blocking I/O. Why? The answer reveals important lessons about API design, cross-platform compatibility, and the gap between theoretical elegance and practical engineering. The future may lie in io_uring, a modern interface that addresses many of AIO's shortcomings.

ML Infra design for the GPU Poor

· 5 min read

Taming the Beast: How to Design a Queueing System for GPU-Intensive Workloads

TL;DR:

When designing for scale, the limiting factor is the GPU availability. So all rate limits / queueing must be designed around GPU availability.