Databricks, a cloud data and AI platform company, changed how it protects cluster creation from overload. It had used fixed request-rate limits, but an API call returns in milliseconds while provisioning virtual machines (VMs) can take minutes. The team now counts in-flight provisioning workflows, shrinks that concurrency limit when latency rises, and holds extra requests in a bounded queue with fair per-customer scheduling. The company says default limits rose 7.5x, rate-limit errors fell by more than 99.99%, and over 100 customer-specific overrides were removed.
The fixed limits failed both ways: they rejected work when capacity was free and accepted too much when cloud providers slowed. The new design has costs. Queued requests wait longer, and retries inside accepted work can still prolong overload; the authors list retry backoff as future work. The post omits absolute limit values and the exact scope of the error figure.
