System Design - Rate Limiting/Throttling 方法论框架

Contents

System Design 限流/节流系统面试方法论

System Design Rate Limiting/Throttling Methodology

🎯 核心问题:什么时候需要 Rate Limiting?

Core Question: When is Rate Limiting Needed?

Rate Limiting 是保护系统的关键! 几乎所有 API 系统都需要限流来防止过载。

Rate Limiting is crucial for protecting systems! Almost all API systems need rate limiting to prevent overload.

关键特征

Key Characteristics

维度 / Dimension Rate Limiting 系统
核心功能 / Core Function 限制请求速率,防止系统过载
输入 / Input API 请求
输出 / Output 允许或拒绝请求
关键挑战 / Key Challenge 精确计数、分布式一致性、低延迟
典型题目 / Typical Problems Design Rate Limiter, Design API Gateway

📊 决策树:识别 Rate Limiting 题目

Decision Tree: Identify Rate Limiting Problems

面试题目分析
Interview Problem Analysis
    │
    ├─ 是否涉及"限流"、"节流"、"API Gateway"、"防止滥用"?
    │   Does it involve "rate limiting", "throttling", "API Gateway", "prevent abuse"?
    │   │
    │   ├─ YES → Rate Limiting 系统
    │   │   └─ 继续判断具体需求...
    │   │
    │   └─ NO → 可能是其他类型
    │
    ├─ 限流粒度?
    │   Rate Limiting Granularity?
    │   │
    │   ├─ 用户级别 → 按 user_id 限流
    │   │   User-level → Rate limit by user_id
    │   ├─ IP 级别 → 按 IP 地址限流
    │   │   IP-level → Rate limit by IP address
    │   └─ API 级别 → 按 API endpoint 限流
    │       API-level → Rate limit by API endpoint
    │
    ├─ 分布式需求?
    │   Distributed Requirements?
    │   │
    │   ├─ 单机限流 → 内存计数器
    │   │   Single machine → In-memory counter
    │   └─ 分布式限流 → Redis/分布式存储
    │       Distributed → Redis/distributed storage
    │
    └─ 精确性要求?
        Accuracy Requirements?
        │
        ├─ 精确限流 → 滑动窗口日志
        │   Exact → Sliding window log
        └─ 近似限流 → 固定窗口/令牌桶
            Approximate → Fixed window/token bucket

🔍 核心特征识别

Core Characteristics Identification

典型题目关键词

Typical Problem Keywords

  1. Rate Limiting 问题
    • Design Rate Limiter
    • Design API Gateway
    • Design Throttling System
    • Prevent DDoS Attack
  2. 相关场景
    • API 限流
    • 用户行为限制
    • 资源配额管理

核心需求模式

Core Requirement Patterns

输入:API 请求
Input: API Requests
- HTTP requests
- RPC calls
- 任何需要限制的请求
  Any requests that need limiting

输出:允许或拒绝
Output: Allow or Deny
- 200 OK (允许)
- 429 Too Many Requests (拒绝)
- Rate limit headers

关键挑战:
Key Challenges:
- 精确计数(避免漏网)
  Accurate counting (avoid leaks)
- 分布式一致性(多服务器)
  Distributed consistency (multiple servers)
- 低延迟(不影响正常请求)
  Low latency (don't affect normal requests)
- 内存效率(大规模用户)
  Memory efficiency (large scale users)

🏗️ 标准架构模式

Standard Architecture Patterns

模式一:单机限流架构

Pattern 1: Single Machine Rate Limiting

适用场景: Use Cases:

  • 单服务器应用
  • 简单的限流需求
  • 不需要跨服务器一致性

核心组件: Core Components:

API Request
    ↓
Rate Limiter (In-Memory)
    ├─→ Counter/Token Bucket
    └─→ Decision Logic
    ↓
Allow/Deny Response

实现方式: Implementation:

  • 内存计数器
  • 令牌桶算法
  • 漏桶算法

模式二:分布式限流架构

Pattern 2: Distributed Rate Limiting

适用场景: Use Cases:

  • 多服务器部署
  • 需要跨服务器一致性
  • 大规模用户

核心组件: Core Components:

API Request
    ↓
Load Balancer
    ↓
API Server
    ↓
Rate Limiter Service
    ├─→ Redis (Distributed Counter)
    └─→ Rate Limit Rules
    ↓
Allow/Deny Response

关键设计点: Key Design Points:

  1. 分布式存储选择 Distributed Storage Selection:
    • Redis: 快速、支持原子操作
    • Memcached: 简单、但功能有限
    • Database: 持久化、但性能较低
  2. 一致性保证 Consistency Guarantee:
    • 使用 Redis 原子操作(INCR, EXPIRE)
    • 使用 Lua 脚本保证原子性
    • 处理 Redis 故障(降级策略)

📋 核心设计决策点

Core Design Decision Points

1. 限流算法选择

1. Rate Limiting Algorithm Selection

Fixed Window Counter(固定窗口计数器)

原理:
Principle:
- 将时间分成固定窗口(如 1 分钟)
  Divide time into fixed windows (e.g., 1 minute)
- 每个窗口维护一个计数器
  Maintain a counter for each window
- 超过限制则拒绝
  Reject if exceeds limit

优点:
Advantages:
- 实现简单
  Simple implementation
- 内存效率高
  Memory efficient

缺点:
Disadvantages:
- 窗口边界可能突发(2x limit)
  Window boundaries may have bursts (2x limit)
- 不够精确
  Not very accurate

示例:
Example:
Limit: 100 requests/minute
Window 1 (0:00-0:59): 100 requests
Window 2 (1:00-1:59): 100 requests
At 0:59 and 1:00, can send 200 requests in 2 seconds!

Sliding Window Log(滑动窗口日志)

原理:
Principle:
- 记录每个请求的时间戳
  Record timestamp of each request
- 维护一个时间窗口内的请求列表
  Maintain list of requests within time window
- 超过限制则拒绝
  Reject if exceeds limit

优点:
Advantages:
- 精确限流
  Accurate rate limiting
- 不会突发
  No bursts

缺点:
Disadvantages:
- 内存消耗大(存储所有时间戳)
  High memory consumption (store all timestamps)
- 实现复杂
  Complex implementation

Sliding Window Counter(滑动窗口计数器)

原理:
Principle:
- 结合固定窗口和滑动窗口
  Combine fixed window and sliding window
- 使用多个固定窗口,加权计算
  Use multiple fixed windows, weighted calculation

优点:
Advantages:
- 相对精确
  Relatively accurate
- 内存效率较高
  Relatively memory efficient

缺点:
Disadvantages:
- 实现较复杂
  More complex implementation
- 仍可能有小误差
  May still have small errors

Token Bucket(令牌桶)

原理:
Principle:
- 桶中有固定数量的令牌
  Bucket has fixed number of tokens
- 按固定速率补充令牌
  Refill tokens at fixed rate
- 请求消耗令牌,无令牌则拒绝
  Requests consume tokens, reject if no tokens

优点:
Advantages:
- 允许突发(桶满时)
  Allows bursts (when bucket is full)
- 平滑限流
  Smooth rate limiting

缺点:
Disadvantages:
- 实现较复杂
  More complex implementation
- 需要维护令牌状态
  Need to maintain token state

Leaky Bucket(漏桶)

原理:
Principle:
- 请求进入桶(队列)
  Requests enter bucket (queue)
- 按固定速率处理请求
  Process requests at fixed rate
- 桶满则拒绝
  Reject if bucket is full

优点:
Advantages:
- 输出速率恒定
  Constant output rate
- 平滑流量
  Smooth traffic

缺点:
Disadvantages:
- 不允许突发
  Doesn't allow bursts
- 可能增加延迟
  May increase latency

2. 分布式实现策略

2. Distributed Implementation Strategy

Redis-based Rate Limiting

# Fixed Window with Redis
def is_allowed(user_id, limit, window):
    key = f"rate_limit:{user_id}"
    current = redis.incr(key)
    
    if current == 1:
        redis.expire(key, window)
    
    return current <= limit

# Sliding Window with Redis
def is_allowed_sliding(user_id, limit, window):
    key = f"rate_limit:{user_id}"
    now = time.time()
    
    # Remove old entries
    redis.zremrangebyscore(key, 0, now - window)
    
    # Count current requests
    current = redis.zcard(key)
    
    if current < limit:
        redis.zadd(key, {str(now): now})
        redis.expire(key, window)
        return True
    
    return False

一致性保证

Consistency Guarantee

挑战:
Challenges:
- 多服务器同时更新
  Multiple servers updating simultaneously
- Redis 故障处理
  Redis failure handling

解决方案:
Solutions:
1. 使用 Redis 原子操作
   Use Redis atomic operations
2. 使用 Lua 脚本保证原子性
   Use Lua scripts for atomicity
3. Redis 故障降级(本地限流)
   Redis failure fallback (local rate limiting)

3. 限流粒度选择

3. Rate Limiting Granularity Selection

用户级别限流(User-level)

适用场景:
Use Cases:
- 防止单个用户滥用
  Prevent single user abuse
- 公平使用资源
  Fair resource usage

实现:
Implementation:
Key: "rate_limit:user:{user_id}"

IP 级别限流(IP-level)

适用场景:
Use Cases:
- 防止 DDoS 攻击
  Prevent DDoS attacks
- 匿名用户限流
  Rate limit anonymous users

实现:
Implementation:
Key: "rate_limit:ip:{ip_address}"

API 级别限流(API-level)

适用场景:
Use Cases:
- 保护特定 API
  Protect specific APIs
- 不同 API 不同限制
  Different limits for different APIs

实现:
Implementation:
Key: "rate_limit:api:{api_path}"

组合限流(Combined)

适用场景:
Use Cases:
- 多层次保护
  Multi-level protection
- 精细控制
  Fine-grained control

实现:
Implementation:
- 用户级别 + API 级别
  User-level + API-level
- 全局限制 + 用户限制
  Global limit + user limit

🎯 标准解题流程

Standard Problem-Solving Process

Step 1: 需求澄清(Requirements Clarification)

Step 1: Requirements Clarification

必须明确的问题: Questions to Clarify:

  1. 限流粒度 Rate Limiting Granularity:
    • 用户级别?IP 级别?API 级别?
    • User-level? IP-level? API-level?
  2. 限流规则 Rate Limiting Rules:
    • 限制数量(如 100 requests/minute)
    • 时间窗口(1 分钟?1 小时?)
    • 不同用户不同限制?
  3. 分布式需求 Distributed Requirements:
    • 单机还是分布式?
    • Single machine or distributed?
    • 是否需要跨服务器一致性?
    • Need cross-server consistency?
  4. 精确性要求 Accuracy Requirements:
    • 精确限流还是可以近似?
    • Exact or approximate?
    • 是否可以接受小误差?

Step 2: 算法选择

Step 2: Algorithm Selection

选择标准: Selection Criteria:

精确性要求高 + 内存充足
High accuracy + sufficient memory
→ Sliding Window Log

允许近似 + 内存受限
Allow approximation + memory constrained
→ Fixed Window Counter

需要允许突发
Need to allow bursts
→ Token Bucket

需要平滑输出
Need smooth output
→ Leaky Bucket

平衡选择
Balanced choice
→ Sliding Window Counter

Step 3: 基础设计(Basic Design)

Step 3: Basic Design

最小可行方案: Minimum Viable Solution:

1. Rate Limiter Service
   - 接收请求
     Receive requests
   - 检查限流规则
     Check rate limit rules
   - 返回允许/拒绝
     Return allow/deny

2. Storage (Redis)
   - 存储计数器
     Store counters
   - 维护时间窗口
     Maintain time windows

Step 4: 优化设计(Optimized Design)

Step 4: Optimized Design

核心优化方向: Core Optimization Directions:

  1. 性能优化 Performance Optimization:
    • 本地缓存(减少 Redis 调用)
    • 批量操作
    • 异步更新
  2. 可靠性优化 Reliability Optimization:
    • Redis 故障降级
    • 多 Redis 实例
    • 监控和告警
  3. 扩展性优化 Scalability Optimization:
    • 分片(按用户 ID)
    • 水平扩展
    • 负载均衡

📚 典型题目分类

Problem Categories

Rate Limiter 问题

  1. Design Rate Limiter
    • 核心:限制 API 请求速率
    • 关键:算法选择、分布式实现
    • 挑战:精确性、一致性、性能
  2. Design API Gateway
    • 核心:API 网关 + 限流
    • 关键:多级限流、规则管理
    • 挑战:高并发、低延迟

🎯 面试策略总结

Interview Strategy Summary

开场策略

Opening Strategy

1. 识别题目类型
   Identify problem type
   "这是一个限流系统设计问题"
   "This is a rate limiting system design problem"

2. 明确核心需求
   Clarify core requirements
   "需要限制 [用户/IP/API] 的请求速率"
   "Need to limit request rate for [user/IP/API]"

3. 询问关键参数
   Ask key parameters
   - 限流粒度
     Rate limiting granularity
   - 限流规则(数量、时间窗口)
     Rate limit rules (count, time window)
   - 分布式需求
     Distributed requirements

算法选择策略

Algorithm Selection Strategy

1. 先问精确性要求
   Ask accuracy requirements first
   "是否需要精确限流,还是可以接受近似?"
   "Need exact rate limiting or can accept approximation?"

2. 根据需求选择算法
   Select algorithm based on requirements
   - 精确 → Sliding Window Log
   - 近似 → Fixed Window / Token Bucket
   - 突发 → Token Bucket
   - 平滑 → Leaky Bucket

3. 讨论权衡
   Discuss trade-offs
   "精确性 vs 内存,性能 vs 准确性"
   "Accuracy vs memory, performance vs accuracy"

📝 快速检查清单

Quick Checklist

需求澄清 Checklist

  • 限流粒度(用户/IP/API)
  • 限流规则(数量、时间窗口)
  • 分布式需求
  • 精确性要求
  • 性能要求(延迟、吞吐量)

设计 Checklist

  • 算法选择(Fixed/Sliding/Token Bucket)
  • 存储选择(Redis/Memcached/Database)
  • 分布式一致性(原子操作/Lua 脚本)
  • 故障处理(降级策略)
  • 监控和告警

🚀 实战模板

Practical Templates

开场话术

Opening Script

"这是一个限流系统设计问题。
"This is a rate limiting system design problem.

核心需求是:
Core requirements:
1. 限制 [用户/IP/API] 的请求速率
   Limit request rate for [user/IP/API]
2. 支持 [精确/近似] 限流
   Support [exact/approximate] rate limiting
3. [单机/分布式] 部署
   [Single machine/Distributed] deployment

让我先澄清几个关键问题:
Let me clarify a few key questions:
- 限流粒度是什么?(用户/IP/API)
  What's the rate limiting granularity? (user/IP/API)
- 限流规则是什么?(如 100 requests/minute)
  What are the rate limit rules? (e.g., 100 requests/minute)
- 是否需要精确限流?
  Do we need exact rate limiting?"

算法选择话术

Algorithm Selection Script

"根据需求,我选择 [算法名称]:
"Based on requirements, I choose [algorithm name]:

原因:
Reasons:
1. [精确性/内存效率/允许突发] 要求
   [Accuracy/Memory efficiency/Allow bursts] requirement
2. [单机/分布式] 部署场景
   [Single machine/Distributed] deployment scenario
3. [低延迟/高吞吐] 性能需求
   [Low latency/High throughput] performance needs

实现方式:
Implementation:
- 使用 [Redis/内存] 存储计数器
  Use [Redis/memory] to store counters
- [固定窗口/滑动窗口/令牌桶] 算法
  [Fixed window/Sliding window/Token bucket] algorithm
- [原子操作/Lua 脚本] 保证一致性
  [Atomic operations/Lua scripts] for consistency"

记住:这类题目的核心是选择合适的限流算法、实现分布式一致性、处理故障场景。重点是算法选择、Redis 使用、故障降级! Remember: The core of these problems is choosing the right rate limiting algorithm, implementing distributed consistency, and handling failure scenarios. Focus on algorithm selection, Redis usage, and failure fallback!

Contents