Load Balancer¶

Load balancing strategies including consistent hashing, round-robin, and least-connections.

Load balancer components for traffic distribution.

This package provides load balancing abstractions with pluggable strategies for distributing requests across backend servers.

Example

from happysimulator.components.load_balancer import ( LoadBalancer, HealthChecker, RoundRobin, LeastConnections, )

Create backends¶

servers = [Server(name=f"server_{i}", ...) for i in range(3)]

Create load balancer with round-robin strategy¶

lb = LoadBalancer( name="api_lb", backends=servers, strategy=RoundRobin(), )

Optionally add health checking¶

health_checker = HealthChecker( name="health_check", load_balancer=lb, interval=5.0, timeout=1.0, )

BackendHealthState `dataclass` ¶

BackendHealthState(
    consecutive_successes: int = 0,
    consecutive_failures: int = 0,
    last_check_time: Instant | None = None,
    last_check_passed: bool | None = None,
    is_checking: bool = False,
)

Health state tracking for a backend.

HealthChecker ¶

HealthChecker(
    name: str,
    load_balancer: LoadBalancer,
    interval: float = 10.0,
    timeout: float = 5.0,
    healthy_threshold: int = 2,
    unhealthy_threshold: int = 3,
    check_event_type: str = "health_check",
)

Bases: Entity

Periodically checks backend health.

Sends health check probes to backends and tracks consecutive successes/failures to determine health status. Updates the load balancer when backends become healthy or unhealthy.

Attributes:

Name	Type	Description
`name`		Health checker identifier for logging.
`load_balancer`	`LoadBalancer`	The load balancer to update.
`interval`	`float`	Time between health checks in seconds.
`timeout`	`float`	Maximum time to wait for health check response.
`healthy_threshold`	`int`	Consecutive successes to mark healthy.
`unhealthy_threshold`	`int`	Consecutive failures to mark unhealthy.

Initialize the health checker.

Parameters:

Name	Type	Description	Default
`name`	`str`	Health checker identifier.	required
`load_balancer`	`LoadBalancer`	Load balancer to manage.	required
`interval`	`float`	Seconds between checks (default 10).	`10.0`
`timeout`	`float`	Seconds before check times out (default 5).	`5.0`
`healthy_threshold`	`int`	Successes to mark healthy (default 2).	`2`
`unhealthy_threshold`	`int`	Failures to mark unhealthy (default 3).	`3`
`check_event_type`	`str`	Event type for health check probes.	`'health_check'`

Raises:

Type	Description
`ValueError`	If parameters are invalid.

stats `property` ¶

stats: HealthCheckStats

Return a frozen snapshot of health check statistics.

load_balancer `property` ¶

load_balancer: LoadBalancer

The load balancer being monitored.

interval `property` ¶

interval: float

Seconds between health checks.

timeout `property` ¶

timeout: float

Seconds before a check times out.

healthy_threshold `property` ¶

healthy_threshold: int

Consecutive successes to mark healthy.

unhealthy_threshold `property` ¶

unhealthy_threshold: int

Consecutive failures to mark unhealthy.

is_running `property` ¶

is_running: bool

Whether health checking is active.

start ¶

start() -> Event

Start periodic health checking.

Returns an event that begins the health check cycle. Schedule this event to start health checking.

Returns:

Type	Description
`Event`	Event to schedule.

stop ¶

stop() -> None

Stop periodic health checking.

get_backend_state ¶

get_backend_state(backend: Entity) -> BackendHealthState

Get the health state for a backend.

handle_event ¶

handle_event(
    event: Event,
) -> (
    Generator[float, None, list[Event] | Event | None]
    | list[Event]
    | Event
    | None
)

Handle health checker events.

Parameters:

Name	Type	Description	Default
`event`	`Event`	The event to handle.	required

Returns:

Type	Description
`Generator[float, None, list[Event] \| Event \| None] \| list[Event] \| Event \| None`	Events to schedule.

get_backend_state_by_name ¶

get_backend_state_by_name(
    name: str,
) -> BackendHealthState | None

Get health state by backend name.

HealthCheckStats `dataclass` ¶

HealthCheckStats(
    checks_performed: int = 0,
    checks_passed: int = 0,
    checks_failed: int = 0,
    checks_timed_out: int = 0,
    backends_marked_healthy: int = 0,
    backends_marked_unhealthy: int = 0,
)

Statistics tracked by HealthChecker.

BackendInfo `dataclass` ¶

BackendInfo(
    backend: Entity,
    weight: int = 1,
    is_healthy: bool = True,
    consecutive_successes: int = 0,
    consecutive_failures: int = 0,
    total_requests: int = 0,
    total_failures: int = 0,
)

Information tracked for each backend.

LoadBalancer ¶

LoadBalancer(
    name: str,
    backends: list[Entity] | None = None,
    strategy: LoadBalancingStrategy | None = None,
    on_no_backend: str = "reject",
)

Bases: Entity

Distributes requests across multiple backend servers.

The load balancer receives incoming requests and forwards them to one of the configured backends based on the selected strategy. It tracks backend health and can exclude unhealthy backends from the routing pool.

Attributes:

Name	Type	Description
`name`		Load balancer identifier for logging.
`backends`		List of backend entities to route to.
`strategy`	`LoadBalancingStrategy`	Algorithm for selecting backends.

Initialize the load balancer.

Parameters:

Name	Type	Description	Default
`name`	`str`	Load balancer identifier.	required
`backends`	`list[Entity] \| None`	Initial list of backend entities.	`None`
`strategy`	`LoadBalancingStrategy \| None`	Load balancing strategy (default RoundRobin).	`None`
`on_no_backend`	`str`	Action when no backend available.	`'reject'`

Raises:

Type	Description
`ValueError`	If on_no_backend is invalid.

stats `property` ¶

stats: LoadBalancerStats

Return a frozen snapshot of load balancer statistics.

strategy `property` ¶

strategy: LoadBalancingStrategy

The load balancing strategy.

all_backends `property` ¶

all_backends: list[Entity]

All registered backends (healthy and unhealthy).

healthy_backends `property` ¶

healthy_backends: list[Entity]

Only healthy backends available for routing.

unhealthy_backends `property` ¶

unhealthy_backends: list[Entity]

Backends currently marked as unhealthy.

backend_count `property` ¶

backend_count: int

Total number of registered backends.

healthy_count `property` ¶

healthy_count: int

Number of healthy backends.

get_backend_info_by_name ¶

get_backend_info_by_name(name: str) -> BackendInfo | None

Get backend info by backend name.

add_backend ¶

add_backend(backend: Entity, weight: int = 1) -> None

Add a backend to the load balancer.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend entity to add.	required
`weight`	`int`	Weight for weighted strategies (default 1).	`1`

Raises:

Type	Description
`ValueError`	If weight is less than 1.

remove_backend ¶

remove_backend(backend: Entity) -> None

Remove a backend from the load balancer.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend entity to remove.	required

mark_unhealthy ¶

mark_unhealthy(backend: Entity) -> None

Mark a backend as unhealthy.

Unhealthy backends are excluded from the routing pool until marked healthy again.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend to mark unhealthy.	required

mark_healthy ¶

mark_healthy(backend: Entity) -> None

Mark a backend as healthy.

Healthy backends are included in the routing pool.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend to mark healthy.	required

get_backend_info ¶

get_backend_info(backend: Entity) -> BackendInfo | None

Get tracking information for a backend.

record_success ¶

record_success(backend: Entity) -> None

Record a successful request to a backend.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend that handled the request.	required

record_failure ¶

record_failure(backend: Entity) -> None

Record a failed request to a backend.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend that failed.	required

handle_event ¶

handle_event(
    event: Event,
) -> (
    Generator[float, None, list[Event] | Event | None]
    | list[Event]
    | Event
    | None
)

Handle incoming events.

Routes requests to backends and handles responses.

Parameters:

Name	Type	Description	Default
`event`	`Event`	The event to handle.	required

Returns:

Type	Description
`Generator[float, None, list[Event] \| Event \| None] \| list[Event] \| Event \| None`	Events to schedule.

LoadBalancerStats `dataclass` ¶

LoadBalancerStats(
    requests_received: int = 0,
    requests_forwarded: int = 0,
    requests_failed: int = 0,
    no_backend_available: int = 0,
    backends_marked_unhealthy: int = 0,
    backends_marked_healthy: int = 0,
)

Statistics tracked by LoadBalancer.

ConsistentHash ¶

ConsistentHash(
    virtual_nodes: int = 100,
    get_key: Callable[[Event], str] | None = None,
)

Consistent hashing with virtual nodes.

Provides stable routing that minimizes remapping when backends are added or removed. Each backend is mapped to multiple points on a hash ring (virtual nodes) for better distribution.

Initialize consistent hash strategy.

Parameters:

Name	Type	Description	Default
`virtual_nodes`	`int`	Number of virtual nodes per backend.	`100`
`get_key`	`Callable[[Event], str] \| None`	Function to extract routing key from request.	`None`

add_backend ¶

add_backend(backend: Entity) -> None

Add a backend to the hash ring.

remove_backend ¶

remove_backend(backend: Entity) -> None

Remove a backend from the hash ring.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select backend using consistent hashing.

IPHash ¶

IPHash(get_key: Callable[[Event], str] | None = None)

Consistent hashing based on client identifier.

Routes requests from the same client to the same backend, useful for session affinity. Falls back to round-robin if no key found.

Initialize IP hash strategy.

Parameters:

Name	Type	Description	Default
`get_key`	`Callable[[Event], str] \| None`	Function to extract routing key from request. Defaults to looking for 'client_id' in metadata.	`None`

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select backend based on hashed client key.

LeastConnections ¶

Selects backend with fewest active connections.

Directs traffic to the least loaded backend, which helps balance load when request handling times vary. Requires backends to expose their current connection count.

The strategy looks for an active_connections or active_requests property on backends, falling back to 0 if not found.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select the backend with fewest active connections.

LeastResponseTime ¶

LeastResponseTime(alpha: float = 0.3)

Selects backend with lowest recent average response time.

Tracks response times and directs traffic to the fastest backend. Uses exponential moving average to weight recent responses more heavily.

Initialize least response time strategy.

Parameters:

Name	Type	Description	Default
`alpha`	`float`	Smoothing factor for EMA (0-1). Higher values give more weight to recent observations.	`0.3`

record_response_time ¶

record_response_time(
    backend: Entity, response_time: float
) -> None

Record a response time observation for a backend.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend that handled the request.	required
`response_time`	`float`	Time taken to handle the request in seconds.	required

get_response_time ¶

get_response_time(backend: Entity) -> float

Get the current average response time for a backend.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select backend with lowest average response time.

LoadBalancingStrategy ¶

Bases: Protocol

Protocol for load balancing algorithms.

Implementations select which backend should handle a given request from the list of available (healthy) backends.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select a backend to handle the request.

Parameters:

Name	Type	Description	Default
`backends`	`list[Entity]`	List of available backends to choose from.	required
`request`	`Event`	The incoming request event.	required

Returns:

Type	Description
`Entity \| None`	The selected backend, or None if no backend available.

PowerOfTwoChoices ¶

Pick two random backends, choose the one with fewer connections.

A probabilistic approach that provides near-optimal load balancing with low overhead. Better than pure random, simpler than least connections.

Reference: "The Power of Two Choices in Randomized Load Balancing" by Mitzenmacher, Richa, and Sitaraman.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select from two random backends the one with fewer connections.

Random ¶

Random backend selection.

Simple strategy that randomly selects from available backends. Provides good distribution over many requests without maintaining state.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select a random backend.

RoundRobin ¶

RoundRobin()

Cycles through backends in sequential order.

Simple and fair distribution that gives each backend an equal share of requests over time. Best when backends have similar capacity and request handling times are uniform.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select the next backend in round-robin order.

reset ¶

reset() -> None

Reset the round-robin counter.

WeightedLeastConnections ¶

WeightedLeastConnections()

Least connections with weights.

Combines connection count with backend weights. A backend with weight 2 and 4 connections is equivalent to a backend with weight 1 and 2 connections.

Score = connections / weight (lower is better)

set_weight ¶

set_weight(backend: Entity, weight: int) -> None

Set the weight for a backend.

get_weight ¶

get_weight(backend: Entity) -> int

Get the weight for a backend.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select backend with lowest weighted connection score.

WeightedRoundRobin ¶

WeightedRoundRobin()

Round-robin with weighted distribution.

Backends with higher weights receive proportionally more requests. Useful when backends have different capacities.

Example

server1 gets 3x the traffic of server2¶

strategy = WeightedRoundRobin() strategy.set_weight(server1, 3) strategy.set_weight(server2, 1)

set_weight ¶

set_weight(backend: Entity, weight: int) -> None

Set the weight for a backend.

Parameters:

Name	Type	Description	Default
`backend`	`Entity`	The backend entity.	required
`weight`	`int`	Weight value (higher = more traffic).	required

Raises:

Type	Description
`ValueError`	If weight is less than 1.

get_weight ¶

get_weight(backend: Entity) -> int

Get the weight for a backend.

select ¶

select(
    backends: list[Entity], request: Event
) -> Entity | None

Select backend using weighted round-robin (smooth weighted).

Load Balancer¶

Create backends¶

Create load balancer with round-robin strategy¶

Optionally add health checking¶

BackendHealthState dataclass ¶

HealthChecker ¶

stats property ¶

load_balancer property ¶

interval property ¶

timeout property ¶

healthy_threshold property ¶

unhealthy_threshold property ¶

is_running property ¶

start ¶

stop ¶

get_backend_state ¶

handle_event ¶

get_backend_state_by_name ¶

HealthCheckStats dataclass ¶

BackendInfo dataclass ¶

LoadBalancer ¶

stats property ¶

strategy property ¶

all_backends property ¶

healthy_backends property ¶

unhealthy_backends property ¶

backend_count property ¶

healthy_count property ¶

get_backend_info_by_name ¶

add_backend ¶

remove_backend ¶

mark_unhealthy ¶

mark_healthy ¶

get_backend_info ¶

record_success ¶

record_failure ¶

handle_event ¶

LoadBalancerStats dataclass ¶

ConsistentHash ¶

add_backend ¶

remove_backend ¶

select ¶

IPHash ¶

select ¶

LeastConnections ¶

select ¶

LeastResponseTime ¶

record_response_time ¶

get_response_time ¶

select ¶

LoadBalancingStrategy ¶

select ¶

PowerOfTwoChoices ¶

select ¶

Random ¶

select ¶

RoundRobin ¶

select ¶

reset ¶

WeightedLeastConnections ¶

set_weight ¶

get_weight ¶

select ¶

WeightedRoundRobin ¶

server1 gets 3x the traffic of server2¶

set_weight ¶

get_weight ¶

select ¶

BackendHealthState `dataclass` ¶

stats `property` ¶

load_balancer `property` ¶

interval `property` ¶

timeout `property` ¶

healthy_threshold `property` ¶

unhealthy_threshold `property` ¶

is_running `property` ¶

HealthCheckStats `dataclass` ¶

BackendInfo `dataclass` ¶

stats `property` ¶

strategy `property` ¶

all_backends `property` ¶

healthy_backends `property` ¶

unhealthy_backends `property` ¶

backend_count `property` ¶

healthy_count `property` ¶

LoadBalancerStats `dataclass` ¶