How to Handle High Concurrency in Java Applications: 6 Core Architectural Patterns

Handle high concurrency in Java applications by implementing a layered architecture that combines system partitioning, distributed caching, asynchronous message queues, database sharding, read/write separation, and Elasticsearch to distribute load and eliminate single points of failure.

The doocs/advanced-java repository provides a comprehensive guide for building high-concurrency systems, organizing solutions into six essential pillars. These patterns address specific bottlenecks that emerge when Java applications face tens of thousands of concurrent requests, offering practical strategies to achieve horizontal scalability and fault tolerance.

The Six Pillars of High-Concurrency Design

According to docs/high-concurrency/high-concurrency-design.md, the repository groups essential techniques into six core pillars: system partitioning, caching, message queues, sharding, read/write separation, and Elasticsearch. Each mitigates a specific bottleneck, and together they enable a robust, scalable solution.

System Partitioning and Microservices

Monolithic architectures concentrate all traffic on a few JVMs and a single database, quickly exhausting CPU, memory, or DB connections under load.

Split the application into independent services (e.g., order, inventory, user) communicating over RPC (Dubbo) or HTTP/REST. Each service owns its own database, distributing traffic across many JVMs and DB instances. The System拆分 section in docs/high-concurrency/high-concurrency-design.md explains this pattern and includes an illustration of a typical high-concurrency topology.

Distributed Caching with Redis

Most workloads are read-heavy; hitting the database for each request creates a hot spot.

Implement read-through and write-through caches using Redis while MySQL stores the master copy. Supplement with local caches (Caffeine) for ultra-low latency. Cache invalidation must be deterministic—use version stamps or message-queue-driven invalidation to avoid stale reads.

As detailed in docs/high-concurrency/redis-single-thread-model.md, Redis’s single-threaded model eliminates context-switching overhead and outperforms multi-threaded caches for typical workloads. To prevent catastrophic failures, implement strategies from docs/high-concurrency/redis-caching-avalanche-and-caching-penetration.md to avoid cache avalanche, penetration, and breakdown.

Asynchronous Processing with Message Queues

Direct synchronous calls (e.g., order → inventory) create back-pressure that can cascade and crash the whole system under traffic spikes.

Publish business events to a durable MQ (Kafka, RocketMQ, ActiveMQ); downstream services consume at their own pace, smoothing peaks. Deploy clustered brokers with replication and enable exactly-once delivery where needed. Implement idempotent consumers using unique request IDs and deduplication tables to guarantee at-most-once processing.

The docs/high-concurrency/mq-design.md file provides an overview of MQ architecture, reliability, idempotency, and scaling. For cluster resilience, refer to docs/high-concurrency/how-to-ensure-high-availability-of-message-queues.md.

Database Scaling: Sharding and Read/Write Separation

A single MySQL instance cannot sustain millions of QPS; partitioning data spreads the load.

Vertical sharding separates domains (order, profile, analytics) into different databases. Horizontal sharding splits large tables based on a sharding key (e.g., userId % N). Implement global ID generation using Snowflake or DB-auto-increment with offset to ensure unique primary keys across shards.

For read-heavy workloads, implement master-slave replication where the master handles writes and replicas handle reads. Use a load-balancer or proxy (HAProxy, ProxySQL) to route read queries to replicas and write queries to the master. The docs/high-concurrency/database-shard.md file covers sharding concepts, key generation, and routing, while docs/high-concurrency/mysql-read-write-separation.md explains replication mechanisms and lag mitigation.

Elasticsearch for Search and Analytics

Complex search, analytics, and aggregations are CPU-intensive for relational DBs.

Index frequently accessed fields into an ES cluster; query ES for search-heavy endpoints. Keep ES in sync with the source DB via CDC (Canal, Debezium) or MQ-driven updates. The docs/high-concurrency/es-architecture.md file details clustering and performance tuning for high QPS.

Practical Java Implementation Examples

Thread-Pool-Backed Executor for HTTP Requests

Create a bounded thread pool to prevent the servlet container from spawning unlimited threads, a common cause of OOM under burst traffic.

// Create a bounded thread pool (core=200, max=400, queue=1000)
ThreadPoolExecutor executor = new ThreadPoolExecutor(
        200,
        400,
        60L, TimeUnit.SECONDS,
        new ArrayBlockingQueue<>(1000),
        new ThreadPoolExecutor.CallerRunsPolicy());

// Submit a heavy task (e.g., order processing) to the pool
CompletableFuture<Void> future = CompletableFuture.runAsync(() -> {
    // business logic that eventually publishes a message
    orderService.process(order);
}, executor);

Publishing Events to Kafka

The asynchronous send call returns immediately, allowing the request thread to finish while the MQ handles downstream processing.

// Spring Kafka producer
@Autowired
private KafkaTemplate<String, OrderEvent> kafkaTemplate;

public void publish(OrderEvent event) {
    // The topic is partitioned; Kafka will balance load across brokers
    ListenableFuture<SendResult<String, OrderEvent>> future =
            kafkaTemplate.send("order-events", event.getId(), event);
    future.addCallback(
        success -> log.info("Sent {}", event.getId()),
        failure -> log.error("Failed {}", event.getId(), failure));
}

Read-Through Cache with Redis

If the DB is under heavy load, most reads will be served by Redis, which can sustain ~50K QPS per instance.

public Order getOrder(Long orderId) {
    String key = "order:" + orderId;
    String json = jedis.get(key);
    if (json != null) {
        return objectMapper.readValue(json, Order.class);
    }
    // Cache miss → load from DB
    Order order = orderDao.findById(orderId);
    // Populate cache with a TTL to avoid stale data
    jedis.setex(key, 300, objectMapper.writeValueAsString(order));
    return order;
}

Database Sharding with MyBatis-Plus

By routing writes to different physical databases, the write throughput scales linearly with the number of shards.

public void insertOrder(Order order) {
    // Choose shard based on orderId % 4
    int shard = (int) (order.getId() % 4);
    DataSource ds = DataSourceRouter.getDataSource("order_shard_" + shard);
    try (SqlSession session = MyBatisUtil.openSession(ds)) {
        session.getMapper(OrderMapper.class).insert(order);
    }
}

Summary

• System partitioning splits monolithic applications into independent microservices (Dubbo/RPC) to distribute JVM and database load across multiple instances.
• Distributed caching with Redis eliminates database hot spots; implement read-through patterns and guard against cache avalanche, penetration, and breakdown.
• Message queues (Kafka, RocketMQ) decouple services asynchronously, smoothing traffic spikes and preventing cascading failures through idempotent consumers.
• Database sharding (horizontal and vertical) partitions data to overcome single-instance QPS limits, requiring global ID generation strategies like Snowflake.
• Read/write separation routes write traffic to masters and read traffic to replicas, implemented via proxies like ProxySQL or HAProxy.
• Elasticsearch offloads complex search and analytics from relational databases, synchronized via CDC or MQ-driven updates.

Frequently Asked Questions

What is the most common bottleneck when handling high concurrency in Java applications?

The database connection pool and single-instance database throughput represent the most common bottlenecks. As traffic increases, a monolithic Java application exhausts available connections to a single MySQL instance, causing thread blocking and cascading timeouts. Implementing database sharding and read/write separation distributes this load across multiple physical instances.

How does Redis handle high concurrency with a single-threaded model?

Redis uses an event-driven, single-threaded architecture that eliminates context-switching overhead and lock contention present in multi-threaded caches. According to docs/high-concurrency/redis-single-thread-model.md, this design allows Redis to sustain approximately 50,000 QPS per instance by processing commands sequentially in memory without blocking I/O operations.

What is the difference between cache avalanche and cache penetration?

Cache avalanche occurs when a massive number of cache keys expire simultaneously or the cache service restarts, causing a sudden flood of requests to the database. Cache penetration happens when queries request non-existent data (cache miss) and bypass the cache entirely, repeatedly hitting the database. Both scenarios require distinct mitigation strategies: random TTL distribution for avalanche prevention and null-value caching or Bloom filters for penetration defense, as detailed in docs/high-concurrency/redis-caching-avalanche-and-caching-penetration.md.

When should I use message queues versus direct service calls?

Use message queues (Kafka, RocketMQ) when operations can be processed asynchronously without blocking the user request, such as order fulfillment, inventory updates, or notification sending. Direct synchronous calls (Dubbo RPC, HTTP) should be reserved for real-time, user-facing queries requiring immediate consistency. MQs decouple services to prevent cascading failures during traffic spikes, as implemented in docs/high-concurrency/mq-design.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}