# Linkis EngineConn Connection Pool Architecture and Tuning Guide for Optimal Performance > Explore the Linkis EngineConn connection pool architecture. Learn how to tune engine reuse for optimal performance with this comprehensive guide. - Repository: [The Apache Software Foundation/linkis](https://github.com/apache/linkis) - Tags: architecture - Published: 2026-02-24 --- **The Linkis EngineConn connection pool implements a thread-pool-backed, cache-centric architecture that manages reusable engine instances through async reuse and creation pathways rather than traditional JDBC-style pooling.** The Apache Linkis computation governance framework handles compute engine lifecycle management through a sophisticated reuse mechanism centered on the Linkis EngineConn connection pool. Unlike conventional database connection pools, this system coordinates engine instances using dedicated thread pools and an in-memory executor cache to minimize engine startup overhead. Understanding the architecture of `DefaultEngineAskEngineService` and the tunable parameters in `AMConfiguration` enables operators to optimize throughput for mixed workloads. ## Architecture of the Linkis EngineConn Connection Pool Linkis does not use a classic JDBC-style connection pool for EngineConn. Instead, it manages **engine-instance executors** through a combination of **thread pools** and a **cache of reusable EngineConn executors** that act as the logical connection between client requests and running engines. ### Core Components The pool architecture consists of five primary components defined across the application manager and orchestrator modules: - **Async-Reuse Thread Pool**: Executes the reuse path (`engineReuseService.reuseEngine`) in a non-blocking manner. This pool is instantiated in `DefaultEngineAskEngineService` using `Utils.newCachedExecutionContextWithExecutor` at lines 81-86. The maximum thread size is controlled by `wds.linkis.manager.reuse.max.thread.size` (default `200`), mapped to `AMConfiguration.REUSE_ENGINE_ASYNC_MAX_THREAD_SIZE` in [`AMConfiguration.java`](https://github.com/apache/linkis/blob/main/AMConfiguration.java). - **Async-Create Thread Pool**: Handles the creation path (`engineCreateService.createEngine`) when reuse fails. Created at lines 88-93 in [`DefaultEngineAskEngineService.scala`](https://github.com/apache/linkis/blob/main/DefaultEngineAskEngineService.scala), this pool uses the configuration `wds.linkis.manager.create.max.thread.size` (default `200`), defined as `AMConfiguration.CREATE_ENGINE_ASYNC_MAX_THREAD_SIZE`. - **Async-Error-Send Thread Pool**: Transmits asynchronous error responses back to the entrance when futures fail. Initialized at lines 95-100 in `DefaultEngineAskEngineService`, it reads from `wds.linkis.manager.ask.error.max.thread.size` (default value), exposed as `AMConfiguration.ASK_ENGINE_ERROR_ASYNC_MAX_THREAD_SIZE`. - **EngineConnExecutor Cache**: Stores `EngineConnExecutor` objects keyed by `ServiceInstance` to enable rapid reuse. The abstract `EngineConnManager` class maintains this as `engineConnExecutorCache` (a `ConcurrentHashMap`) at lines 105-113 in [`EngineConnManager.scala`](https://github.com/apache/linkis/blob/main/EngineConnManager.scala). Cache boundaries are governed indirectly by reuse-count limits and cache enablement flags. - **Engine-Reuse Semaphore**: Limits concurrent reuse attempts per engine type and tenant using a token-bucket pattern. The `DefaultEngineAskEngineService.getKeyAndSemaphore` method (lines 60-88) builds a `Semaphore` per `engineCreateKey` and stores it in `engineCreateSemaphoreMap`. The default limits are configured via `linkis.am.engine.ask.max.number` (e.g., `appconn=5,trino=10`), accessible as `AMConfiguration.AM_ENGINE_ASK_MAX_NUMBER` at lines 174-188 in [`AMConfiguration.java`](https://github.com/apache/linkis/blob/main/AMConfiguration.java). ### Request Flow Through the Pool When an entrance service requests an engine, the Linkis EngineConn connection pool processes the request through the following async flow: 1. **Entrance** invokes `askEngine` on `DefaultEngineAskEngineService`. 2. The service launches a **reuse future** using the **async-reuse thread pool** to execute `engineReuseService.reuseEngine`. 3. If a usable `EngineNode` is returned, the engine is assigned immediately. If `null` is returned, a **create future** is launched on the **async-create thread pool**. 4. The create future obtains a semaphore token (per engine-type/tenant), calls `engineCreateService.createEngine`, registers the new engine in the `EngineConnExecutorCache`, and releases the semaphore. 5. Any exception from either future is handed to the **async-error-send thread pool**, which pushes an `EngineCreateError` back to the entrance. This architecture decouples reuse lookup from engine creation, allowing the system to handle high concurrency without blocking entrance threads. ## Tuning Engine Reuse Parameters for Optimal Performance Fine-tuning the Linkis EngineConn connection pool requires adjusting thread-pool capacities, concurrency limits, and cache lifecycles in `linkis-application-manager.properties`. ### Thread Pool Sizing To increase parallel reuse handling, modify `wds.linkis.manager.reuse.max.thread.size` (default `200`). Raising this value allows more concurrent reuse attempts during burst traffic, though you must balance against available CPU and memory resources. For faster fallback creation when reuse fails, adjust `wds.linkis.manager.create.max.thread.size` (default `200`). This parameter maps to `AMConfiguration.CREATE_ENGINE_ASYNC_MAX_THREAD_SIZE` and determines how many engines can be instantiated simultaneously. The error-sending capacity is controlled by `wds.linkis.manager.ask.error.max.thread.size`, which should be sized proportionally to the sum of reuse and create pools to prevent backlog during failure storms. ### Per-Engine Concurrency Limits Prevent resource monopolization by configuring `linkis.am.engine.ask.max.number` with comma-separated key-value pairs (e.g., `appconn=10,trino=20,spark=8`). This setting populates `AMConfiguration.AM_ENGINE_ASK_MAX_NUMBER` and defines the semaphore permits available per engine type and tenant. ### Cache Lifecycle and Reuse Limits Control how long idle engines remain available for reuse with `wds.linkis.manager.am.engine.reuse.max.time` (default `5m`). Longer durations reduce creation overhead but increase resource consumption. Restrict total concurrent sharing per engine via `wds.linkis.manager.am.engine.reuse.count.limit` (default `2`). Higher values increase parallelism but may cause resource contention on memory-intensive engines like Spark. Enable fast metadata lookup by setting `wds.linkis.manager.am.engine.reuse.enable.cache` to `true` (default `false`). When enabled, tune `wds.linkis.manager.am.engine.reuse.cache.expire.time` (default `5s`) and `wds.linkis.manager.am.engine.reuse.cache.max.size` (default `1000`) to balance lookup latency against memory usage. ### Practical Tuning Workflow 1. **Profile current load** by monitoring logs from `DefaultEngineAskEngineService` that report `reuseExecutor: poolSize: X, activeCount: Y, queueSize: Z`. 2. **Increase thread-pool sizes** only when `queueSize` frequently exceeds 70% of the pool capacity. 3. **Adjust reuse-max-time** based on average task duration—use shorter timeouts for ephemeral jobs and longer values for batch workloads. 4. **Set engineReuseCountLimit** according to engine resource footprints; keep limits low for heavy engines to prevent memory exhaustion. 5. **Enable caching** in production clusters showing high reuse lookup latency, verifying memory impact through JVM metrics. ## Monitoring and Configuration Examples ### Monitoring Thread Pool Metrics Retrieve real-time statistics from the running service to assess pool health: ```scala import org.apache.linkis.manager.am.service.engine.DefaultEngineAskEngineService // Assuming injected bean reference val reusePool = engineAskService.reuseThreadPool // ThreadPoolExecutor val createPool = engineAskService.createThreadPool // ThreadPoolExecutor val errorPool = engineAskService.errorSendThreadPool println(s"Reuse pool – size:${reusePool.getPoolSize} active:${reusePool.getActiveCount} queue:${reusePool.getQueue.size()}") println(s"Create pool – size:${createPool.getPoolSize} active:${createPool.getActiveCount} queue:${createPool.getQueue.size()}") println(s"Error pool – size:${errorPool.getPoolSize} active:${errorPool.getActiveCount} queue:${errorPool.getQueue.size()}") ``` ### Updating Configuration at Runtime While runtime updates are possible, they only affect new thread-pool instantiations: ```java import org.apache.linkis.manager.am.conf.AMConfiguration; // Example: doubling the reuse thread pool size int current = AMConfiguration.REUSE_ENGINE_ASYNC_MAX_THREAD_SIZE; int newSize = current * 2; CommonVars.apply("wds.linkis.manager.reuse.max.thread.size", newSize); ``` **Note**: Existing pools retain their initial size. Restart the **linkis-application-manager** service after modifying `linkis-application-manager.properties` for consistent behavior. ### Sample Production Configuration ```properties # Thread pools wds.linkis.manager.reuse.max.thread.size=400 wds.linkis.manager.create.max.thread.size=400 wds.linkis.manager.ask.error.max.thread.size=150 # Engine reuse behavior wds.linkis.manager.am.engine.reuse.max.time=10m wds.linkis.manager.am.engine.reuse.count.limit=4 wds.linkis.manager.am.engine.reuse.enable.cache=true wds.linkis.manager.am.engine.reuse.cache.max.size=5000 wds.linkis.manager.am.engine.reuse.cache.expire.time=30s # Per-engine concurrency limits linkis.am.engine.ask.max.number=appconn=10,trino=20,spark=8 ``` ## Summary - The Linkis EngineConn connection pool uses **three dedicated thread pools** (reuse, create, error) rather than traditional JDBC pooling, implemented in `DefaultEngineAskEngineService`. - **EngineConnExecutorCache** in `EngineConnManager` provides in-memory storage of reusable engine instances keyed by `ServiceInstance`. - **Semaphore-based limits** per engine type prevent resource monopolization, configured via `linkis.am.engine.ask.max.number` in [`AMConfiguration.java`](https://github.com/apache/linkis/blob/main/AMConfiguration.java). - **Thread pool sizes** default to `200` for reuse and creation paths, tunable through `wds.linkis.manager.reuse.max.thread.size` and `wds.linkis.manager.create.max.thread.size`. - **Reuse lifecycle** is governed by `wds.linkis.manager.am.engine.reuse.max.time` and `wds.linkis.manager.am.engine.reuse.count.limit`, while optional caching improves lookup performance. - Configuration changes require service restart to recreate thread pools with new dimensions. ## Frequently Asked Questions ### How does the Linkis EngineConn connection pool differ from a standard JDBC connection pool? Standard JDBC pools maintain a set of database connections ready for immediate checkout, whereas the Linkis EngineConn connection pool manages **engine-instance executors** through asynchronous pathways. The system uses separate thread pools for reuse attempts and creation fallbacks, coordinated by semaphores and an in-memory cache, because engine startup involves complex initialization that must not block entrance threads. ### What is the default size of the async reuse thread pool and when should I increase it? The default size is **200 threads**, defined by `wds.linkis.manager.reuse.max.thread.size` in [`AMConfiguration.java`](https://github.com/apache/linkis/blob/main/AMConfiguration.java). Increase this value when monitoring logs from `DefaultEngineAskEngineService` show the `queueSize` consistently exceeding 70% of the pool capacity, indicating that incoming requests are waiting for available threads to process reuse attempts. ### How do I prevent a single engine type from monopolizing the reuse pool? Configure the **engine-reuse semaphore** using `linkis.am.engine.ask.max.number` (e.g., `spark=8,trino=20`). This parameter, processed by `DefaultEngineAskEngineService.getKeyAndSemaphore`, creates distinct semaphores per engine type and tenant, ensuring that resource-heavy engines cannot exhaust the global thread pool capacity. ### Why are my engine reuse attempts failing even when engines are idle? Reuse failures typically occur when engines exceed the **reuse count limit** (`wds.linkis.manager.am.engine.reuse.count.limit`, default `2`) or the **reuse max time** (`wds.linkis.manager.am.engine.reuse.max.time`, default `5m`). Additionally, if `wds.linkis.manager.am.engine.reuse.enable.cache` is disabled, the lookup latency may cause timeout failures before the system identifies available engines. Verify these settings in `AMConfiguration` and adjust based on your job duration patterns.