# How to Deploy AgentScope Agents as a Serverless Service or on Kubernetes

> Deploy AgentScope agents to serverless platforms or Kubernetes using the official Docker image. Learn how to package your agent code for easy deployment and scalability.

- Repository: [AgentScope-AI/agentscope](https://github.com/agentscope-ai/agentscope)
- Tags: how-to-guide
- Published: 2026-03-09

---

**You can deploy AgentScope agents to serverless platforms like Google Cloud Run or AWS Lambda, as well as Kubernetes clusters, by packaging your agent code with the official `agentscope-runtime` Docker image.**

AgentScope is an open-source multi-agent framework ([agentscope-ai/agentscope](https://github.com/agentscope-ai/agentscope)) that lets you build AI agents in pure Python. Because the SDK ships as a containerized runtime, you can deploy AgentScope agents to any cloud platform that supports containers, from fully managed serverless environments to self-managed Kubernetes clusters.

## Architecture Overview

AgentScope deployments consist of four core components working together to expose your agent via HTTP:

- **Agent code**: Your custom Python script that imports `agentscope` and defines a `run()` entry point. The repository provides a ready-to-use example in [`examples/deployment/planning_agent/main.py`](https://github.com/agentscope-ai/agentscope/blob/main/examples/deployment/planning_agent/main.py) that demonstrates the standard structure expected by the runtime.
- **agentscope-runtime image**: A lightweight Docker image based on Python 3.11 that pre-installs the `agentscope` package, selected LLM providers (OpenAI, Ollama), and optional tooling like vector stores or TTS. The image is available on Docker Hub as `agentscope/agentscope-runtime:latest`.
- **Container orchestrator**: The platform that executes the container. Serverless platforms like AWS Lambda (via container images), Google Cloud Run, or Azure Container Apps start the container on demand, while Kubernetes runs it inside a pod behind a Service.
- **Ingress / API gateway**: Exposes the agent’s HTTP endpoint (`/run` by default). Serverless platforms handle this automatically; in Kubernetes you configure an Ingress resource (NGINX, GKE-Ingress, etc.).

The request flow is straightforward: an HTTP request hits the Ingress, which routes to the container running the **agentscope-runtime**, which invokes your agent’s `run()` function and returns the response. The runtime also supports **Server-Sent Events** for streaming output, as validated in the `tests/realtime_*_test.py` files.

## Deploying to Serverless Platforms

Serverless deployment eliminates infrastructure management by running your container only when requests arrive. The following example uses Google Cloud Run, but the same Dockerfile works for AWS Lambda container images or Azure Container Apps.

### Step 1: Containerize Your Agent

Create a `Dockerfile` that copies your agent code into the runtime image:

```dockerfile
FROM agentscope/agentscope-runtime:latest

WORKDIR /app
COPY examples/deployment/planning_agent/main.py .
ENTRYPOINT ["python", "main.py"]

```

The runtime expects an entrypoint that exposes a `run` function decorated with `@agentscope.agent` or `@agentscope.tool`.

### Step 2: Build and Push

Build the image and push it to a container registry:

```bash
docker build -t gcr.io/<PROJECT_ID>/planning-agent:latest .
docker push gcr.io/<PROJECT_ID>/planning-agent:latest

```

### Step 3: Deploy to Cloud Run

Deploy the image to Cloud Run to automatically create an HTTPS endpoint:

```bash
gcloud run deploy planning-agent \
    --image gcr.io/<PROJECT_ID>/planning-agent:latest \
    --platform managed \
    --region us-central1 \
    --allow-unauthenticated

```

Cloud Run automatically scales to zero when idle and spins up instances on demand. The endpoint accepts JSON payloads matching your agent’s input schema as defined by the decorators in your source code.

## Deploying to Kubernetes

For persistent workloads or complex multi-agent systems requiring sidecars (vector databases, tool sandboxes), Kubernetes provides fine-grained control over scaling and networking.

### Step 1: Create the Deployment

Define a Deployment that runs your agent container. Reference the GitHub Container Registry image `ghcr.io/agentscope-ai/agentscope-runtime:latest`:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: planning-agent
spec:
  replicas: 2
  selector:
    matchLabels:
      app: planning-agent
  template:
    metadata:
      labels:
        app: planning-agent
    spec:
      containers:
        - name: agent
          image: ghcr.io/agentscope-ai/agentscope-runtime:latest
          command: ["python", "main.py"]
          env:
            - name: AGENTSCOPE_LOG_LEVEL
              value: "INFO"

```

### Step 2: Expose with a Service

Create a Service to route traffic to the pods on port **8000**, the default agentscope-runtime HTTP port:

```yaml
apiVersion: v1
kind: Service
metadata:
  name: planning-agent-svc
spec:
  selector:
    app: planning-agent
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000

```

### Step 3: Configure Ingress

Add an Ingress resource to terminate TLS and route external traffic:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: planning-agent-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
    - host: agentscope.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: planning-agent-svc
                port:
                  number: 80

```

Apply the manifests:

```bash
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml
kubectl apply -f ingress.yaml

```

Your agent is now reachable at `https://agentscope.example.com/run`. Scale horizontally with `kubectl scale deployment planning-agent --replicas=5` or add a **HorizontalPodAutoscaler** for automatic scaling based on CPU or request latency.

## Production Best Practices

Deploying AgentScope agents to production requires attention to security, observability, and performance.

- **Secret handling**: Store LLM API keys and database credentials as Kubernetes Secrets or mounted secrets in Cloud Run. Never hard-code credentials in your agent source code.
- **Observability**: Enable OpenTelemetry tracing by setting the `OTEL_SERVICE_NAME` environment variable. The runtime automatically emits traces for each request, as demonstrated in [`docs/tutorial/en/src/task_tracing.py`](https://github.com/agentscope-ai/agentscope/blob/main/docs/tutorial/en/src/task_tracing.py).
- **Low-latency LLM backends**: When using vLLM or OpenAI’s streaming API, configure the runtime with `--enable-auto-tool-choice` and `--tool-call-parser` flags documented in [`docs/tutorial/en/src/task_model.py`](https://github.com/agentscope-ai/agentscope/blob/main/docs/tutorial/en/src/task_model.py).
- **Tool sandboxing**: For agents executing arbitrary tools (browsers, VNC), use the `agentscope-runtime` sandbox that isolates processes in separate user namespaces. Examples are available in `examples/functionality/vector_store/` and `examples/functionality/tts/`.
- **Graceful shutdown**: The runtime listens for SIGTERM and finishes in-flight requests before exiting, ensuring zero-downtime rolling updates in Kubernetes.

## Summary

- AgentScope agents are standard Python programs that run inside the `agentscope-runtime` container image.
- You can deploy to serverless platforms (Cloud Run, AWS Lambda) by building a custom image and pushing it to a registry.
- Kubernetes deployments use standard Deployment, Service, and Ingress manifests, with the runtime listening on port **8000**.
- Production deployments require proper secret management, OpenTelemetry tracing, and graceful shutdown handling for high availability.

## Frequently Asked Questions

### Can I deploy AgentScope agents on AWS Lambda?

Yes. AWS Lambda supports container images up to 10 GB. Build your image using the `agentscope-runtime` base, push it to Amazon ECR, and configure the Lambda function to use the container image instead of a ZIP deployment. Ensure your `run()` handler conforms to Lambda’s invocation interface or use an adapter like `aws-lambda-web-adapter` for HTTP-mode functions.

### How do I handle scaling for high-traffic agent workloads?

For serverless platforms, scaling is automatic based on concurrent request limits you configure. In Kubernetes, use a **HorizontalPodAutoscaler** targeting CPU or custom metrics (request latency). The runtime is stateless, so you can safely run multiple replicas behind a load balancer. For stateful agents requiring shared memory, deploy a Redis sidecar or use a centralized vector store like Milvus (examples in `examples/functionality/vector_store/`).

### Can I use local LLMs like Ollama when deploying to the cloud?

Yes. Include Ollama in your container image or deploy it as a separate sidecar container in the same Kubernetes pod. Configure your agent to point to `localhost:11434` (or the appropriate service name if using a sidecar pattern). For serverless deployments, consider using vLLM with the flags documented in [`docs/tutorial/en/src/task_model.py`](https://github.com/agentscope-ai/agentscope/blob/main/docs/tutorial/en/src/task_model.py) for efficient GPU utilization.

### What is the difference between serverless and Kubernetes deployment for AgentScope?

**Serverless** (Cloud Run, Lambda) is optimal for event-driven, bursty traffic with cost-effective scale-to-zero behavior, but imposes request timeout limits (e.g., 60 minutes for Cloud Run, 15 minutes for Lambda). **Kubernetes** is better for long-running agents, persistent connections (WebSockets, Server-Sent Events), and complex topologies requiring sidecars for vector databases or tool sandboxes. Both use the same `agentscope-runtime` image and agent code.