How JVM Agent Injection Works for Java Chaos Engineering in ChaosBlade

JVM agent injection in ChaosBlade works by attaching a lightweight Java sandbox agent to a running target process, exposing a local HTTP control interface that receives fault-instruction payloads via POST requests.

ChaosBlade implements JVM agent injection through a privileged attachment flow that requires either a process name or PID to identify the target Java application. The mechanism relies on the Java Attach API to load sandbox-agent.jar into the target's address space, after which the agent opens a localhost HTTP server used to trigger, monitor, and destroy chaos experiments without restarting the application.

The Injection Pipeline

The attachment flow is orchestrated by PrepareJvmCommand in cli/cmd/prepare_jvm.go. It delegates the actual work to jvm.CheckFlagValues and jvm.Attach, spinning up a sidecar process that embeds an HTTP API inside the target JVM.

Parameter Validation and Process Discovery

Before any code runs, CheckFlagValues (defined in exec/jvm/executor.go at line 302) validates that the operator supplied either --process or --pid. If a process name is provided, ChaosBlade resolves it to an actual PID.

Once the target is identified, the system determines ownership. getUsername and getUserid (see exec/jvm/sandbox.go lines 200‑230) inspect /proc/<pid>/status to extract the Uid and running user. This identity is critical because the sandbox must be started with the same privileges as the target JVM, otherwise the Attach API throws an AccessDeniedException.

Runtime Resolution and Token Generation

Next, getJavaBinAndJavaHome (sandbox.go lines 332‑358) locates the correct java binary. If JAVA_HOME is undefined in the environment, the function parses the target process’s command line (/proc/<pid>/cmdline) to infer the JDK path.

A unique session identifier is then created. getSandboxToken (sandbox.go lines 181‑188) generates a random token by shelling out to a tiny checksum command; this token is later written to ~/.sandbox.token and used to authenticate HTTP requests.

The tool also resolves the tools.jar dependency via getToolJar (sandbox.go lines 191‑199). It first attempts to use the tools.jar provided by the target JDK; if unavailable, it falls back to the bundled copy under $CHAOSBLADE_HOME/lib/sandbox.

Privilege Escalation and Agent Attachment

With the runtime and token ready, getAttachJvmOpts (sandbox.go lines 170‑178) builds the final JVM flags:

-Xms128M -Xmx128M -Xnoclassgc -ea -Xbootclasspath/a:<tools.jar>
-jar <sandbox-lib>/sandbox-core.jar <pid> "home=<sandboxHome>;token=<token>;server.ip=127.0.0.1;server.port=<port>;namespace=chaosblade"

The home=… parameter passes configuration context to sandbox-agent.jar, while token and port establish the HTTP control plane.

Because the ChaosBlade CLI often runs as root or a different user, the execution layer must switch credentials. The code in exec/jvm/sandbox.go (lines 39‑48) checks checkSudoAvailable; if present, it prefixes the command with sudo -u <username>, otherwise it falls back to su - <username> -c '…'. If JAVA_TOOL_OPTIONS is detected in the environment, it is explicitly unset for the child process to prevent interference.

Finally, channel.NewLocalChannel().Run executes the constructed command. On success, the sandbox writes its authentication token to ~/.sandbox.token, and the attach function verifies responsiveness by querying http://127.0.0.1:<port>/sandbox/chaosblade/module/http/chaosblade/status.

HTTP Control Interface and Experiment Execution

Once the JVM agent injection is complete, the sandbox exposes a REST API on a configurable localhost port. The Executor in exec/jvm/executor.go converts CLI flags into JSON payloads and POSTs them to:

http://127.0.0.1:<port>/sandbox/chaosblade/module/http/chaosblade/create

The payload includes the experiment target (e.g., jvm), action (e.g., delay), and specific flags (method name, line number, delay duration). The agent parses this inside the target JVM using bytecode manipulation libraries, injects the fault, and returns a spec.Response indicating success or failure.

Detaching and Cleanup

To remove the agent, the Detach (alias shutdown) operation sends a request to:

http://127.0.0.1:<port>/sandbox-control/shutdown

This triggers the sandbox to unload itself from the target JVM and delete the ~/.sandbox.token file, returning the process to its original state without restart.

Practical Implementation Examples

CLI Preparation

Attach the sandbox to a Tomcat process named "tomcat" using the built-in prepare command:

blade prepare jvm --process tomcat

This resolves the PID, escalates privileges if necessary, injects the agent, and returns a preparation UID that subsequent experiments can reference.

Programmatic Attachment

The same flow can be invoked programmatically via the ChaosBlade Go SDK:

import (
    "context"
    "github.com/chaosblade-io/chaosblade/exec/jvm"
)

func attachToProcess(ctx context.Context, processName string) (string, error) {
    // Validate flags and resolve PID
    pid, resp := jvm.CheckFlagValues(ctx, processName, "")
    if !resp.Success {
        return "", resp.Err
    }

    // Attach agent; port "0" lets the system select an available port
    agentResp, user, uid := jvm.Attach(ctx, "0", "", pid)
    if !agentResp.Success {
        return "", agentResp.Err
    }
    
    // agentResp.Result contains the port number
    return agentResp.Result, nil
}

Triggering Faults

With the agent running, execute a CPU load experiment by POSTing to the local HTTP interface:

import (
    "context"
    "github.com/chaosblade-io/chaosblade/exec/jvm"
    "github.com/chaosblade-io/chaosblade-spec-go/spec"
)

func injectCPU(ctx context.Context, port string) (*spec.Response, error) {
    exec := jvm.NewExecutor()
    model := &spec.ExpModel{
        Target: "jvm",
        ActionName: "cpufullload",
        ActionFlags: map[string]string{
            "cpu-count": "2",
            "duration":  "60s",
        },
    }
    
    // Exec sends the request to the sandbox HTTP endpoint
    return exec.Exec(port, ctx, model), nil
}

Detaching the Agent

Clean up the sandbox when experiments are finished:

import "github.com/chaosblade-io/chaosblade/exec/jvm"

func cleanup(ctx context.Context, port string) error {
    resp := jvm.Detach(ctx, port)
    if !resp.Success {
        return resp.Err
    }
    return nil
}

Summary

• JVM agent injection requires either a process name or PID, validated by CheckFlagValues in exec/jvm/executor.go.
• Identity resolution (getUsername, getUserid) and privilege switching (sudo/su) ensure the sandbox runs with the same rights as the target JVM.
• getJavaBinAndJavaHome and getToolJar locate the JDK and tools.jar, while getSandboxToken generates a session token stored in ~/.sandbox.token.
• The attachment command built by getAttachJvmOpts launches sandbox-core.jar, which loads sandbox-agent.jar into the target process and opens an HTTP control plane on 127.0.0.1.
• Experiments are created by POSTing JSON to /sandbox/chaosblade/module/http/chaosblade/create and terminated by calling /sandbox-control/shutdown.

Frequently Asked Questions

What permissions are required for JVM agent injection?

The ChaosBlade process must be able to switch to the same user that owns the target Java process. If running as root, the tool automatically uses sudo -u <user> or su -c to drop privileges. Without root, the CLI must be executed by the identical user running the JVM, otherwise the Java Attach API will refuse the connection.

How does ChaosBlade verify the agent started successfully?

After executing the attachment command, the code performs a blocking check by sending an HTTP GET to http://127.0.0.1:<port>/sandbox/chaosblade/module/http/chaosblade/status. It also greps the generated token from ~/.sandbox.token to confirm the sandbox wrote its startup credentials. If either step fails, the preparation is marked as failed and the error is returned to the CLI.

Can I attach the agent to a PID instead of a process name?

Yes. The CheckFlagValues function accepts an empty process name when a PID is supplied. Provide the --pid flag to blade prepare jvm (or the equivalent pid argument in the SDK), and the validation logic skips process name resolution and uses the supplied integer directly.

Where is the sandbox agent code located in the repository?

The core attachment logic resides in exec/jvm/sandbox.go, which handles token generation, privilege escalation, and JVM option construction. The high-level orchestration (CLI command, async mode, and metadata recording) is implemented in cli/cmd/prepare_jvm.go, while the HTTP client that sends experiment payloads to the agent is defined in exec/jvm/executor.go.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}