How to Configure Google Drive Integration for Automatic Result Backup in Twinkle Eval

Enable the google_services.google_drive block in config.yaml, provide a service account JSON file, and Twinkle Eval will automatically upload logs and results to a timestamped Google Drive folder after every evaluation run.

Twinkle Eval is an open-source evaluation framework that supports automatic archival of experiment artifacts. By configuring the Google Drive integration, teams can ensure evaluation logs and result files are immediately backed up to cloud storage without manual intervention. This guide walks through the complete setup using the actual implementation in the ai-twinkle/eval repository.

Prerequisites: Google Cloud Credentials

Before modifying configuration files, you must obtain valid Google Cloud credentials. Twinkle Eval supports two authentication methods as implemented in GoogleDriveUploader.__init__ within twinkle_eval/google_services.py (lines 17-240).

Service Account (Recommended for CI/CD):

• Create a service account in Google Cloud Console under IAM & Admin → Service Accounts
• Generate a JSON key file containing project_id, private_key, client_email, and token_uri
• Share your target Drive folder with the service account email (e.g., name@project.iam.gserviceaccount.com) granting Editor permissions

OAuth 2.0 (Interactive Use):

• Configure a "Desktop App" OAuth 2.0 client in Google Cloud Console
• Download the client secrets JSON file
• Use this method only for local development where browser-based authentication is feasible

Step-by-Step Configuration

1. Update config.yaml

Add the Google Drive configuration block to your config.yaml file. The schema is defined in twinkle_eval/config.template.yaml (lines 61-69) and validated by ConfigurationManager in twinkle_eval/config.py (lines 210-274).

google_services:
  google_drive:
    enabled: true
    auth_method: "service_account"  # or "oauth"

    credentials_file: "service_account.json"  # Relative path to repo root

    log_folder_id: "1AbCdEFgHiJKlMnOpQrStU"   # Optional: Drive folder ID

If log_folder_id is omitted, files upload to the Drive root. The credentials_file path is resolved relative to the working directory where Twinkle Eval executes.

2. Validate the Configuration

Run a configuration dry-run to verify your credentials file is accessible and properly formatted:

python -m twinkle_eval.main --config config.yaml --list-exporters

The ConfigurationManager._validate_google_drive_config() method (lines 262-285 in config.py) checks for file existence and JSON validity. Missing or malformed credentials trigger a ConfigurationError with specific details about the failure.

3. Execute an Evaluation

Run your evaluation as normal:

twinkle-eval --config config.yaml --export json csv

Upon completion, TwinkleEval._handle_google_services() (lines 41-56 in twinkle_eval/main.py) instantiates GoogleDriveUploader and calls upload_latest_files(start_time, "logs", "results"). The console displays confirmation messages:

成功建立資料夾: Eval_20240223_1542 (1AbCdEFgHiJKlMnOpQrStU)
成功上傳 2 個檔案到 Google Drive
  - log: eval_20240223_1542.log
  - results: results_20240223_1542.json

Technical Implementation Details

The integration follows a three-component architecture:

Configuration Loading: ConfigurationManager.load_config() parses config.yaml and validates the google_services.google_drive section using _validate_google_drive_config(). This ensures the credentials file exists before any evaluation begins.

Authentication Flow: GoogleDriveUploader._authenticate() builds the Google Drive API service using googleapiclient.discovery.build(). For service accounts, it uses google.oauth2.service_account.Credentials.from_service_account_file().

Upload Logic: The upload_latest_files() method (starting at line 240 in google_services.py) performs three operations:

1. Creates a timestamped folder named Eval_<start_time> inside the configured log_folder_id
2. Identifies the most recent log file matching the start time
3. Uploads matching result files from the results directory

All operations include error handling and logging via log_info() and log_error() calls, ensuring permission failures or network issues are immediately visible in the console output.

Advanced Usage: Manual Upload Trigger

For custom workflows, instantiate the uploader directly without running a full evaluation:

from twinkle_eval.google_services import GoogleDriveUploader

drive_config = {
    "enabled": True,
    "auth_method": "service_account",
    "credentials_file": "service_account.json",
    "log_folder_id": "1AbCdEFgHiJKlMnOpQrStU",
}

uploader = GoogleDriveUploader(drive_config)
result = uploader.upload_latest_files(
    start_time="20240223_1542",
    logs_directory="logs",
    results_directory="results"
)

print(result["folder_id"])      # Drive ID of created folder

print(result["uploaded_files"]) # List of uploaded file names

This approach is useful for backfilling historical results or integrating Twinkle Eval artifacts into external pipeline orchestrators.

Summary

• Configuration: Add the google_services.google_drive block to config.yaml with enabled: true, auth_method, and credentials_file paths.
• Permissions: Service account emails require explicit sharing access to the target Drive folder.
• Automation: The GoogleDriveUploader class handles authentication, folder creation, and file uploads automatically after each evaluation run via _handle_google_services() in main.py.
• Validation: Use --list-exporters to verify configuration without running a full evaluation cycle.

Frequently Asked Questions

What file types does Twinkle Eval upload to Google Drive?

Twinkle Eval uploads the most recent log files and any result files whose filenames contain the evaluation's start_time timestamp. This typically includes .log files from the logs directory and exported results such as .json or .csv files from the results directory.

Can I use OAuth instead of a service account?

Yes. Set auth_method: "oauth" in config.yaml and provide the path to your OAuth client secrets JSON file. However, this requires interactive browser authentication during the first run, making it unsuitable for automated CI/CD pipelines where service accounts are preferred.

Why do I get a ConfigurationError about missing credentials?

The _validate_google_drive_config() method in twinkle_eval/config.py verifies that the credentials_file path exists and contains valid JSON. Ensure the path is relative to your execution directory, or use an absolute path. Also verify the service account has not been deleted or disabled in Google Cloud Console.

How do I find my Google Drive folder ID?

Navigate to your target folder in Google Drive web interface. The URL contains the folder ID after /folders/. For example, in https://drive.google.com/drive/folders/1AbCdEFgHiJKlMnOpQrStU, the ID is 1AbCdEFgHiJKlMnOpQrStU. Share this folder with your service account email and grant Editor permissions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}