How to Troubleshoot Document Parsing Failures for Specific File Types in AnythingLLM

To troubleshoot document parsing failures in AnythingLLM, verify the Collector service is online at the /health endpoint, confirm your file extension appears in the Collector's /accepts list, ensure requests use the MIME type application/anythingllm-document, and inspect collector/logs/collector.log for specific parser errors.

AnythingLLM relies on a dedicated Collector service to transform uploaded files into plain text chunks for vector indexing. When document parsing fails for specific file types, the issue typically stems from service connectivity, unsupported extensions, malformed request headers, or file corruption. Understanding the code path from the upload endpoint through the Collector API helps you diagnose whether the failure occurs before processing or during the actual parsing phase.

Understanding the Document Parsing Architecture

The parsing process begins at POST /workspace/:slug/parse in server/endpoints/workspacesParsedFiles.js. This endpoint retrieves the uploaded file and immediately checks Collector health via Collector.online() before invoking Collector.parseDocument(originalname) defined in server/utils/collectorApi/index.js.

The Collector API wrapper constructs a signed POST request to http://0.0.0.0:<COLLECTOR_PORT>/parse with security headers X-Integrity and X-Payload-Signer. The Collector service then reads the file from collector/hotdir, validates the MIME type against its internal accepted types list via the /accepts endpoint, executes the appropriate parser (PDF-ium, docx, txt, etc.), and returns a JSON response containing success, documents, and optional reason fields.

Step-by-Step Troubleshooting Workflow

Verify Collector Service Availability

Before investigating file-specific issues, confirm the Collector process is reachable. In server/utils/collectorApi/index.js, the online() method pings the Collector's health endpoint.

Run this command to check status:

curl -s http://localhost:${COLLECTOR_PORT:-8888}/health || echo "Collector offline"

If the request fails, restart the Collector container with docker compose up collector or the local Node process. The parsing chain cannot proceed if Collector.online() returns false.

Validate Supported File Types

The Collector maintains an internal whitelist of accepted MIME types and extensions. Query the /accepts endpoint to verify support for your specific file type:

curl -s http://localhost:${COLLECTOR_PORT:-8888}/accepts | jq .

The response lists available parsers:

{
  "pdf": true,
  "docx": true,
  "txt": true,
  "pptx": false
}

If your extension shows false or is missing, the Collector rejects the request before parsing. Modify collector/src/constants.js to update the ACCEPTED_TYPES object, then restart the service.

Inspect Request Signing and Headers

The Collector requires signed headers for security. The parseDocument function in server/utils/collectorApi/index.js#L55-L69 adds X-Integrity and X-Payload-Signer headers. Malformed headers cause immediate rejection regardless of file validity.

Test the complete request flow:

FILENAME="my-report.pdf"
DATA=$(jq -n --arg fn "$FILENAME" '{filename:$fn, options:{whisperProvider:"local"}}')
curl -X POST http://localhost:${COLLECTOR_PORT:-8888}/parse \
  -H "Content-Type: application/json" \
  -H "X-Integrity: $(node -e "console.log(require('./server/utils/comKey').CommunicationKey.sign('$DATA')")" \
  -d "$DATA"

Analyze the JSON response. A reason field such as "Unsupported file type" or "Failed to open PDF" indicates the specific failure point.

Confirm File Presence in Hot Directory

The Collector reads files from collector/hotdir. After upload, verify the file exists:

ls -l collector/hotdir | grep my-report.pdf

If the file is absent, the upload middleware in server/utils/files/multer.js failed to transfer the file. Check disk space and directory permissions before proceeding.

Analyze Collector Logs

Runtime parsing errors appear in collector/logs/collector.log. Monitor this file while reproducing the error:

tail -f collector/logs/collector.log

Look for entries like:

[INFO] Received parse request for file my-report.pdf
[ERROR] PDF parsing failed: Invalid PDF header

These logs distinguish between file corruption and parser-specific implementation errors.

Common Failure Scenarios and Solutions

Frontend MIME Type Mismatch

AnythingLLM treats uploads as documents only when the MIME type is exactly application/anythingllm-document. The check occurs in server/utils/chats/apiChatHandler.js:

if (attachment.mime && attachment.mime.toLowerCase() === "application/anythingllm-document") { … }

If the client sends application/pdf or application/vnd.openxmlformats-officedocument.wordprocessingml.document, the server treats the file as an image and bypasses the Collector entirely. Ensure your upload client sets file.type = "application/anythingllm-document" in the FormData payload.

Corrupt or Password-Protected Files

When the Collector receives a damaged PDF or encrypted DOCX, the parser throws internal errors visible in the logs. Test with a known-good plain-text file:

echo "Test content" > collector/hotdir/test.txt

If test.txt parses successfully but my-report.pdf fails, verify the PDF opens locally and contains no password protection or structural damage.

Programmatic Debugging Example

Use the Collector API wrapper directly to isolate issues without the HTTP interface:

const { CollectorApi } = require('./server/utils/collectorApi');

(async () => {
  const collector = new CollectorApi();
  if (!await collector.online()) {
    console.error('Collector unreachable');
    return;
  }
  
  const { success, reason, documents } = await collector.parseDocument('sample.pdf');
  if (!success) {
    console.error('Parse error:', reason);
    process.exit(1);
  }
  console.log(`Parsed ${documents.length} chunks`);
})();

This script replicates the exact logic used in server/endpoints/workspacesParsedFiles.js without the web server overhead.

Summary

• Verify connectivity: Use Collector.online() or curl the /health endpoint before debugging file issues.
• Check acceptance lists: Query /accepts to confirm the Collector supports your file extension as defined in collector/src/constants.js.
• Validate MIME types: Ensure uploads use application/anythingllm-document to trigger document parsing rather than image handling in server/utils/chats/apiChatHandler.js.
• Inspect hot directory: Confirm files reach collector/hotdir via the Multer middleware in server/utils/files/multer.js.
• Review logs: Check collector/logs/collector.log for parser-specific error messages and stack traces.
• Test isolation: Use simple text files to determine if the issue is file-specific or systemic.

Frequently Asked Questions

Why does my PDF return "success: false" but works in other applications?

The Collector uses strict parsers like PDF-ium that reject malformed PDF structures. Open the file locally and re-save it to rebuild the header. Check collector/logs/collector.log for "Invalid PDF header" errors to confirm corruption.

How do I add support for new file extensions like PPTX?

Edit collector/src/constants.js and add the extension to the ACCEPTED_TYPES object with a true value. Restart the Collector service and verify via the /accepts endpoint that the new type appears in the supported list.

What causes "X-Integrity header missing" errors?

This occurs when the X-Integrity or X-Payload-Signer headers are missing or malformed. The Collector API wrapper in server/utils/collectorApi/index.js automatically generates these using CommunicationKey.sign(). Ensure your custom requests include properly signed headers matching the implementation in the source code.

Why does the server treat my document as an image?

AnythingLLM checks the MIME type in server/utils/chats/apiChatHandler.js. If the upload lacks the application/anythingllm-document MIME type, the system routes the file to image processing instead of the Collector. Verify your frontend sets this exact MIME string in the FormData request payload.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}