How Lemon AI's File Type Response System Detects and Handles Different Formats

Lemon AI detects file formats by examining the file extension in the frontend, then instructs the HTTP client to return either a parsed JSON object or a binary Blob based on an internal whitelist of extensions.

The file type response system in the hexdocom/lemonai repository automatically determines how to transport file contents from the server to the client. By inspecting file extensions before making API calls, the system ensures that binary assets like PDFs and images are handled as downloadable Blobs while configuration files remain as parsed JSON objects.

Extension-Based Detection in the Frontend

The detection logic resides entirely in the frontend utility layer, where filename strings are parsed to determine the appropriate transport format.

The getFileReponseTypeByName Function

In frontend/src/utils/file.js, the getFileReponseTypeByName function extracts the substring after the last dot in the filename and checks it against an internal whitelist named blobTypeDict. This whitelist contains common binary extensions such as png, jpg, pdf, docx, and xlsx. If the extension exists in the whitelist, the function returns the string 'blob'; otherwise, it returns 'json'.

// https://github.com/hexdocom/lemonai/blob/main/frontend/src/utils/file.js#L7-L13
function getFileReponseTypeByName(filename){
    const fileExtendName = filename.split('.').pop();
    if(blobTypeDict.includes(fileExtendName)){
        return 'blob';
    }else{
        return 'json';
    }
};

Configuring the HTTP Client Response Type

Once the response type is determined, it must be communicated to the HTTP client so that the underlying axios instance parses the payload correctly.

Workspace Service Integration

The frontend/src/services/workspace.js module imports the file utility and invokes getFileReponseTypeByName before issuing the POST request to /api/file/read. The returned value ('blob' or 'json') is passed as the fourth argument to the HTTP wrapper, ensuring the client expects the correct data format.

// https://github.com/hexdocom/lemonai/blob/main/frontend/src/services/workspace.js#L22-L27
const baseUrl = `/api/file/read`;
const responseType = fileUtils.getFileReponseTypeByName(path)
const response = await http.post(baseUrl, { path }, {}, responseType);

Axios Wrapper Implementation

In frontend/src/utils/http.js, the post method accepts a responseType parameter that defaults to 'json'. This value is forwarded directly to the axios request options. When 'blob' is supplied, axios returns a raw Blob object instead of parsing the response body as JSON, allowing binary data to remain intact for client-side processing.

// https://github.com/hexdocom/lemonai/blob/main/frontend/src/utils/http.js#L73-L80
post(url, params, header = {}, responseType='json') {
    const options = { url, method: "POST", data: params,
        headers: Object.assign({ 'Content-Type': 'application/json' }, header),
        responseType: responseType,
    }
    return instance.request(options);
},

Server-Side File Streaming

The backend remains agnostic to file types. In src/routers/file/file.js, the /api/file/read endpoint creates a read stream using fs.createReadStream and pipes the raw bytes to the response. The server does not inspect MIME types or extensions; it simply streams the file as-is, leaving interpretation entirely to the frontend’s predetermined response type.

// https://github.com/hexdocom/lemonai/blob/main/src/routers/file/file.js#L66-L78
router.post('/read', async ({ request, response }) => {
    const { path: filePath } = request.body;
    …
    const stream = fs.createReadStream(filePath);
    response.file(path.basename(filePath), stream);
});

Handling Binary Downloads

When the response type is 'blob', the frontend must convert the received Blob into a downloadable file. The handleFileDownload function in frontend/src/utils/file.js performs this by creating a temporary anchor element, setting the href to an object URL created from the Blob, and triggering a click event to open the browser’s save dialog. It also maps file extensions to MIME types to ensure the downloaded file receives the correct content type.

// https://github.com/hexdocom/lemonai/blob/main/frontend/src/utils/file.js#L16-L74
async function handleFileDownload(file) {
    …
    const mimeType = mimeTypes[fileExt] || "application/octet-stream";
    const response = await fetch('/api/file/read', { … });
    const fileContent = await response.blob();
    const blob = new Blob([fileContent], { type: mimeType });
    const url = window.URL.createObjectURL(blob);
    const link = document.createElement('a');
    link.href = url; link.download = fileName;
    …
}

Summary

• Extension whitelist in blobTypeDict determines whether a file is transported as 'blob' or 'json'.
• getFileReponseTypeByName parses the filename and returns the appropriate response type string.
• Workspace service passes the response type to the HTTP wrapper before calling /api/file/read.
• Axios configuration receives the responseType parameter to parse binary data as a Blob instead of JSON.
• Server-side endpoint streams raw bytes without type inspection, relying on the client to interpret the format.
• Download helper converts received Blobs into downloadable files using temporary object URLs and anchor elements.

Frequently Asked Questions

How does Lemon AI determine whether to return JSON or Blob?

Lemon AI checks the file extension against an internal whitelist (blobTypeDict) defined in frontend/src/utils/file.js. If the extension matches known binary formats such as PNG, PDF, or DOCX, the system requests a Blob; otherwise, it defaults to JSON.

What file extensions are treated as binary blobs?

The whitelist includes common binary formats such as PNG, JPG, PDF, DOCX, and XLSX. Any extension not explicitly listed in blobTypeDict is treated as JSON data.

How does the backend handle different file types?

The backend endpoint in src/routers/file/file.js treats all files identically, streaming raw bytes using fs.createReadStream. The frontend is responsible for interpreting whether those bytes represent a JSON object or a binary Blob based on the extension detected before the request.

Can I manually override the response type for a specific file request?

Yes. While the workspace service automatically determines the type via getFileReponseTypeByName, you can bypass this by directly calling the HTTP utility in frontend/src/utils/http.js and passing a custom responseType parameter (either 'blob' or 'json') to the post method.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}