How Website-downloader Handles User Disconnection and Cleanup of Partial Downloads

When a user disconnects from the WebSocket server, the system terminates active wget and archiver processes and deletes partially downloaded files to prevent orphaned processes and disk space waste.

The AhmadIbrahiim/Website-downloader repository uses WebSocket connections to manage website downloads. When clients disconnect abruptly, the application implements a multi-layered cleanup mechanism to handle user disconnection and cleanup of partial downloads, ensuring no background processes or temporary files remain on the server.

WebSocket Disconnection Detection

In socket/socket.js, the server listens for the 'disconnect' event to detect when a client closes the connection. The handler logs the disconnection and immediately initiates cleanup procedures for any active processes attached to that socket.

socket.on('disconnect', function () {
    console.log("User disconnected");
    // Stop the wget process and remove partially downloaded files
    if (socket.wgetProcess) {
        socket.wgetProcess.kill();          // send SIGTERM
    }
    // Stop the archiving process if the user disconnects
    if (socket.archiverProcess) {
        socket.archiverProcess.abort();     // abort zip creation
    }
});

Terminating Active Download Processes

When a download is in progress, the system stores the wget child process reference in socket.wgetProcess. Upon disconnection, the handler invokes .kill() to send a SIGTERM signal, gracefully terminating the download.

The wget module in wget/index.js spawns the download using child_process and listens for process termination:

// Handle process termination and cleanup
child.on('exit', (code, signal) => {
    if (signal === 'SIGTERM') {
        console.log('Process terminated');
        removePartiallyDownloadedFiles(website);
    }
});

This ensures that when the WebSocket handler kills the process, the cleanup function triggers automatically.

Cleaning Up Partial Files

The removePartiallyDownloadedFiles function in wget/index.js handles deletion of incomplete downloads. It uses Node.js's fs.rmdir with the recursive: true option to remove the entire directory containing partial website data:

function removePartiallyDownloadedFiles(website) {
    const directory = path.join(__dirname, '../', website);
    fs.rmdir(directory, { recursive: true }, (err) => {
        if (err) console.error(`Error while removing partially downloaded files: ${err.message}`);
        else console.log('Partially downloaded files removed successfully');
    });
}

This prevents incomplete downloads from consuming disk space indefinitely.

Aborting Archive Generation

If the download has progressed to the archiving stage, the system stores the archiver instance in socket.archiverProcess. The disconnect handler calls .abort() on this instance to stop ZIP file creation and prevent corrupt archives from remaining on the server.

Summary

• The WebSocket server in socket/socket.js detects client disconnections via the 'disconnect' event
• Active wget processes are terminated using .kill(), which sends a SIGTERM signal
• The wget module listens for termination signals and invokes removePartiallyDownloadedFiles() to delete partial data
• Incomplete archives are aborted via .abort() on the archiver process
• All cleanup operations occur automatically without requiring manual intervention

Frequently Asked Questions

What happens if a user disconnects during an active download?

When a user disconnects, the WebSocket server immediately terminates any running wget process by calling .kill() on socket.wgetProcess. It also aborts any active archiver. The wget module then automatically removes partially downloaded files using fs.rmdir() with recursive deletion enabled.

Where is the disconnect handling logic implemented?

The disconnect handling logic is implemented in socket/socket.js. This file sets up the WebSocket connection and registers the 'disconnect' event listener that orchestrates the cleanup of both download and archival processes.

How does the system detect that a process was terminated due to disconnection?

The system checks the termination signal in the 'exit' event listener within wget/index.js. When signal === 'SIGTERM', the system knows the process was intentionally killed by the disconnect handler and proceeds to clean up the associated files.

Does the cleanup handle both files and archives?

Yes. The cleanup mechanism handles both partial website files and incomplete ZIP archives. The wget module removes downloaded files via removePartiallyDownloadedFiles(), while the archiver process is aborted via .abort() to stop ZIP generation and prevent corrupt archive files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}