Storage Options Compared for S3-Like Object Storage: Block vs File vs Object

Block storage provides mutable, high-performance blocks ideal for databases and virtual machines; file storage offers general-purpose hierarchical access via NFS/SMB protocols; while object storage delivers immutable, cost-efficient, massively scalable storage through RESTful APIs optimized for unstructured data and cold storage scenarios.

The liquidslr/system-design-notes repository provides a comprehensive analysis of distributed storage architectures, including a detailed examination of storage options compared for S3-like object storage systems. This guide explores the fundamental architectural distinctions between block, file, and object storage models as documented in 24. S3-like Object Storage/README.md, illustrating why object storage has become the foundation for modern cloud-scale applications requiring vast scalability.

Block vs File vs Object: Core Architectural Differences

According to the source code analysis in the repository's S3-like Object Storage chapter, three primary storage categories serve distinct workloads in distributed systems. The comparison table located in 24. S3-like Object Storage/README.md (lines 27-36) outlines seven critical differentiating attributes:

Attribute	Block Storage	File Storage	Object Storage
Mutable Content	Yes	Yes	No (objects are immutable, but versioning can be enabled)
Cost	High	Medium to high	Low
Performance	Medium-to-high, very high	Medium-to-high	Low to medium
Consistency	Strong consistency	Strong consistency	Strong consistency (as used by S3)
Data Access	SAS/iSCSI/FC (block protocols)	Standard file protocols (CIFS/SMB, NFS)	RESTful API
Scalability	Medium scalability	High scalability	Vast scalability
Best Suited For	Virtual machines, databases	General-purpose file system access	Binary, unstructured data (e.g., backups, archives)

Block storage operates at the raw storage level, presenting fixed-size blocks to operating systems through protocols like iSCSI or Fibre Channel. File storage abstracts these blocks into hierarchical file systems accessible via NFS or SMB, providing directory structures and file-level permissions. Object storage eliminates the hierarchy entirely, storing data as discrete objects with metadata and globally unique identifiers, accessed exclusively through HTTP-based RESTful APIs as illustrated in the repository's visual comparison (24. S3-like Object Storage/images/storage-comparison.png).

Object Storage Characteristics in S3-Like Systems

Object storage distinguishes itself through specific architectural constraints and capabilities that prioritize durability and scale over raw performance.

Immutability and Versioning

Unlike block and file systems, object storage treats data as fundamentally immutable. Once written, an object cannot be modified in place; applications must upload a new version to replace existing data. However, as noted in the comparison table, systems like Amazon S3 enable versioning capabilities that preserve historical iterations, providing audit trails and accidental deletion protection without compromising the core immutability principle that ensures data integrity at massive scale.

RESTful API Access Patterns

While block storage relies on SAN protocols (SAS, iSCSI, FC) and file storage uses NAS protocols (NFS, CIFS/SMB), object storage exclusively utilizes RESTful APIs over HTTP/HTTPS. This architectural decision enables limitless scalability and global accessibility, allowing clients to interact with storage using standard HTTP verbs (GET, PUT, DELETE) rather than specialized storage network infrastructure, though this introduces the "low to medium" performance characteristics identified in the repository analysis.

Economic and Scalability Advantages

The repository analysis identifies object storage as the lowest-cost option with vast scalability, contrasting sharply with block storage's high cost and medium scalability constraints. This economic profile makes object storage optimal for cold data scenarios—backups, archives, and media repositories—where durability and petabyte-scale expansion outweigh latency-sensitive performance needs.

Practical Implementation with Python boto3

The repository provides Python code examples demonstrating how applications interact with S3-like object storage using the AWS SDK (boto3). These operations illustrate the RESTful API pattern and immutability constraints discussed in the architectural comparison.

import boto3
from botocore.exceptions import ClientError

# Initialize the S3 client (assumes credentials are configured in ~/.aws)

s3 = boto3.client('s3')

# 1️⃣ Create a new bucket

def create_bucket(bucket_name, region='us-east-1'):
    try:
        s3.create_bucket(
            Bucket=bucket_name,
            CreateBucketConfiguration={'LocationConstraint': region}
        )
        print(f'Bucket {bucket_name} created.')
    except ClientError as e:
        print(e)

# 2️⃣ Upload an object (immutable; versioning can be enabled separately)

def upload_object(bucket_name, object_key, data):
    try:
        s3.put_object(Bucket=bucket_name, Key=object_key, Body=data)
        print(f'Object {object_key} uploaded to {bucket_name}.')
    except ClientError as e:
        print(e)

# 3️⃣ Download an object

def download_object(bucket_name, object_key):
    try:
        response = s3.get_object(Bucket=bucket_name, Key=object_key)
        content = response['Body'].read()
        print(f'Object {object_key} downloaded (size: {len(content)} bytes).')
        return content
    except ClientError as e:
        print(e)

# Example usage

if __name__ == '__main__':
    bucket = 'my-example-bucket'
    key = 'example.txt'
    payload = b'Hello, object storage!'

    create_bucket(bucket)
    upload_object(bucket, key, payload)
    download_object(bucket, key)

The put_object operation demonstrates the immutable nature of object storage—each upload creates a discrete object that cannot be appended or partially modified without complete replacement. This contrasts sharply with block storage, where individual bytes within a block can be modified randomly through direct disk access protocols.

Summary

• Block storage delivers mutable, high-performance raw blocks accessed via iSCSI/FC, ideal for databases and virtual machines requiring strong consistency and medium scalability but at high cost.
• File storage provides hierarchical file system access through NFS/SMB protocols, balancing medium-to-high performance with high scalability for general-purpose workloads requiring mutable content.
• Object storage offers immutable, low-cost, vastly scalable storage via RESTful APIs, optimized for unstructured data, backups, and archives where write-once-read-many patterns and massive scale dominate requirements.
• The liquidslr/system-design-notes repository documents these distinctions in 24. S3-like Object Storage/README.md, including the detailed comparison table at lines 27-36 and supporting architectural diagrams.
• Python implementations using boto3 demonstrate practical application of object storage principles through bucket creation, immutable object uploads via put_object, and retrieval operations using get_object.

Frequently Asked Questions

What is the primary difference between block storage and object storage?

Block storage presents raw, mutable storage blocks to operating systems, allowing random access modifications and high-performance I/O required by databases and transactional systems. Object storage treats data as immutable objects accessed via RESTful APIs, optimizing for massive scale and cost efficiency rather than low-latency random writes. As detailed in the repository's comparison table, block storage uses protocols like iSCSI and FC, while object storage relies exclusively on HTTP-based APIs with strong consistency guarantees.

Why does object storage have lower performance than block storage?

Object storage prioritizes durability, vast scalability, and cost efficiency over raw throughput and latency. The architectural overhead of RESTful API processing, metadata management, and the immutability constraint—requiring complete object replacement rather than partial updates—introduces higher latency compared to direct block access. According to the liquidslr/system-design-notes analysis, object storage operates in the "low to medium" performance range, making it unsuitable for transactional database workloads but ideal for throughput-heavy archival storage scenarios.

When should I choose file storage over object storage?

Select file storage when applications require hierarchical directory structures, concurrent file-level locking, or compatibility with legacy applications expecting standard POSIX file system semantics. File storage supports mutable content and standard protocols (NFS, CIFS/SMB) that object storage does not natively provide. Choose object storage when handling massive unstructured datasets, static web assets, or backup archives where the flat namespace, immutability, and RESTful API access provide operational and economic advantages.

How does S3 provide strong consistency despite being an object store?

While the comparison table indicates that object storage offers strong consistency, this refers to read-after-write consistency guarantees for PUT and DELETE operations in modern S3-like systems. Once an object is successfully written using put_object, any subsequent read operation immediately returns the latest version across all availability zones. This contrasts with eventual consistency models where replicas might lag, and aligns object storage with block and file storage consistency guarantees despite its distributed, highly scalable architecture.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}