Unify object, file, and block access
The platform exposes S3, Swift, NFS, and iSCSI through a shared distributed storage environment while retaining interface-specific responsibilities.
Modernization and Distributed Platforms
DataMesh - Distributed Object Storage Platform
A major engineering contribution to a distributed scale-out storage platform spanning S3, Swift, NFS, iSCSI, raw-device access, Node.js runtime engineering, native C/C++ integration, and distributed cluster management.
Distributed storageRaw-device accessNode.js runtime engineeringCluster coordinationStorage interfaces
The system
DataMesh is a distributed scale-out storage product that exposes object, file, and block access across a cluster of storage nodes. Client applications can use S3-compatible and Swift-compatible object interfaces, NFS file access, or iSCSI block access while the platform coordinates storage services and low-level device operations across the distributed environment.
The architecture spans several implementation layers. JavaScript and Node.js services provide management and platform functionality; native Node.js addons bridge those services into C/C++ storage components; low-level C code accesses raw storage devices; and cluster services coordinate nodes, quorum, objects, tenants, and service state.
Some product requirements reached below ordinary application development. OPTIME modified portions of the Node.js runtime to address asynchronous-operation behavior required by the storage system and built native addons connecting JavaScript services directly with existing native storage components. Exact runtime patches and asynchronous algorithms remain confidential.
Engineering relationship
Across a four-year distributed-systems program, OPTIME became one of the principal engineering contributors to the platform. The team was responsible for approximately half of the product implementation across storage interfaces, raw-device access, distributed cluster management, management APIs, native integration, runtime modifications, and build/release infrastructure.
This contribution was substantial but not exclusive: OPTIME did not independently invent or own the complete product. Its responsibilities crossed the data plane, management plane, runtime boundary, distributed coordination, and engineering-delivery workflows required to turn the wider product into a maintainable storage platform.
Two OPTIME engineers retained deep subsystem context across the program, working in C, C++, JavaScript, Node.js, and Ruby while coordinating changes that crossed native storage components and higher-level management services.
Engineering constraints
- Expose object, file, and block interfaces through one distributed scale-out storage product.
- Connect JavaScript management services to performance-sensitive native C/C++ storage components.
- Modify runtime-level asynchronous behavior only where product requirements could not be met at the application layer.
- Provide low-level raw-device access without exposing private disk layouts, metadata structures, allocation algorithms, or on-disk formats.
- Coordinate cluster, tenant, object, and service state across distributed nodes.
- Support quorum management and node synchronization without overstating ownership of the complete consistency model.
- Offer a management API without publishing private endpoint paths or schemas.
- Maintain repeatable build, test, packaging, and delivery workflows across native and Node.js components.
- Describe contribution and delivery accurately without inventing downstream installation counts.
What OPTIME engineered
- Approximately 50% of the wider product implementation across multiple platform layers.
- S3-compatible and Swift-compatible object-storage access functionality.
- NFS file-storage access and iSCSI block-storage access.
- Low-level C functionality for direct access to raw storage devices.
- Native C/C++ storage and distributed-systems components.
- A REST API server for cluster, tenant, object, service, and platform management.
- Cluster-management and service-management functionality.
- Tenant-management and object-management functionality.
- Quorum-management and node-synchronization functionality.
- Product-specific modifications to the Node.js runtime for required asynchronous-operation behavior.
- Native Node.js addons connecting JavaScript services with existing C/C++ storage components.
- JavaScript and Node.js platform services and integration functionality.
- Ruby and Node.js automation supporting build, test, packaging, and delivery workflows.
- Cross-layer integration, stabilization, maintenance, and product delivery.
Architecture
Storage/data plane - client applications
Applications reach the scale-out platform through object, file, or block interfaces.
Storage/data plane - S3 / Swift
S3-compatible and Swift-compatible services expose object-storage access.
Storage/data plane - NFS / iSCSI
NFS provides file access and iSCSI provides block-storage access.
Storage/data plane - distributed services
Storage services coordinate requests across the distributed platform.
Storage/data plane - native C/C++ components
Performance-sensitive native components implement storage and systems functionality.
Storage/data plane - raw devices
Low-level C code provides direct access to raw storage devices at a deliberately generalized boundary.
Control/cluster plane - REST management API
The management server exposes controlled operations for the platform without publishing private schemas.
Control/cluster plane - management services
Cluster, tenant, object, and service-management responsibilities operate above the distributed nodes.
Control/cluster plane - quorum and synchronization
Nodes coordinate quorum and state synchronization without exposing the proprietary consistency implementation.
Runtime/integration plane - Node.js services
JavaScript and Node.js services provide higher-level platform and management functionality.
Runtime/integration plane - modified runtime and native addons
Runtime changes address required asynchronous behavior, while native addons bridge Node.js services into C/C++ storage components.
Delivery plane - Ruby / Node.js automation
Build, test, packaging, and delivery automation supports the multi-language product lifecycle.
Key engineering decisions
Bridge JavaScript services to native storage code
Native Node.js addons connect management and platform services directly with existing C/C++ components instead of duplicating the native storage implementation.
Change the runtime only when necessary
Where application-level work could not satisfy required asynchronous behavior, OPTIME modified portions of the Node.js runtime. Private patches and scheduler details remain intentionally omitted.
Separate management from low-level device access
REST and cluster-management services operate above native storage components and C raw-device access, creating clear responsibility boundaries across the stack.
Engineering ownership and delivery
OPTIME’s scope grew across the product stack until it represented approximately half of the implementation. Long-lived context across storage interfaces, native code, runtime behavior, cluster services, and release automation reduced the cost of coordinating changes across subsystem boundaries.
Ruby and Node.js automation supported build, test, packaging, and delivery workflows for a product spanning native and managed-runtime components. No specific CI/CD product or private release process is disclosed.
Verified capability
OPTIME completed and delivered its engineering scope as part of the distributed storage product. The technology subsequently became part of the customer’s broader storage platform; OPTIME does not have verified information about the number of downstream deployments.
Verified result
OPTIME delivered approximately half of the product implementation across storage access, native systems code, distributed management, runtime integration, and build/release infrastructure without claiming ownership of the complete customer product.
Verified metrics
Approximately 50% of the product
Owner-confirmed estimate of OPTIME’s implementation contribution across the wider storage product.
Technology & Engineering Role
- C
- Raw-device access and low-level storage and systems functionality.
- C++
- Native storage and distributed-systems components.
- JavaScript / Node.js
- Management services, platform components, and integration layer.
- Modified Node.js runtime
- Runtime-level asynchronous-operation changes required by the storage platform.
- Native Node.js addons
- Bridge between JavaScript services and native C/C++ storage components.
- S3
- S3-compatible object-storage interface.
- Swift
- Swift-compatible object-storage interface.
- NFS
- File-storage interface.
- iSCSI
- Block-storage interface.
- REST API
- Cluster, tenant, object, service, and platform management.
- Quorum management
- Distributed-node coordination.
- Node synchronization
- Coordination and state synchronization across storage nodes.
- Raw-device access
- Direct low-level storage access through C components.
- Ruby / Node.js
- CI/CD and build, test, packaging, and delivery automation.