Hierarchical Storage Management

 

The CloudTier Storage Tiering Filter Driver SDK is a Hierarchical Storage Management (HSM) development kit. It provides a kernel-level framework to implement transparent data migration between high-cost (hot) and low-cost (cold) storage media. By replacing migrated files with Stub Files, it enables organizations to optimize storage efficiency and reduce OpEx/CapEx without disrupting existing applications.

System Architecture

The SDK operates at the file system filter level, acting as an "invisible traffic cop" between user applications and the Windows NTFS/ReFS file system.

Kernel-Mode Filter Driver: Intercepts I/O requests (IRP_MJ_READ, IRP_MJ_CREATE) and manages reparse points. It evaluates requests against business rules to determine if data resides locally or requires a remote fetch.

User-Mode Service (Policy & Logic): Manages cloud connectivity (S3, Azure, etc.) and executes the Lifecycle Management policies.

Stub Management: Leverages Windows reparse points and sparse files. Stubs retain full original metadata (timestamps, ACLs, file size) while consuming minimal local disk space.

Understanding the Stub File

A core component of the CloudTier architecture is the Stub File, which acts as a seamless placeholder for data that has been migrated to the cloud or secondary storage.

Technical Composition

Technically, a stub file is a sparse file tagged with a reparse point attribute. This reparse point acts as a trigger for the CloudTier kernel-level filter driver. When an application attempts to open or read the stub, the OS recognizes the reparse point and hands control to the filter driver, which then fetches the requested data from the remote tier.

Metadata and Custom Data

A stub file mimics a standard physical file in appearance and behavior. It retains 100% of the original file's metadata, including:

File name and extension

Logical file size (applications see the full original size, not the size on disk)

Creation time, last write time, and last access time

Security attributes and Access Control Lists (ACLs)

Additionally, stub files can store custom tag data embedded directly within the stub metadata, allowing applications to track internal database IDs, migration job details, or routing URIs.

The User Experience

Unlike physical files, a stub file consumes virtually zero local disk space because it does not store the actual file content. However, to end-users and applications, the file appears entirely normal. Because the file remains visible in Windows Explorer and command-line tools, CloudTier prevents the "broken link" syndrome common in legacy archiving software.

Comments

Post a Comment

Popular posts from this blog

EaseTag Cloud Storage Connect

  The challenges to connect the public cloud storage Enterprises are increasingly adopting cloud storage options because they need more capacity, elastic capacity and a better way to manage storage costs over time. The growing amount of enterprise data is proving too difficult for IT departments to manage using their data center alone. Migrating and managing your data storage in the cloud can offer significant value to the business. A cloud storage migration is when a company moves some or all of its local data into the cloud, usually to run on the cloud-based infrastructure provided by a cloud service provider such as AWS and Azure.The main challenge of the cloud storage migration here is how to carry out your migration with minimal disruption to normal operation, at the lowest cost, and over the shortest period of time. If your data becomes inaccessible to users during a migration, you risk impacting your business operations.The biggest challenge of the cloud storage migration ...
Read more

EaseFilter Comprehensive Complete File Security SDK

Nowadays file security is more and more important to all sizes companies, it has been a major task for IT professional. Threats to file security – both external and internal - is extensive and growing. To maintain the confidentiality, integrity and availability of the file could be complicated. EaseFilter can help address these threats by securing sensitive files so that they’re always under your control.With cyber attacks ravaging the networks of governments and corporations,enterprise IT and security professionals need to adjust their data protection strategies. EaseFilter can help protect your enterprise from threats such as external hacks and internal leaks. Whether you need to enable personal productivity, facilitate team collaboration, or mobilize and transform your entire business, EaseFilter Comprehensive File Security SDK is the best choice for secure file collaboration. EaseFilter Comprehensive File Security SDK EaseFilter Comprehensive File Security SDK is a set ...
Read more