HuskHoard is an automated, transparent data-tiering engine for Linux. It keeps your NVMe drives fast by silently archiving cold data to LTO tape, spinning drives, or cloud — while every file stays fully visible to your OS.
Enterprise vendors charge thousands to lock your data inside proprietary black boxes. HuskHoard does it for free, in user-space, using formats you already own.
Files stay visible to your OS at full logical size. Applications never know the difference.
When a process opens a stubbed file, HuskHoard intercepts via the Linux fanotify kernel API
and recalls it in real-time — no FUSE overhead, no mount magic.
Watch 4K video directly from LTO tape or S3 via a local HTTP bridge. Plex, Jellyfin, and VLC can seek through massive files instantly using HTTP Range requests — with zero SSD impact and no full recall required.
Built for M&E pipelines
All payload data is stored in standard Zstd streams verified by BLAKE3.
You can extract everything using only dd and zstd.
No proprietary formats. No license keys. No asking permission.
Native SCSI driver for LTO-5 through LTO-9. SMR-safe log-structured writes that eliminate the "write wall." Cloud replication via rclone to 40+ providers, packed into optimal 16MB Zstd frames to minimize API costs.
LTO · SMR · S3 · rcloneEach part of HuskHoard does one job with precision. Together they form a continuous, self-managing storage pipeline.
Tracks every file, its complete version history, and its exact byte-offset on physical media. The single source of truth for the entire system.
A lightweight kernel-space listener. Detects when an application requests a stubbed file, blocks the process, triggers instant recall, then resumes.
Background daemon that identifies cold data by age, file extension, or directory policy. Queues candidates for the Archive Worker with zero disruption.
Compresses data into seekable Zstd frames, multiplexes writes across the full storage pool, and issues low-level SCSI commands for tape hardware.
Built on the Linux fanotify kernel API,
HuskHoard operates entirely in user-space — no kernel modules, no root daemons.
N-way replication mirrors cold data simultaneously across local drives, physical tapes,
and cloud buckets. Point-in-time recovery lets you roll back any file to a previous
version. The BLAKE3 bit-rot scrubber cryptographically verifies offline storage
on any schedule you set. Have a look at the Blog for detailed explainers on architecture.
HuskHoard eliminates the three biggest sources of unnecessary storage spend: cloud egress fees on partial reads, always-on RAID arrays burning electricity to store data you access once a month, and hardware that charges thirty times more per gigabyte than it needs to.
Cloud providers charge per byte downloaded. Naive archivers retrieve entire files.
HuskHoard slices every archive into independent 16 MB Zstd frames and embeds a
TLV index at the head of each object — so any read issues a single HTTP
Range request
for exactly the frames needed. A 5 MB document inside a 150 GB object
costs 5 MB of egress — not 150 GB. For pre-compressed media like MP4 and MOV,
byte offsets are 1-to-1 and seeks are O(1) with no decompression overhead at all.
A standard RAID-Z2 array stripes data across every disk — so reading one file spins up all eight drives. Even a background metadata ping from Plex or Samba prevents the array from ever sleeping. HuskHoard stores all metadata on the NVMe hot tier. Archive drives become pure payload tanks that power down completely, drawing 0W until their specific volume is needed.
At scale, the storage medium matters more than any software optimization. LTO-9 tape costs roughly six times less per gigabyte than CMR hard drives and fifteen times less than NAS SSDs — with a 30-year rated media life and 0W idle. HuskHoard provides the software layer that makes tape as searchable and accessible as a local SSD, without the cost.
Cloud replication follows the same logic. HuskHoard packs files into large sequential
husk_XXXX.bin objects
rather than uploading one file per S3 object. Thousands of small files become a handful
of API calls, slashing transaction costs. When a file is read back, a targeted HTTP Range
request retrieves only the relevant 16 MB frame — so you never pay to download a 150 GB
archive object to retrieve a single document inside it.
Read the full cloud architecture post →
HuskHoard ships as a single CLI binary. No Docker required. No configuration daemons. Run as your standard user — not root.
The open-source core is free forever under AGPL v3. Enterprise sidecar modules — designed to run alongside the core binary — are on the way.
Real-time "tank gauge" monitoring. Visual capacity planning and tiering activity.
Coming Soon/metrics endpoint ready for Grafana. Full observability for production deployments.
Coming SoonFine-grained per-project replication policies. SLA-backed recovery windows.
Coming SoonDedicated support channels, guaranteed response times, and onboarding assistance.
Coming SoonInterested in early access to enterprise features, or want to discuss a commercial deployment?
info@huskhoard.com →Local HTTP bridge enabling video players to seek through tapes via HTTP Range requests. Plex, Jellyfin, VLC, mpv — all supported.
Real-time visual "tank gauge" monitoring for storage capacity, tiering activity, and system health.
A native /metrics endpoint for full production observability. Drop-in for existing Grafana stacks.
Optional commercial extensions — advanced replication policies, SLA support, and more — running alongside the AGPL core.
Get notified about performance improvements, new hardware drivers, and architecture deep dives.