The Magic of the "Sticky Drive" Algorithm

Before we look at the configs, you need to unlearn how traditional storage arrays work. HuskHoard does not stripe data across all drives like RAID 0, and it doesn't try to perfectly balance free space across an array like Unraid.

Instead, HuskHoard uses a proprietary logic called the "Sticky Drive" Sequential Fill. Under the hood, the engine scans your array of drives and sorts them by Most Used Space First.

If you have 40 drives, HuskHoard will write exclusively to Drive 1 until it hits your configured safety limit (e.g., 50GB free). Once Drive 1 is full, it permanently leaves it alone and "spills over" into Drive 2.

Standard RAID Array

The Striping Problem

Data is fragmented across all 40 disks.
All 40 drives must spin to write or read a single file.
Losing 3 drives destroys the entire 40-drive array.
HuskHoard Sticky Fill

The Archival Advantage

Drives 2-40 stay spun down and sleeping for months.
Massive power savings and extended hardware lifespan.
Every drive is independent. If Drive 1 dies, you only lose Drive 1.
Edge Case: Repacking & Backfilling

What happens if Drive 1 fills up, Husk moves to Drive 2, and then you run the Garbage Collection (Repack) command on Drive 1, freeing up 50% of its space?

HuskHoard will finish what it started. Because it sorts by "most used space", it will continue filling Drive 2 to the brim. Drive 1 simply becomes the "next in line," ensuring you don't have multiple drives constantly spinning up and down to backfill small holes.

Config 1: The High-Availability Mirror (20/20 Split)

If you have 40 drives and prioritize absolute data safety over total capacity, you want to set up a mirrored array. HuskHoard makes this incredibly easy. You assign half your drives to the primary pool, and half to the replication pool.

# Prevent Husk from choking the filesystem (leave 50GB breathing room) min_free_space_gb = 50 # Tier 1: The main drives Husk will read/write from primary_volumes = [ "/mnt/hdd01/vol.img", "/mnt/hdd02/vol.img", "/mnt/hdd03/vol.img", ... down to hdd20 ] # Tier 2: Real-time mirror backups replication_volumes = [ "/mnt/hdd21/vol.img", "/mnt/hdd22/vol.img", "/mnt/hdd23/vol.img", ... down to hdd40 ] # Tell Husk to keep exactly 1 copy in the replication pool replicas = 1

When the Janitor runs, HuskHoard pulls the cold file from your Hot Tier, compresses it in RAM, and uses its MultiTapeWriter to write the data to hdd01 and hdd21 at the exact same time. It multiplexes the I/O, guaranteeing perfect 1:1 redundancy with zero latency penalty.

Config 2: The Infinite Spillover (JBOD + Cloud)

What if those 40 drives are already backed up to Backblaze B2 at the block level, and you just want a massive, unified 800TB volume? You can dump all 40 drives into the primary pool and use a Cloud bucket as your emergency overflow.

min_free_space_gb = 50 # All 40 drives act as a massive sequential pool primary_volumes = [ "/mnt/hdd01/vol.img", "/mnt/hdd02/vol.img", ... down to hdd40 ] # Emergency overflow: Only used if ALL 40 local drives are full failover_volumes = [ "rclone:my_aws_remote:husk-archive/" ] replication_volumes = [] replicas = 0

Husk will fill drives 1 through 40 sequentially. If Drive 40 finally hits the 50GB limit, Husk seamlessly pivots to the failover_volumes array. It spawns an rclone rcat subprocess and begins streaming incoming data directly into your AWS bucket without ever touching the full local disks.

Safety Mechanism: The Physical Fit Check

What if you try to archive a 100GB video file, but the active drive only has 55GB of free space left? (Which is still technically above your 50GB limit).

HuskHoard calculates the uncompressed size of the incoming file before writing a single byte. It will see that 100GB > 55GB, cleanly reject that drive, and gracefully move to the next drive (or your Cloud Failover). You will never end up with a half-written, corrupted file due to an out-of-space error mid-stream.

Config 3: The Hybrid Tape Archive

If you are a post-production house running an LTO-8 or LTO-9 Tape Library, HuskHoard shines as a native Tape Management System (LTFS replacement). In this setup, we write to an intermediate local hard drive, but replicate directly to Physical Tape and the Cloud simultaneously.

primary_volumes = ["/mnt/staging_hdd/archive.img"] # Replicate to physical tape AND the cloud simultaneously replication_volumes = [ "/dev/nst0", # Linux Character Device for SCSI Tape "rclone:b2_remote:husk_deep_archive/" # Backblaze B2 Object Storage ] # Keep TWO replicas (1 on Tape, 1 in Cloud) replicas = 2

By using the character device /dev/nst0, HuskHoard automatically translates POSIX writes into Linux MTIO (Magnetic Tape I/O) commands. It handles the SCSI rewinds, filemarks, and end-of-media (MTEOM) appends natively. If a user requests a file, HuskHoard will pull it from the staging HDD first. If the HDD is wiped, it will automatically spin up the tape drive to fetch the byte-range.

Config 4: The M&E Ingest Pipeline (Zero-Compression & Spillover)

Post-production houses deal with massive .braw and .mxf video files that are already highly compressed. Wasting CPU cycles running these through Zstd compression only to get a 1% size reduction is a massive bottleneck. Furthermore, multi-terabyte camera offloads can fill up a Hot Tier NVMe drive instantly, bypassing standard age-based archiving policies.

HuskHoard handles this beautifully by acting as a high-speed Drop Zone.

# Save the NVMe: If Hot Tier exceeds 80% full, trigger Emergency Spillover hot_tier_max_usage_percent = 80 # Ignore 'max_age_days' for these folders. Archive and stub immediately. immediate_archive_dirs = ["/HotTier/Ingest_Drop/"] # Bypass Zstd compression entirely for pre-compressed media formats no_compress_extensions = ["mp4", "mkv", "avi", "mov", "mxf", "braw", "r3d"]

When an editor drops a 500GB camera card into /HotTier/Ingest_Drop/, the daemon immediately queues it. Because Zstd is bypassed via no_compress_extensions, HuskHoard engages a High-Performance Raw Copy, streaming the bytes to the archive array at the maximum physical speed of your drives.

If someone dumps data outside the Ingest folder and the NVMe hits 80% capacity, the Janitor will intercept the high-water mark, engage Emergency Spillover, and begin stubbing the oldest files on the drive immediately to prevent a production halt.

Taming the Janitor: Nightly Batch Processing

In your testing sandbox, you probably had janitor_interval_secs = 60 so you could watch files get stubbed instantly. In a 40-drive production environment, constant scanning is unnecessary IO overhead.

In husk_config.toml, you should switch from the interval timer to the absolute schedule timer:

# Run once a day at 2:00 AM (Overrides the interval timer) janitor_schedule_time = "02:00" # Fallback interval (Ignored when schedule_time is active) janitor_interval_secs = 60

Now, HuskHoard acts as a nightly batch processor. Your users work all day on the Hot Tier. At 2:00 AM, the Janitor wakes up, scans the database for files older than your max_age_days limit, spins up the active archive drive, writes the day's batch, and gracefully spins the array back down.

Zero
RAID Rebuilds

Because data is sequentially packed and tracked via SQLite, a dead drive only affects the specific files on it. No agonizing 50-hour RAID 6 rebuilds.

100%
Hardware Portability

HuskHoard writes a UUID to LBA 0 of every volume. You can unplug a USB drive, move it to a different SATA port, and the database will automatically find it.

O(1)
Cloud Failover

If your local disks fill up on a Friday night, your users won't notice. Husk seamlessly routes incoming archives to S3/B2 until you add more drives on Monday.

Moving to 40+ drives doesn't mean moving to 40x the complexity. With the right configuration, HuskHoard turns a massive rack of disks into a self-managing, power-sipping, infinitely scalable vault.