[ZFS] Replace failed disk on my Proxmox Host


Yesterday evening I got an email that on my Proxmox server a disk has failed. In my ZFS Raidz1 I have 4 different drives of two manufactures: 2x HGST and 2x Seagate.
In the last 7 years I also used some Western Digitals. The only faulty hard drives I had in this years were from Seagate. This was the third… So this morning I bought a new hard disk, this time a Western Digital Red, and replaced the failed disk.

SSH into my server and checked the zpool data. Because I already removed the failed disk, it’s marked as unavailable.

failed disk: wwn-0x5000c5009c14365b

Now I had to find the Id of my new disk. With fdisk -l, I found my new disk as /dev/sde, but there was no id listed.

sudo fdisk -l

To be sure I checked again with:

sudo lsblk -f

With disk by-id I now got the Id.

ls /dev/disk/by-id/ -l | grep sde

new disk: ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1CSDLRT
and again the failed disk: wwn-0x5000c5009c14365b

Before replacing the disks, I did a short SMART test.

sudo smartctl -a /dev/sde
sudo smartctl -t short /dev/sde
sudo smartctl -a /dev/sde

The new disk had no errors. And because it is a new disk, I don’t had to wipe any file systems from it.

So first I took the failed disk offline. Not sure if that was necessary, but to be on the safe side…

sudo zpool offline data 2664887927330352988

Next run the replace command.

sudo zpool replace data /dev/disk/by-id/wwn-0x5000c5009c14365b-part2
/dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1CSDLRT

The resilver process for the 3TB disk took about 10 hours.

[Proxmox] NFSv4 client saves files as “nobody” and “nogroup” on ZFS Share

I’m running a Proxmox Cluster with PVE1 and PVE2. On PVE2 a VM is running Debian Buster, which is mounting an zfs nfs share from PVE1. Inside the VM a script is running as root saving a backup on this nfs share. If I create a file locally (Test1) on …

[ZFS] Encryption

Native encryption in ZFS is supported since version 0.8.0. Check your current ZFS version with: First activate the encryption feature on your pool: To get an overview of all pools with enabled encryption use the following command: To create a new encrypted dataset with a passphrase: Check the keystatus, the …

[ZFS] Basic Commands

Documentation: https://github.com/zfsonlinux/zfs/wiki/Admin-Documentation
Manual Pages: http://zfs.datto.com/man/
Milestones: https://github.com/zfsonlinux/zfs/milestones

modinfo zfs                               //check current ZFS version
zfs list                                  //list pool with datasets
zfs list -r pool                          //show all datasets in a pool with size and mountpoint
zfs list -r -o name,mountpoint,mounted    //check if datasets are mounted   
zpool status (pool)
zpool list
zpool list -v
zpool iostat (pool 1)
zpool iostat -v

Activate NFS on dataset:

zfs set sharenfs=on pool/dataset
zfs get sharenfs pool/dataset

Usefull comands when replacing a failed disk:

ls -l /dev/disk/by-id/                // Disk ID's
zdb                                   // Display zpool debugging and consistency information
smartctl -a /dev/ada0                 // S.M.A.R.T info
wipefs -a new_hdd                     // remove ext4 filesystem 
zpool replace data old_hdd new_hdd    // Replace HDD

Scrub cronjob:

cat /etc/cron.d/zfsutils-linux 

Add and remove Log & L2ARC:

zpool add data log sda1
zpool add data cache sda2

zpool remove data log sda1
zpool remove data cache sda2