1 What is this ?
ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs.
2 Usage
2.1 Show status
If you want to list all zfs pool and snapshot, use:
root@bus-ind-zfs-01:[~]# zfs list -t all
NAME USED AVAIL REFER MOUNTPOINT
zfs-datastore 663K 3.40T 140K /zfs-datastore
zfs-datastore@28July2017 0 - 140K -
If you want to show the status of RAIDz use:
root@bus-ind-zfs-01:[~]# zpool status
pool: zfs-datastore
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Sun Jul 9 00:24:02 2017
config:
NAME STATE READ WRITE CKSUM
zfs-datastore ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
scsi-36001c230c6dacc00204ad9df50371381-part1 ONLINE 0 0 0
scsi-36001c230c6dacc00204ad9e94401c004-part1 ONLINE 0 0 0
scsi-36001c230c6dacc00204ad9f3373942be-part1 ONLINE 0 0 0
scsi-36001c230c6dacc00204ad9fb2db0d88d-part1 ONLINE 0 0 0
errors: No known data errors
root@bus-ind-bak-sd-01:[~]# zpool status
pool: zroot
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
repaired.
scan: scrub repaired 52K in 43h23m with 0 errors on Mon Nov 12 19:47:06 2018
config:
NAME STATE READ WRITE CKSUM
zroot DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
scsi-36a4badb009ce8300210a00c906a9ca2b ONLINE 0 0 0
scsi-36a4badb009ce8300210a00d8078c1ef6 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00e108174999 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00e9088d9443 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00f109055358 ONLINE 0 0 0
scsi-36a4badb009ce8300210af40b0750b87f FAULTED 1 375 0 too many errors
errors: No known data errors
2.2 Filesystem use
Presuming I have a pool called zfs-datastore create a /zfs-datastore/apache filesystem
zfs create zfs-datastore/apache
For destroying zpool, you can use
zfs destroy zfs-datastore/apache
If you want remove all recursive, use the recusive options -r = all children, -R = all dependants
2.2.1 How to mount zfs pool
zfs mount data01
# you can create temporary mount that expires after unmounting
zfs mount -o mountpoint=/tmpmnt data01/oracle
Note: there are all the normal mount options that you can apply i.e ro/rw, setuid
2.2.2 What is zfs snapshot ?
Snapshotting is like taking a picture, delta changes are recorded to the snapshot when the original file system changes, to remove a dataset all previous snapshots have to be removed, you can also rename snapshots. You cannot destroy a snapshot if it has a clone
# creating a snapshot
zfs snapshot zfs-datastore@28July2017
# renaming a snapshot
zfs snapshot zfs-datastore@28July2017 zfs-datastore@today_1145
# destroying a snapshot
zfs destroy zfs-datastore@today_1145
The advantage of snapshot is rollback. By default you can only rollback to the latest snapshot, to rollback to older one you must delete all newer snapshots
zfs rollback zfs-datastore@today_1145
2.2.3 How to replace broken disk ?
List the zpools and identify the failed disk
# zpool status
pool: zroot
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
repaired.
scan: scrub repaired 52K in 43h23m with 0 errors on Mon Nov 12 19:47:06 2018
config:
NAME STATE READ WRITE CKSUM
zroot DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
scsi-36a4badb009ce8300210a00c906a9ca2b ONLINE 0 0 0
scsi-36a4badb009ce8300210a00d8078c1ef6 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00e108174999 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00e9088d9443 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00f109055358 ONLINE 0 0 0
scsi-36a4badb009ce8300210af40b0750b87f FAULTED 1 375 0 too many errors
=> Here you can see that the disk with the id scsi-36a4badb009ce8300210af40b0750b87f is broken
Take the disk offline:
# zpool offline zroot scsi-36a4badb009ce8300210af40b0750b87f
Next, physically replace the disk
Next, make sure that the new disk has a GPT partition label (and not MBR):
# parted /dev/sdg # mklabel gpt # quit
Next, get the scsi-id of the new disk:
# ls -al /dev/disk/by-id/ total 0 lrwxrwxrwx 1 root root 9 Jan 14 16:36 scsi-36a4badb009ce8300210a00e9088d9443 -> ../../sde lrwxrwxrwx 1 root root 10 Jan 14 16:43 scsi-36a4badb009ce8300210a00e9088d9443-part1 -> ../../sde1 lrwxrwxrwx 1 root root 10 Jan 14 16:36 scsi-36a4badb009ce8300210a00e9088d9443-part9 -> ../../sde9 lrwxrwxrwx 1 root root 9 Jan 14 16:36 scsi-36a4badb009ce8300210a00f109055358 -> ../../sdf lrwxrwxrwx 1 root root 10 Jan 14 16:43 scsi-36a4badb009ce8300210a00f109055358-part1 -> ../../sdf1 lrwxrwxrwx 1 root root 10 Jan 14 16:36 scsi-36a4badb009ce8300210a00f109055358-part9 -> ../../sdf9 lrwxrwxrwx 1 root root 9 Jan 14 16:43 scsi-36a4badb009ce830023cf681c2fe62523 -> ../../sdg
In this example, the new /dev/sdg disk has the id scsi-36a4badb009ce830023cf681c2fe62523.
Then, perform the replacement in the ZFS config:
# zpool replace zroot scsi-36a4badb009ce8300210af40b0750b87f scsi-36a4badb009ce830023cf681c2fe62523
Verify the status of the ZFS pool:
# zpool status
pool: zroot
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Mon Jan 14 16:43:23 2019
62.8G scanned out of 19.7T at 29.9M/s, 191h18m to go
9.94G resilvered, 0.31% done
config:
NAME STATE READ WRITE CKSUM
zroot DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
scsi-36a4badb009ce8300210a00c906a9ca2b ONLINE 0 0 0
scsi-36a4badb009ce8300210a00d8078c1ef6 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00e108174999 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00e9088d9443 ONLINE 0 0 0
scsi-36a4badb009ce8300210a00f109055358 ONLINE 0 0 0
replacing-5 OFFLINE 0 0 0
scsi-36a4badb009ce8300210af40b0750b87f OFFLINE 0 0 0
scsi-36a4badb009ce830023cf681c2fe62523 ONLINE 0 0 0 (resilvering)