ZFS Fun - Revision history

5.28.185.78: /* ZFS features and limitations */

2013-02-18T10:11:03Z

‎ZFS features and limitations

Palica at 21:17, 21 January 2013

2013-01-21T21:17:06Z

71.204.51.184: /* ZFS alternatives */

2013-01-18T23:39:37Z

‎ZFS alternatives

76.217.25.82: /* Linux */

2012-09-20T05:49:13Z

‎Linux

Mmatk: /* BSD /

2012-02-25T21:43:43Z

‎*BSD

178.59.229.7: /* BSD /

2012-02-09T19:46:48Z

‎*BSD

Drobbins at 15:58, 5 January 2012

2012-01-05T15:58:10Z

178.76.181.133 at 10:39, 2 January 2012

2012-01-02T10:39:59Z

404 Error: /* Some statistics */

2011-09-13T13:16:15Z

‎Some statistics

404 Error: /* Some statistics */

2011-09-13T13:15:30Z

‎Some statistics

@@ Line 20: / Line 20: @@
 * Dynamic striping to optimize data throughput
 * Variable block length
-* Data duplication
+* Data deduplication
 * Automatic pool re-silvering
 * Transparent data compression / encryption (later requires Solaris 11)

@@ Line 68: / Line 68: @@
 * WAFL seems to have severe limitation [http://unixconsult.org/wafl/ZFS%20vs%20WAFL.html] (document is not dated), also an interesting article lies [http://blogs.netapp.com/dave/2008/12/is-wafl-a-files.html here]
 * BTRFS is advancing every week but it still lacks such features like the capability of emulating a virtual block device over a storage pool (zvol) and it has a built-in support for RAID-0/1 only. At date of writing, it is still experimental where as ZFS is used on big production servers.
-* VxFS has also been targeted by comparisons like [http://blogs.oracle.com/dom/entry/zfs_v_vxfs_iozone this one] (a bit [http://www.symantec.com/connect/blogs/suns-comparision-vxfs-and-zfs-scalability-flawed controversial]). VxFS has been known in the industry since 1993 and has a legendary flexibility. Symantec acquired VxFS and now propose a basic version (no clustering for example) otf it under the same [http://www.symantec.com/enterprise/sfbasic/index.jsp Veritas Storage Foundation Basic]
+* VxFS has also been targeted by comparisons like [http://blogs.oracle.com/dom/entry/zfs_v_vxfs_iozone this one] (a bit [http://www.symantec.com/connect/blogs/suns-comparision-vxfs-and-zfs-scalability-flawed controversial]). VxFS has been known in the industry since 1993 and is known for its legendary flexibility. Symantec acquired VxFS and proposed a basic version (no clustering for example) of it under the same [http://www.symantec.com/enterprise/sfbasic/index.jsp Veritas Storage Foundation Basic]
 * An interesting discussion about modern filesystems can be found on [http://www.osnews.com/story/19665/Solaris_Filesystem_Choices OSNews.com]
@@ Line 76: / Line 76: @@
 * both are transactional filesystems (in BTRFS a a transaction is a sequence of low level operations)
 * both implement for example the pool concept (called a "volume" in BTRFS)
-* both can do snapshots although in ZFS a snapshot is a read only thing and its attributes can't be modified. BTRFS on his side has has writable snapshots (known as clones in ZFS)
+* both can do snapshots although in ZFS a snapshot is a read only thing and its attributes can't be modified. BTRFS on the other hand has writable snapshots (known as clones in ZFS)
 * both can organize their storage pool in several logical divisions (called datasets in ZFS and subvolumes in BTRFS).
-* As their equivalent in in BTRFS (subvolumes), ZFS datasets appears as directories
+* As their equivalent in BTRFS (subvolumes), ZFS datasets appears as directories
 * Where as a ZFS snapshot is "hidden" in a sub-directory (named .zfs), BTRFS snapshots appears as visible directories
-* While ZFS manages rollback in a transparent manner (the filesystem knows where and how rollback the data), rollingback data in BTRFS requires a bit more work as the system administrator must umount/remount a BTRFS subvolume.
+* While ZFS manages rollback in a transparent manner (the filesystem knows where and how to rollback the data), rollingback data in BTRFS requires a bit more work as the system administrator must umount/remount a BTRFS subvolume.
 * ZFS has a kind of sophisticated RAID-5 called RAID-Z (and now RAID-Z2 ~ RAID-6), similar capabilities are planned for BTRFS but not yet available as of september 2011
 * A ZFS filesytem can be snapshotted and sent through the network, BTRFS has not yet reach that integration level

@@ Line 67: / Line 67: @@
 * WAFL seems to have severe limitation [http://unixconsult.org/wafl/ZFS%20vs%20WAFL.html] (document is not dated), also an interesting article lies [http://blogs.netapp.com/dave/2008/12/is-wafl-a-files.html here]
-* BTRFS is advancing every week but it still lacks suck a feature like the capability of emulating a virtual block device over a storage pool (zvol) and it has a built-in support for RAID-0/1 only. At date of writing, it is still experimental where as ZFS is used on big production servers.
+* BTRFS is advancing every week but it still lacks such features like the capability of emulating a virtual block device over a storage pool (zvol) and it has a built-in support for RAID-0/1 only. At date of writing, it is still experimental where as ZFS is used on big production servers.
 * VxFS has also been targeted by comparisons like [http://blogs.oracle.com/dom/entry/zfs_v_vxfs_iozone this one] (a bit [http://www.symantec.com/connect/blogs/suns-comparision-vxfs-and-zfs-scalability-flawed controversial]). VxFS has been known in the industry since 1993 and has a legendary flexibility. Symantec acquired VxFS and now propose a basic version (no clustering for example) otf it under the same [http://www.symantec.com/enterprise/sfbasic/index.jsp Veritas Storage Foundation Basic]
 * An interesting discussion about modern filesystems can be found on [http://www.osnews.com/story/19665/Solaris_Filesystem_Choices OSNews.com]

@@ Line 47: / Line 47: @@
 * [http://zfsonlinux.org ZFS on Linux]: a native implementation of ZFS in kernel mode. The project claims to have ''"a functional and stable SPA, DMU, ZVOL, and Posix Layer (ZPL)"''. Current upstream version is 0.6.0-rc5 (can mount ZFS filesystems and support zpool version 28), however neither Gentoo and Funtoo have ebuilds for this port (yet). As ZFS on Linux is an out-of-tree Linux kernel implementation, patches must be waited after each Linux kernel release. As of september 2011, the project claims to have '''support for Linux 2.6.26 up to Linux 3.0.0''', Linux 3.1 series kernels are not officially supported and ZFS on Linux is far from being mature and usable on production systems. It suffers from a couple of major issues like:
-** Crash when used with a preemptable kernel (see [https://github.com/zfsonlinux/zfs/issues/83 issue 83])
+** <s>Crash when used with a preemptable kernel (see [https://github.com/zfsonlinux/zfs/issues/83 issue 83])</s> Fixed Aug 27, 2012
-** Deadlocks can happen with some debug options (see [https://github.com/zfsonlinux/zfs/issues/167 issue 167])
+** <s>Deadlocks can happen with some debug options (see [https://github.com/zfsonlinux/zfs/issues/167 issue 167])</s> Fixed Oct 19, 2011
 === Solaris/OpenIndiana ===

@@ Line 60: / Line 60: @@
 * '''FreeBSD''': ZFS is present in FreeBSD since FreeBSD 7 (zpool version 6) and FreeBSD can boot on a ZFS volume (zfsboot). ZFS support has been vastly enhanced in FreeBSD 8.x (8.2 supports zpool version 15, version 8.3 will support version 28) and FreeBSD 9 (supporting zpool version 28). ZFS in FreeBSD is now considered as fully functional and mature. FreeBSD derivatives such as the popular [http://www.freenas.org FreeNAS] takes befenits of ZFS and integrated it in their tools. In the case of that latter,  it have, for example, supports for zvol though its Web management interface (FreeNAS >= 8.0.1).
-* '''NetBSD''': ZFS has been srtaed to be ported as a GSoC project in 2007 and is present in the NetBSD mainstream since 2009 (zpool version 13).
+* '''NetBSD''': ZFS has been started to be ported as a GSoC project in 2007 and is present in the NetBSD mainstream since 2009 (zpool version 13).
 * '''OpenBSD''': No ZFS support yet and not planned until Oracle changes some policies according to the project FAQ.

@@ Line 58: / Line 58: @@
 === *BSD ===
-* '''FreeBSD''': ZFS is present in FreeBSD since FreeBSD 7 (zpool version 6) and FreeBSD can boot on a ZFS volume (zfsboot). ZFS support has been vastly enhanced in FreeBSD 8.x (8.0 supports zpool version 13) and FreeBSD 9 (currently at beta-1, that later supporting zpool version 28). ZFS in FreeBSD is now considered as fully functional and mature. FreeBSD derivatives such as the popular [http://www.freenas.org FreeNAS] takes befenits of ZFS and integrated it in their tools. In the case of that latter,  it have, for example, supports for zvol though its Web management interface (FreeNAS >= 8.0.1).
+* '''FreeBSD''': ZFS is present in FreeBSD since FreeBSD 7 (zpool version 6) and FreeBSD can boot on a ZFS volume (zfsboot). ZFS support has been vastly enhanced in FreeBSD 8.x (8.2 supports zpool version 15, version 8.3 will support version 28) and FreeBSD 9 (supporting zpool version 28). ZFS in FreeBSD is now considered as fully functional and mature. FreeBSD derivatives such as the popular [http://www.freenas.org FreeNAS] takes befenits of ZFS and integrated it in their tools. In the case of that latter,  it have, for example, supports for zvol though its Web management interface (FreeNAS >= 8.0.1).
 * '''NetBSD''': ZFS has been srtaed to be ported as a GSoC project in 2007 and is present in the NetBSD mainstream since 2009 (zpool version 13).

← Older revision		Revision as of 15:58, 5 January 2012
Line 1,009:		Line 1,009:
	[[Category:Labs]]		[[Category:Labs]]
	[[Category:Articles]]		[[Category:Articles]]
		+	[[Category:Filesystems]]

@@ Line 3: / Line 3: @@
 == ZFS features and limitations ==
-ZFS offers an impressive amount of features even putting aside its hybrid nature (both a filesystem and a volume manager -- zvol) covered in detail on [http://en.wikipedia.org/wiki/ZFS Wikipedia]. One of the most fundamental point to keep in mind about ZFS is it '''targets a legendary reliability in terms of preserving data integrity'''. ZFS uses several techniques to detect and repair (self-healing) corrupted data, simply speaking it makes an aggressive use of checksums and relies on data redundancy, the price pay is it requires a bit more CPU processing power than traditional filesystems and RAID solution. However, the [http://en.wikipedia.org/wiki/ZFS Wikipedia article about ZFS] also mention it is strongly discouraged to use ZFS over classic RAID arrays as it can not control the data redundancy,thus ruining most of its benefits.
+ZFS offers an impressive amount of features even putting aside its hybrid nature (both a filesystem and a volume manager -- zvol) covered in detail on [http://en.wikipedia.org/wiki/ZFS Wikipedia]. One of the most fundamental points to keep in mind about ZFS is it '''targets a legendary reliability in terms of preserving data integrity'''. ZFS uses several techniques to detect and repair (self-healing) corrupted data, simply speaking it makes an aggressive use of checksums and relies on data redundancy, the price pay is it requires a bit more CPU processing power than traditional filesystems and RAID solution. However, the [http://en.wikipedia.org/wiki/ZFS Wikipedia article about ZFS] also mention it is strongly discouraged to use ZFS over classic RAID arrays as it can not control the data redundancy,thus ruining most of its benefits.
 In short, ZFS has the following features (not exhaustive):

@@ Line 919: / Line 919: @@
 * Even ECC memory cannot be always be helpful: 3 double-bit errors (uncorrectable) occurred in 3 months on 1300 nodes. Bad news: it should be '''zero'''.
-* RAID systems cannot protect in all cases: monitoring 492 RAID controller for 4 weeks showed an average error rate of ~10^14, giving roughly 300 errors for every 2.4 petabytes
+* RAID systems cannot protect in all cases: monitoring 492 RAID controller for 4 weeks showed an average error rate of 1 per ~10^14 bits, giving roughly 300 errors for every 2.4 petabytes
 * Magnetic storage is still not reliable even on high-end datacenter class drives: 500 errors found over 100 nodes while writing 2 GB file to 3000+ nodes every 2 hours then read it again and again for 5 weeks.

@@ Line 905: / Line 905: @@
 == Some statistics ==
-It is not a secret to tell that a general trend in the IT industry is the exponential growth of data quantities. Just thinking about the amount of data Youtube, Google or Facebook generates every day taking the case of the first (http://www.website-monitoring.com/blog/2010/05/17/youtube-facts-and-figures-history-statistics/):
+It is not a secret to tell that a general trend in the IT industry is the exponential growth of data quantities. Just thinking about the amount of data Youtube, Google or Facebook generates every day taking the case of the first [http://www.website-monitoring.com/blog/2010/05/17/youtube-facts-and-figures-history-statistics some statistics] gives:
 * 24 hours of video is generated every ''minute'' in March 2010 (May 2009 - 20h / October 2008 - 15h / May 2008 - 13h)
 * More than 2 ''billions'' views a day
@@ Line 918: / Line 918: @@
 What is true with Facebook and Youtube is also true with many other cases (think one minutes about the amount of data stored in iTunes) especially with the growing popularity of cloud computing infrastructures. Despite the progress of the technology a "bottleneck" still exists: the storage reliability is nearly the same over the years. If only one organization in the world generate huge quantities of data it would be the [http://public.web.cern.ch CERN] (''Conseil Européen pour la Recherche Nucléaire'', now officially known as ''European Organization for Nuclear Research'') as their experiments can generate spikes of many terabytes of data within a few seconds. A study done in 2007 quoted by a [http://www.zdnet.com/blog/storage/data-corruption-is-worse-than-you-know/191 ZDNet article] reveals that:
-* Even ECC memory error cannot be always be helpful: 3 double-bit errors (uncorrectable) occurred in 3 months on 1300 nodes. Bad news: it should be '''zero'''.
+* Even ECC memory cannot be always be helpful: 3 double-bit errors (uncorrectable) occurred in 3 months on 1300 nodes. Bad news: it should be '''zero'''.
 * RAID systems cannot protect in all cases: monitoring 492 RAID controller for 4 weeks showed an average error rate of ~10^14, giving roughly 300 errors for every 2.4 petabytes
 * Magnetic storage is still not reliable even on high-end datacenter class drives: 500 errors found over 100 nodes while writing 2 GB file to 3000+ nodes every 2 hours then read it again and again for 5 weeks.

ZFS Fun - Revision history

5.28.185.78: /* ZFS features and limitations */

Palica at 21:17, 21 January 2013

71.204.51.184: /* ZFS alternatives */

76.217.25.82: /* Linux */

Mmatk: /* *BSD */

178.59.229.7: /* *BSD */

Drobbins at 15:58, 5 January 2012

178.76.181.133 at 10:39, 2 January 2012

404 Error: /* Some statistics */

404 Error: /* Some statistics */

Mmatk: /* BSD /

178.59.229.7: /* BSD /