Funtoo Filesystem Guide, Part 5 - Revision history

Palica: /* data=journal Mode */

2012-11-03T09:49:02Z

‎data=journal Mode

Drobbins: /* data=journal Mode */

2011-05-23T01:32:44Z

‎data=journal Mode

Drobbins: /* data=writeback Mode */

2011-05-23T01:29:01Z

‎data=writeback Mode

Drobbins: /* Filesystem Check Intervals */

2011-05-23T01:28:51Z

‎Filesystem Check Intervals

Drobbins: /* Creating New Ext3 Filesystems */

2011-05-23T01:28:37Z

‎Creating New Ext3 Filesystems

Drobbins at 01:28, 23 May 2011

2011-05-23T01:28:27Z

Drobbins: /* data=writeback Mode */

2011-05-23T01:27:27Z

‎data=writeback Mode

Drobbins: Created page with "== Introduction == Back around 2002, when this article was originally written, Andrew Morton had a nice little introduction to using ext3 on his Web site. His site has since dis..."

2011-05-23T01:26:57Z

Created page with "== Introduction == Back around 2002, when this article was originally written, Andrew Morton had a nice little introduction to using ext3 on his Web site. His site has since dis..."

New page

== Introduction ==

Back around 2002, when this article was originally written, Andrew Morton had a
nice little introduction to using ext3 on his Web site. His site has since
disappeared, so in this updated article, I'll summarize Andrew Morton's
original documentation. Then, in the second half of the article, we'll delve
into some meatier ext3 topics, ones that I think you'll find very useful.

== Ext3 QuickStart ==

=== (Not) Patching the Kernel ===

These days, there is no need to patch your kernel for ext3. Ext3 has been
incorporated into the Linux kernel for a long time, and is very mature.

=== Converting ext2 to ext3 ===

One of the nice things about ext3 is that you can easily convert an ext2
filesystem to be an ext3 filesystem. All that you need to do is to create a
journal on the ext2 filesystem, as follows:

<pre>tune2fs -j /dev/sdXX</pre>

You can even do this while the filesystem is mounted. If it ''is'' currently
mounted, it will not function as an ext3 filesystem until you are able to
remount it as ext3. Remember to update <tt>/etc/fstab</tt> to so that it refers
to the new filesystem as an <tt>ext3</tt> filesystem.

=== Creating New Ext3 Filesystems ===

Simply use the <tt>-j</tt> option with <tt>mke2fs</tt>, as follows:

<pre>mke2fs -j /dev/sdXX</pre>

=== Switching Between Ext2 and Ext3 ===

It is possible to mount an ext3 filesystem as an ext2 filesystem, as long as
the ext3 filesystem has been cleanly unmounted. Simply specify a filesystem
type of <tt>ext2</tt> when mounting an ext3 filesystem with older kernels. You
can also specify a filesystem type of <tt>auto</tt> in <tt>/etc/fstab</tt>,
which will tell the <tt>mount</tt> and <tt>fsck</tt> commands to utilize the
filesystem as ext3 if such support is available, otherwise falling back to
ext2.

=== Fixing Dirty Ext3 Filesystems ===

If your ext3 filesystem was not cleanly unmounted, then you should be able clean it up using <tt>e2fsck</tt> as follows:

<pre>e2fsck -fy /dev/sdXX</pre>
The filesystem should now be mountable as ext2.

=== Ext3 Root Filesystem Tricks ===

If you want to force a Linux kernel to mount an ext3 filesystem as an ext2
filesystem, add the <tt>rootfstype=ext2</tt> kernel boot parameter to the
kernel boot parameters in <tt>lilo.conf</tt> or <tt>grub.conf</tt>.

To tell the Linux kernel to mount your root ext3 filesystem using a particular
journaling mode, use the <tt>rootflags</tt> kernel boot parameter as follows
(using a <tt>grub.conf</tt> example):

<pre>kernel /boot/bzImage rootflags=data=journal root=/dev/sda3</pre>

=== Filesystem Check Intervals ===

By default, <tt>e2fsck</tt> will perform an exhaustive filesystem check every
now and then, even if the filesystem was cleanly unmounted. By default, this
happens every twentieth mount or every 180 days, whichever comes first.

While this might have been handy for ext2 filesystems, it's not optimal for
ext3. To turn this automatic exhaustive filesystem check off, use the following
<tt>tune2fs</tt> command:

<pre>tune2fs -i 0 -c 0 /dev/sdXX</pre>

You can view the filesystem check interval (as well as lots of other
interesting information) by typing <tt>tune2fs -l /dev/sdXX</tt>.

=== External Journals ===

Ext3 supports the ability to place the journal on a separate persistent device. To create an external journal on <tt>/dev/sdb</tt>, type:

<pre>mke2fs -O journal_dev /dev/sdb</pre>
Then, to create an ext3 filesystem on <tt>/dev/sda3</tt> that uses this external journal, type:

<pre>mke2fs -J device=/dev/sdb /dev/sda3
mount /dev/sda3 /mnt/test -t ext3</pre>
== Ext3 Journaling Options and Write Latency ==

Ext3 allows you to choose from one of three data journaling modes at filesystem
mount time: <tt>data=writeback</tt>, <tt>data=ordered</tt>, and
<tt>data=journal</tt>.

To specify a journal mode, you can add the appropriate string
(<tt>data=journal</tt>, for example) to the options section of your
<tt>/etc/fstab</tt>, or specify the <tt>-o data=journal</tt> command-line
option when calling mount directly.

As we covered earlier in this article, if you'd like to specify the data
journaling method used for your root filesystem (<tt>data=ordered</tt> is the
default), you can to use a special kernel boot option called
<tt>rootflags</tt>. So, if you'd like to put your root filesystem into full
data journaling mode, add <tt>rootflags=data=journal</tt> to your kernel boot
options.

=== <tt>data=writeback</tt> Mode ===

In <tt>data=writeback mode</tt>, ext3 doesn't do any form of data journaling at
all, providing you with similar journaling found in the XFS, JFS, and ReiserFS
filesystems -- metadata only. As I explained in my
[http://www.funtoo.org/en/articles/linux/ffg/4/ previous article], this could
allow recently modified files to become corrupted in the event of an unexpected
reboot. Despite this drawback, <tt>data=writeback</tt> mode should give you the
best ext3 performance under most conditions.

=== <tt>data=ordered</tt> Mode ===In <tt>data=ordered</tt> mode, ext3 only officially journals metadata, but it logically groups metadata and data blocks into a single unit called atransaction. When it's time to write the new metadata out to disk, the
associated data blocks are written first. <tt>data=ordered</tt> mode
effectively solves the corruption problem found in <tt>data=writeback</tt> mode and most other journaled filesystems, and it does so without requiring fulldata journaling. In general, <tt>data=ordered</tt> ext3 filesystems performslightly slower than <tt>data=writeback</tt> filesystems, but significantlyfaster than their full data journaling counterparts.When appending data to files, <tt>data=ordered</tt> mode provides all of the integrity guarantees offered by ext3's full data journaling mode. However, ifpart of a file is being overwritten and the system crashes, it's possible that
the region being written will contain a combination of original blocks
interspersed with updated blocks. This is because <tt>data=ordered</tt>
provides no guarantees as to which blocks are overwritten first, so you can't
assume that just because overwritten block x was updated, that overwrittenblock x-1 was updated as well. Instead, <tt>data=ordered</tt> leaves the writeordering up to the hard drive's write cache. In general, this limitation
doesn't end up negatively impacting people very often, since file appends are
generally much more common than file overwrites. For this reason,<tt>data=ordered</tt> mode is a good higher-performance replacement for fulldata journaling.=== <tt>data=journal</tt> Mode ===
<tt>data=journal</tt> mode provides full data and metadata journaling. All new
data is written to the journal first, and then to its final location. In the
event of a crash, the journal can be replayed, bringing both data and metadata
into a consistent state.

Theoretically, <tt>data=journal</tt> mode is the slowest journaling mode of all, since data gets written to disk twice rather than once. However, it turns
out that in certain situations, <tt>data=journal</tt> mode can be blazingly
fast. Andrew Morton, after hearing reports on LKML that ext3
<tt>data=journal</tt> filesystems were giving people unbelievably greatinteractive filesystem performance, decided to put together a little test.
First, he created simple shell script designed to write data to a test
filesystem as quickly as possible:

<pre>while true
do dd if=/dev/zero of=largefile bs=16384 count=131072
done</pre>
While data was being written to the test filesystem, he attempted to read 16MB of data from another ext2 fil
esystem on the same disk, timing the results:
<pre># time cat 16-meg-file > /dev/null</pre>

The results were astounding. <tt>data=journal</tt> mode allowed the 16 MB file to be read from 9 to over 13 times faster than other ext3 modes, ReiserFS, and even ext2 (which has no journaling overhead):

<table>
<thead>
<tr class="header">
<th align="left">Filesystem</th>
<th align="left">16 MB read time (seconds)</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="left">ext2</td>
<td align="left">78</td>
</tr>
<tr class="even">
<td align="left">ReiserFS</td>
<td align="left">67</td>
</tr>
<tr class="odd">
<td align="left">ext3 data=ordered</td>
<td align="left">93</td>
</tr>
<tr class="even">
<td align="left">ext3 data=writeback</td>
<td align="left">74</td>
</tr>
<tr class="odd">
<td align="left">ext3 data=journal</td>
<td align="left">7</td>
</tr>
</tbody>
</table>

Andrew repeated this test, but tried to read a 16 MB file from the test filesystem (rather than a different filesystem), and he got identical results. So, what does this mean? Somehow, ext3's <tt>data=journal</tt> mode is incredibly well-suited to situations where data needs to be read from and written to disk at the same time. Therefore, ext3's <tt>data=journal</tt> mode, which was assumed to be the slowest of all ext3 modes in nearly all conditions, actually turns out to have a major performance advantage in busy environments where interactive IO performance needs to be maximized. Maybe <tt>data=journal</tt> mode isn't so sluggish after all!

Andrew is still trying to figure out exactly why data=journal mode is doing so much better than everything else. When he does, he may be able to add the necessary tweaks to the rest of ext3 so that data=writeback and data=ordered modes see some benefit as well.

I hope to see you soon when I release the next installment in the Funtoo Filesystem Guide!

@@ Line 173: / Line 173: @@
 I hope to see you soon when I release the next installment in the Funtoo Filesystem Guide!
 [[Category:Filesystem Guides]]
 [[Category:Articles]]

← Older revision		Revision as of 01:28, 23 May 2011
Line 32:		Line 32:

	<pre>mke2fs -j /dev/sdXX</pre>		<pre>mke2fs -j /dev/sdXX</pre>
−

	=== Switching Between Ext2 and Ext3 ===		=== Switching Between Ext2 and Ext3 ===

@@ Line 149: / Line 149: @@
 The results were astounding. <tt>data=journal</tt> mode allowed the 16 MB file to be read from 9 to over 13 times faster than other ext3 modes, ReiserFS, and even ext2 (which has no journaling overhead):
-<table>
+{| {{Table}}
-<thead>
+!Filesystem||16 MB read time (seconds)
-<tr class="header">
+|-
-<th align="left">Filesystem</th>
+|ext2
-<th align="left">16 MB read time (seconds)</th>
+|78
-</tr>
+|-
-</thead>
+|ReiserFS
-<tbody>
+|67
-<tr class="odd">
+|-
-<td align="left">ext2</td>
+|ext3 data=ordered
-<td align="left">78</td>
+|93
-</tr>
+|-
-<tr class="even">
+|ext3 data=writeback
-<td align="left">ReiserFS</td>
+|74
-<td align="left">67</td>
+|-
-</tr>
+|ext3 data=journal
-<tr class="odd">
+|7
-<td align="left">ext3 data=ordered</td>
+|}
-<td align="left">93</td>
-</tr>
-<tr class="even">
-<td align="left">ext3 data=writeback</td>
-<td align="left">74</td>
-</tr>
-<tr class="odd">
-<td align="left">ext3 data=journal</td>
-<td align="left">7</td>
-</tr>
-</tbody>
-</table>
 Andrew repeated this test, but tried to read a 16 MB file from the test filesystem (rather than a different filesystem), and he got identical results. So, what does this mean? Somehow, ext3's <tt>data=journal</tt> mode is incredibly well-suited to situations where data needs to be read from and written to disk at the same time. Therefore, ext3's <tt>data=journal</tt> mode, which was assumed to be the slowest of all ext3 modes in nearly all conditions, actually turns out to have a major performance advantage in busy environments where interactive IO performance needs to be maximized. Maybe <tt>data=journal</tt> mode isn't so sluggish after all!

@@ Line 113: / Line 113: @@
 reboot. Despite this drawback, <tt>data=writeback</tt> mode should give you the
 best ext3 performance under most conditions.
 === <tt>data=ordered</tt> Mode ===

@@ Line 76: / Line 76: @@
 You can view the filesystem check interval (as well as lots of other
 interesting information) by typing <tt>tune2fs -l /dev/sdXX</tt>.
 === External Journals ===

@@ Line 143: / Line 143: @@
 <pre>while true
-do        dd if=/dev/zero of=largefile bs=16384 count=131072
+do
 done</pre>
-While data was being written to the test filesystem, he attempted to read 16MB of data from another ext2 fil
+While data was being written to the test filesystem, he attempted to read 16MB of data from another ext2 filesystem on the same disk, timing the results:
-esystem on the same disk, timing the results:
 <pre># time cat 16-meg-file &gt; /dev/null</pre>
@@ Line 187: / Line 188: @@
 I hope to see you soon when I release the next installment in the Funtoo Filesystem Guide!

@@ Line 117: / Line 117: @@
-=== <tt>data=ordered</tt> Mode ===In <tt>data=ordered</tt> mode, ext3 only officially journals metadata, but it logically groups metadata and data blocks into a single unit called atransaction. When it's time to write the new metadata out to disk, the
+=== <tt>data=ordered</tt> Mode ===
 associated data blocks are written first. <tt>data=ordered</tt> mode
-effectively solves the corruption problem found in <tt>data=writeback</tt> mode and most other journaled filesystems, and it does so without requiring fulldata journaling. In general, <tt>data=ordered</tt> ext3 filesystems performslightly slower than <tt>data=writeback</tt> filesystems, but significantlyfaster than their full data journaling counterparts.When appending data to files, <tt>data=ordered</tt> mode provides all of the integrity guarantees offered by ext3's full data journaling mode. However, ifpart of a file is being overwritten and the system crashes, it's possible that
+In <tt>data=ordered</tt> mode, ext3 only officially journals metadata, but it logically groups metadata and data blocks into a single unit called atransaction. When it's time to write the new metadata out to disk, the associated data blocks are written first. <tt>data=ordered</tt> mode
 the region being written will contain a combination of original blocks
 interspersed with updated blocks. This is because <tt>data=ordered</tt>
@@ Line 125: / Line 126: @@
 assume that just because overwritten block x was updated, that overwrittenblock x-1 was updated as well. Instead, <tt>data=ordered</tt> leaves the writeordering up to the hard drive's write cache. In general, this limitation
 doesn't end up negatively impacting people very often, since file appends are
-generally much more common than file overwrites. For this reason,<tt>data=ordered</tt> mode is a good higher-performance replacement for fulldata journaling.=== <tt>data=journal</tt> Mode ===
+generally much more common than file overwrites. For this reason,<tt>data=ordered</tt> mode is a good higher-performance replacement for fulldata journaling.
 <tt>data=journal</tt> mode provides full data and metadata journaling. All new
 data is written to the journal first, and then to its final location. In the