Comments on navindra's blog: KDE Dot News: Ext3's Miserable Failure

I'm pretty much committed to using the filesystem ...

2005-05-18T16:28:00.000-04:00

I'm pretty much committed to using the filesystem for the backend, but I might consider MySQL for non-critical functions such as maintaining a seachable index. We'll see.

re: "why not mysql":Here's the preface to that doc...

2005-05-18T16:20:00.000-04:00

re: "why not mysql":

Here's the preface to that document:

NOTE: This Document was written in May 2000. Thus, it is outdated and does not represent the latest data concerning MySQL.

I don't know enough to say what the mysql folks have been up to for the last 5 years but i expect it's something.

Yes, I used "du". I obviously had no idea what I ...

2004-10-20T20:24:00.000-04:00

Yes, I used "du". I obviously had no idea what I was getting myself into. :-)

Sadly, I've certainly developed a healthy dose of Fear, Uncertainty and Doubt (FUD) over ReiserFS at this point. Incidentally, KTown is a major KDE server and seems to have been using it for ages.

How did you measure how much space the thing takes...

2004-10-20T19:21:00.000-04:00

How did you measure how much space the thing takes in ReiserFS? I don't know about notail, but with tail packing the usage reported by "du" is very wrong. If I remember correctly, it was seeing 50GB stored on my 25GB drive once. Tail packing also makes ReiserFS a bit slower.

I've heard stories about Ext3 corrupting data, too, years ago. But ReiserFS is particularly nasty. I saw nice articles about ReiserFS from the Gentoo main developer, and then the main Portage server thing went down for a few days a year or so ago.

hmm, please, consider also total amount of non-fix...

2004-10-19T02:27:00.000-04:00

hmm, please, consider also total amount of non-fixed bugs for reiserfs. at the time I looked for the last time (month ago?), there was about 15 of them...

Why Not MySQL?

2004-10-18T22:58:00.000-04:00

Why Not MySQL?

One thing where ext3 shines (as attested by Nigel ...

2004-10-18T19:55:00.000-04:00

One thing where ext3 shines (as attested by Nigel Cunningham (swsusp2)) is for Laptop hibernation.

Since running on FC2/reiserfs, and using hibernation, I've experienced 3-5 data curruption events.

I'm getting a bit paranoid now.

Maybe ext3 is better for hibenation support then.

http://livejournal.com/users/lotso

I can't believe you ended up on OSNews! The commen...

2004-10-18T16:06:00.000-04:00

I can't believe you ended up on OSNews! The comments show as well....

What about a .bz2?

2004-10-18T13:59:00.000-04:00

What about a .bz2?

I would be curious to see all this under XFS. I ...

2004-10-18T13:47:00.000-04:00

I would be curious to see all this under XFS. I have been a huge fan of it, and I would just like to see the numbers.

Sorry to disappoint, but I don't have access to an...

2004-10-18T11:28:00.000-04:00

Sorry to disappoint, but I don't have access to any ReiserFS 4 systems. :)

jayKayEss, I'll keep your comments in mind, thanks. I'll know where to find you when I need MySQL expertise. ;)

Honestly all, this was really just a quick-and-dirty dry run. Don't expect anything to happen soon, I have a thesis to complete first.

I really strongly suggest that you do some mySQL b...

2004-10-18T10:11:00.000-04:00

I really strongly suggest that you do some mySQL benchmarking before you go with a file-based solution (it just sounds really difficult to administrate such a thing to me.) We run a large website with ~8MM users, with message boards, custom search, and ecommerce, 24 servers, all off of a single mySQL database, and the results are great! Also, mySQL has an internal cache so that not every read results in a disk access.

As for storage space, mySQL will probably take up a lot with its indices, but these will really speed access time. You should make sure that your tables are properly indexed, or else your benchmarks will be disappointing.

I agree. Testing reiser4 with the same method woul...

2004-10-18T09:57:00.000-04:00

I agree. Testing reiser4 with the same method would be really cool (hint hint :) )

I'd really love to see some reiser4 figures here. ...

2004-10-18T05:05:00.000-04:00

I'd really love to see some reiser4 figures here. From what I've red it should reduce the used amount of space considerably while still being faster than reiser3.

Note to self: Some interesting comments here. Inc...

2004-10-18T04:42:00.000-04:00

Note to self: Some interesting comments here. Including a reliability study on ReiserFS vs ext3.

Hmm, thanks again for the cool comments, both of y...

2004-10-18T01:59:00.000-04:00

Hmm, thanks again for the cool comments, both of you.

Seems my systems don't have/allow the +I option in chattr, I'll have to look into that or just do the tune2fs thing.

data=writeback does not work with the remount option, so I'm going to have to try that another time. I would have thought that would affect only writing though (I guess it would affect the system in general though). So far my tests have been mostly reading of already written stuff.

Looking forward to the results!

You could try just "chattr +I _dir_". It may turn ...

2004-10-18T01:43:00.000-04:00

You could try just "chattr +I _dir_". It may turn on the dir_index feature without needing to reboot for a single dir (see also chattr -R). But I'm not 100% sure.

The other comment about data=ordered is also very valid.

Even with notail, reiser does pack directories much better thanks to its tree structure. notail would probably give you even more space benefit (if your files are small, as I've understood). That probably does reduce performance somewhat (but caching may help).

notail won't work with ext3, there used to be a tailmerge patch for ext2 (ext3?), but I don't think its supported anymore.

Why not try NTFS? Supposedly NTFS 5.0 (Windows 20...

2004-10-17T23:32:00.000-04:00

Why not try NTFS? Supposedly NTFS 5.0 (Windows 2003 Server's default fs) is pretty efficient...

For the speed thing, you may want to retest ext3 w...

2004-10-17T18:45:00.000-04:00

For the speed thing, you may want to retest ext3 with different default values, as ext3 takes very conservative values that harm performance _at_purpose_

As they already said, ext3 has a "htree" feature which can help in the "lots of files/directories" case. Ext3 also has a new block allocator , "orlov"....and you guess well, if you really want to test it you need to fill your filesystem with a kernel that used orlov to allocate those files (2.6 supports both things)

Another thing is the journaling modes. Ext3 has 3 journaling modes, data=ordered, data=writeback and data=journal. "ordered" is the default, and when it's enabled ext3 ensures that data are written before the metadata that refers to that data. It may look stupid, but xfs jfs and reiser in the default mode don't care about this, and they can write the metadata before the data...in reiser you can enable it with the data=ordered mount parameter if you want the ext3 behaviour. XFS and JFS don't even have that option. If you want to have the xfs/jfs behaviour (more speed but less data safety) then use commit=writeback.

And the last interesting parameter is commit. commit defaults to 5 in ext3, and it means that ext3 will sync all its buffers - yes, that's a sync every 5 seconds. It harms performance a _lot_ and ext3 defaults to that value because ext3 developers are really _paranoic_ about data safety. You can mout ext3 it with huge values to increase performance

In short, if you want to do _fair_ benchmarks reiser vs ext3, you must:
o use htree
o use 2.6 (orlov, also ext3 is a _lot_ faster in 2.6, reiserfs too I guess)
o mount your filesystem with data=writeback
o mount your filesystem with commit=9999 (or whatever huge value you want)
o read Documentation/filesystems/ext3.txt

That said, I don't really think ext3 will shine. ext3 is great for data safety, and the fact that ext3 developers have done things like adding a feature which syncs all your filesystem every 5 seconds by default is a sign of that. I expect ext3 to be "slow", but not _that_ slow

Very interesting and informative comment. Since s...

2004-10-17T16:59:00.000-04:00

Very interesting and informative comment.

Since switching to reiserfs is not a short-term project, your tips to tune ext3 may come in handy.

dir_index is not currently enabled and doesn't seem to be something I can easily try right now (requires tune2fs followed by fsck). It does sound like it will help a lot here.

Incidentally reiserfs is mounted with notail on ktown, but that option didn't work with ext3.

Thanks a lot for the pointers.

It is to be expected that directory tree on ext3 t...

2004-10-17T16:46:00.000-04:00

It is to be expected that directory tree on ext3 takes more space due to ext3's traditional block orientated nature. You could alleviate that somewhat by using smaller block size (1024 or 512) instead of the default (4k, probably.) This the sort of thing that is traditionally left for the administrator (who knows what sort of data will be stored on the fs) on unix - freedom of choice if you like.

Reiserfs on the other hand packs the data much more tightly. With some access patterns and with larger files, this introduces results into some performance loss. There used to be tail merging patches for ext2 (ext3 uses the same on disk layout) that did something similar.

With regards to the ext3 performance: it used to be much words with large amount files/directories under same directory. Nowadays ext3 has dirindex feature that remedies the O(N) behaviour. You could make sure that dirindex was enabled on your fs (tune2fs -l _device_ | grep dir_index; lsattr -d _dir_ | grep I).

How about Reiser4? It will be very insteresting to...

2004-10-17T16:25:00.000-04:00

How about Reiser4? It will be very insteresting to see how Reiser4 will do.

MMh, ext3 sucks horribly, that's not big news. I'v...

2004-10-17T16:18:00.000-04:00

MMh, ext3 sucks horribly, that's not big news. I've tested thoroughly ext3, reiserfs and xfs and ext3 always lose, always. Depending upon your job, either reiserfs or xfs is (usually enormously) better.

You're right. That's a very interesting and perti...

2004-10-17T16:00:00.000-04:00

You're right. That's a very interesting and pertinent question. I've added the numbers.

I would be interested to see what size a raw .tar ...

2004-10-17T14:06:00.000-04:00

I would be interested to see what size a raw .tar is.