Filesystems HOWTO: Introduction

The Filesystems HOWTO is about filesystems and accessing filesystems from various OS. Although this document has been put together to the best of my knowledge, it may and probably does contain mistakes. Please if you find some mistake or outdated information, let me know. I will try to keep this document up to date and as error free as possible. Any contributions are also welcome, so if you want to write anything about filesystems, please contact me via e-mail.

Before you read this HOWTO it's recommended to read Stein Gjoen's Disk-HOWTO (you can obtain it from http://sunsite.unc.edu/LDP/HOWTO/ ).

This HOWTO can be obtained from http://penguin.cz/~mhi/fs/ or http://metalab.unc.edu/filesystems/howto/.

If you are Japanese user, you might be interested that FUJIWARA Teruyoshi translated this HOWTO to Japanese. It is available at http://www.linux.or.jp/JF/JFdocs/Filesystems-HOWTO.html. SGML source file can be downloaded from ftp://ftp.linet.gr.jp/pub/JF/sgml/Filesystems-HOWTO.sgml.gz.

1.1 Copyright

This HOWTO is free document; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This HOWTO is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this document or GNU CC; if not, write to the: Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

1.2 Filesystems mailing-list

You may want to join Filesystems mailing list. It's intended to be a good source of information for both end-users and developers. So if you have anything to do with filesystems, join ;-) To subscribe send email to < majordomo@penguin.cz> and in the BODY (not the subject) of the email message put (without quotes): "subscribe fs-l".

Linux kernel filesystems mailing-list

To join Linux kernel filesystems mailing list linux-fsdev@vger.rutgers.edu, send e-mail to listserv@vger.rutgers.edu. Put "subscribe linux-fsdev" in message body.

FreeBSD filesystems mailing-list

To join techical FreeBSD filesystems mailing list freebsd-fs@FreeBSD.org, send e-mail to majordomo@FreeBSD.org. Put "subscribe freebsd-fs" in message body.

1.3 Filesystems collection at metalab.unc.edu

Filesystems collection is FTP/WWW site providing useful information about filesystems and filesystem-related programs and drivers. It lives at http://metalab.unc.edu/filesystems/, or FTP-only at ftp://metalab.unc.edu/pub/docs/filesystems/.

1.4 Credits

The original "Filesystems access HOWTO" was written by Georgatos Photis (see his homepage at http://students.ceid.upatras.gr/~gef/). This HOWTO contains a lot of information from his webpage. Thanks, Gef.

FUJIWARA Teruyoshi <fujiwara@linux.or.jp> translated this HOWTO to Japanese.

Other people who have contributed or helped me (directly or indirectly) with this HOWTO are, in alphabetical order:

Mariusz Borkowski <borkowsm@ii.pw.edu.pl> - ISO9660/RR info
Remy Card <card@masi.ibp.fr> - Ext2 filesystem introduction
Peter A. Dinda <pdinda@cs.cmu.edu> - HFS filesystem description
Alfonso De Gregorio <adg@speedcom.it> - TCFS filesystem info
Radek Machacka <radekm@sco.com> - Thanks for SCO UnixWare and SCO OpenServer
Andrey Shedel <andreys@cr.cyco.com> - Misc. updates
Peter Todd <retep2@home.com> - SFS filesystem info
Theodore Ts'o <tytso@mit.edu> - Ext2 filesystem introduction
Stephen Tweedie <sct@dcs.ed.ac.uk> - Ext2 filesystem introduction

Many thanks to the above people. If I have forgotten anyone, please let me know.

1.5 Filesystems accessibility map

This is filesystem accessibility "map", alphabetically ordered by operating system. You may find this list a little bit chaotic. It's because Linux sgmltools don't know tables.

YOU SEE THAT THIS `MAP' IS NOT STILL COMPLETE. I WILL TRY TO FINISH IT IN THE NEAR FUTURE.

FreeBSD: BSD FFS | Ext2 | HPFS | NTFS

NetBSD: BSD FFS | FAT12/16 | ISO9660

NetWare 2.x: NWFS-286

NetWare 3.x, 4.x: NWFS-386 | ISO9660

NetWare 5.x: NWFS-386 | NSS | ISO9660

OpenBSD: BSD FFS | FAT12/16

OS/2: Ext2 FS | FAT12/16/32 | HPFS | HPFS | ISO 9660 | JFS | VFAT

QNX 4: FAT12/16 | ISO 9660 | Qnx4 FS

SCO UnixWare: BFS| DTFS| ISO 9660 | System V| VxFS

1.6 Introduction to contiguous allocation filesystems

Some contiguous filesystems: BFS, ISO9660 and extensions.

1.7 Introduction to linked-list allocation filesystems

1.8 Introduction to FAT-based filesystems

(todo)

Some FAT filesystems: FAT12/16/32, VFAT and NetWare filestem.

1.9 Introduction to Inode filesystems

(todo)

1.10 Introduction to extent filesystems

(todo)

Some 'extent' filesystems: EFS and VxFS.

1.11 Introduction to filesystems using balanced trees

(todo)

Some filesystems which use B+ trees: HFS, NSS, Reiser FS and Spiralog filesystem.

1.12 Introduction to logging/journaling filesystems

File systems update their structural information (called metadata) by synchronous writes. Each metadata update may require many separate writes, and if the system crashes during the write sequence, metadata may be in inconsistent state.

At the next boot the filesystem check utility (called fsck) must walk through the metadata structures, examining and repairing them. This operation takes a very very long time on large filesystems. And the disk may not contain sufficient information to correct the structure. This results in misplaced or removed files.

A journaling file system uses a separate area called a log or journal. Before metadata changes are actually performed, they are logged to this separate area. The operation is then performed. If the system crashes during the operation, there is enough information in the log to "replay" the log record and complete the operation.

This approach does not require a full scan of the file system, yielding very quick filesystem check time on large file systems, generally a few seconds for a multiple-gigabyte file system. In addition, because all information for the pending operation is saved, no removals or lost-and-found moves are required. Disadvantage of journaling filesystems is that they are slower than other filesystems.

Some journaling filesystems: BeFS, HTFS, JFS, NSS, Spiralog filesystem, VxFS and XFS.