Ubuntu 11.04 - File Systems (5/6)


General Information
This is the result of personal research through community and official documentation about GNU/Linux and Ubuntu. References used are provided also. It will be presented in 6 posts as listed below:


General overview of the file system (FS) implementation in Linux
Linux uses the UNIX's standard file-system model, where everything appear in the form of files for the user. However a file does not have to be stored as object on a disk, it can be a stream of data. (Operating System Concepts, p.828) The kernel handles all these types of files by hiding the implementation details of any type behind a layer of software, the virtual file system (VFS). (Operating System Concepts, p.828)
The VFS is designed with the object-oriented principles. It has two components. One the specifies what file-system objects are allowed to look like and a layer of software to manipulate the objects. (Operating System Concepts, p.829) . The VFS works with objects each one contains a pointer to a function table. (Operating System Concepts, p.469)
  • The main VFS objects are:
  • the inode object and the file object
    Both are mechanisms to access files. Inode objects contain pointers to the disk blocks and the file object contains pointers to the actual file data.
  • the superblock object
    Its main responsibility is to provide access to inodes. It is a connected set of files that form a self-contained file-system. The VFS keeps in pairs the file-system/inode numbers to identify each node and it used the superblock object to get the inode with that pair.
  • the dentry object
    It represents the directory entries found in a path name.
    (Operating System Concepts, p.469 / p.830)
File systems in Linux are either loaded during boot time, or manually.
File systems and UBUNTU 11.04
Ubuntu 11.04 support the Linux Logical Volume Manager, which allows us to increase the size of a partition online, while the system is running.
In addition, Ubuntu 11.04 allows us to use different devices as mount points for several “heavy/crucial” directories. These are:
  • /boot, the directory containing the basic instructions to load the OS
  • / (root) directory
  • /home, the user(s) home directory, equivalent to “My documents” folder in MS Windows
A full list of the supported file systems in UBUNTU 11.04 can be found using the “cat” command in the virtual file system /proc as shown below.
cat /proc/filesystems
Table 1. Supported file systems in UBUNTU 11.04 according to Ubuntu natty manpages.
minix
is the file system used in the Minix operating system, the first to run under Linux. It has a number of shortcomings: a 64MB partition size limit, short filenames, a single timestamp, etc. It remains useful for floppies and RAM
disks.
ext
an elaborate extension of the minix file system. It has been completely superseded by the second version of the extended file system (ext2) and has been removed from the kernel (in 2.1.21).
ext2
the high performance disk file system used by Linux for fixed disks as well as removable media. The second extended file system was designed as an extension of the extended file system (ext). ext2 offers the best performance (in terms of speed and CPU usage) of the file systems supported under Linux.
ext3
a journaling version of the ext2 file system. It is easy to switch back and forth between ext2 and ext3.
ext4
a set of upgrades to ext3 including substantial performance and reliability enhancements, plus large increases in volume, file, and directory size limits.
Reiserfs
a journaling file system, designed by Hans Reiser, that was integrated into Linux in kernel 2.4.1.
XFS
a journaling file system, developed by SGI, that was integrated into Linux in kernel 2.4.20.
JFS
a journaling file system, developed by IBM, that was integrated into Linux in kernel 2.4.24.
msdos
the file system used by DOS, Windows, and some OS/2 computers. msdos filenames can be no longer than 8 characters, followed by an optional period and 3 character extension.
umsdos
an extended DOS file system used by Linux. It adds capability for long filenames, UID/GID, POSIX permissions, and special files (devices, named pipes, etc.) under the DOS file system, without sacrificing compatibility with DOS.
vfat
an extended DOS file system used by Microsoft Windows95 and Windows NT. VFAT adds the capability to use long filenames under the MSDOS file system.
ntfs
replaces Microsoft Window's FAT file systems (VFAT, FAT32). It has reliability, performance, and space-utilization enhancements plus features like ACLs, journaling, encryption, and so on.
proc
a pseudo file system which is used as an interface to kernel data structures rather than reading and interpreting /dev/kmem. In particular, its files do not take disk space.
iso9660
a CD-ROM file system type conforming to the ISO 9660 standard.
High Sierra
Linux supports High Sierra, the precursor to the ISO 9660 standard for CD-ROM file systems. It is automatically recognized within the iso9660 file-system support under Linux.
Rock Ridge
Linux also supports the System Use Sharing Protocol records specified by the Rock Ridge Interchange Protocol. They are used to further describe the files in the iso9660 file system to a Unix host, and provide information such as long filenames, UID/GID, POSIX permissions, and devices. It is automatically recognized within the iso9660 file-system support under Linux.
hpfs
the High Performance Filesystem, used in OS/2. This file system is read-only under Linux due to the lack of available documentation.
sysv
an implementation of the SystemV/Coherent file system for Linux. It implements all of Xenix FS, SystemV/386 FS, and Coherent FS.
nfs
the network file system used to access disks located on remote computers.
smb
a network file system that supports the SMB protocol, used by Windows for Workgroups, Windows NT, and Lan Manager.
ncpfs
a network file system that supports the NCP protocol, used
by Novell NetWare.
However, support for additional file systems can be added by adding additional modules to kernel.
As it can be seen above, UBUNTU 11.04 supports several file systems. The result when using the cat /proc/filesystems – command, will include several virtual file systems connected to no devices, that are used from Ubuntu kernel for administrative actions, such as memory caching. e.g.
ramfs, a very simple filesystem that exports Linux's disk caching mechanisms (the page cache and dentry cache) as a dynamically resizable RAM-based filesystem.
rootfs, a special instance of ramfs (or tmpfs, if that's enabled), which is
always present in 2.6 systems. You can't unmount rootfs for approximately the
same reason you can't kill the init process; rather than having special code
to check for and handle an empty list, it's smaller and simpler for the kernel
to just make sure certain lists can't become empty.”
Adding to those above, the swap file system must be noted as well, even though it is a raw file system, used for memory caching. Swap is a very important parameter in Linux.
The ones available to use in a physical drive during a clean installation, using the default partitioning tool are:
A very brief comparison of the File systems most commonly found in UBUNTU.
File System
Max File Size
Max Partition Size
Journaling
Notes
Fat16
2 GB
2 GB
No
Legacy
Fat32
4 GB
8 TB
No
Legacy
NTFS
2 TB
256 TB
Yes
For Windows Compatibility) NTFS-3g is installed by default in Ubuntu, allowing Read/Write support)
ext2
2 TB
32 TB
No
Legacy
ext3
2 TB
31 TB
Yes
Standard linux filesystem for many years. Best choice for super-standard installation.
ext4
16 TB
1EB
Yes
Modern iteration of ext3. Best choice for new installations where super-standard isn't necessary.
reiserfs
8 TB
16 TB
Yes
No longer well-maintained.
JFS
4 PB
32 PB
Yes (metadata)
Created by IBM - Not well maintained.
XFS
8 EB
8 EB
Yes (metadata)
Created by SGI. Best choice for a mix of stability and advanced journaling.
GB = Gigabyte (1024 MB) :: TB = Terabyte (1024 GB) :: PB = Petabyte (1024 TB) :: EB = Exabyte (1024 PB)

(https://help.ubuntu.com/community/LinuxFilesystemsExplained)
However, as mentioned before, not all are available during installation to be used in a physical drive, since not all support the read/write and file permissions required for Ubuntu to work properly. NTFS cannot be used to install Linux at all, and FAT16/32 are not recommended to install Linux, since FAT does not have any of the permissions of a true Unix FS. They are used for compatibility reasons with Microsoft Windows.
From the file systems described above, the most commonly used for Ubuntu installation, are:
  • ext2/ext3
  • ext4
    and
  • XFS
Ext2 as shown in table 1, is not a journaling FS. It is still commonly used however, to mount in boot time the /boot directory. Since it is not a journaling system, there are less read/write operations and since the /boot directory is only used in boot time to load the kernel, ext2 can result in faster OS boot.
Ext3 is the successor of ext2. However, it is a journaling system. It has been the default file system for a lot of years and since the LTS versions starting from 6.04 are only now starting to expire, many still use this file system for their / (root) and /home directories. Ubuntu supports the conversion from ext3 to ext4, the new default journaling default file system. The possible limit for a sub directory is 32000. It uses an indirect block mapping scheme to keep track of each block used for the blocks corresponding to the data of a file.
(http://kernelnewbies.org/Ext4)
Ext4 is the evolution of ext3 and it is a journaling file system, but it is an attribute that can be disabled. Ext4 breaks sub directory limit and allows an unlimited number of sub directories. Moreover, it uses “extents” to improve performance for large files. “For example, a 100 MB file can be allocated into a single extent of that size, instead of needing to create the indirect mapping for 25600 blocks (4 KB per block). Huge files are split in several extents. Extents improve the performance and also help to reduce the fragmentation, since an extent encourages continuous layouts on the disk. When Ext3 needs to write new data to the disk, there's a block allocator that decides which free blocks will be used to write the data. But the Ext3 block allocator only allocates one block (4KB) at a time. That means that if the system needs to write the 100 MB data mentioned in the previous point, it will need to call the block allocator 25600 times (and it was just 100 MB!). Not only this is inefficient, it doesn't allow the block allocator to optimize the allocation policy because it doesn't knows how many total data is being allocated, it only knows about a single block. Ext4 uses a "multiblock allocator" (mballoc) which allocates many blocks in a single call, instead of a single block per call, avoiding a lot of overhead. This improves the performance, and it's particularly useful with delayed allocation and extents.” (http://kernelnewbies.org/Ext4)
In addition, delayed allocation procedure is used in ext4 improving performance and reducing fragmentation and it is supported for online de fragmentation.
XFS is a journaled 64-bit file system. It is organized in balanced B+Trees and uses Extends to store data. XFS is fully supported by all Ubuntu-Versions. It is not recommended to be used to mount the /boot directory, due to GRUB support. It I sused to mount the root directory and the /home one.
Advantages
  • Part of official Kernel
  • Online optimizing (defragmenting) filesystem using "xfs_fsr" (part of xfsdump-package)
  • Ability to grow an existing filesystem using "xfs_growfs" (part of xfsprogs-package)
  • Own dump- and restore-utilities to keep fs-specific info
  • Fast seeking through directories
  • Efficient file storage, low fragmentation
  • Relatively low memory usage
  • Journal (internal or external)
  • Balance between speed and safety
  • Delayed allocation and improved sparse file handling
Disadvantages
  • Slower than non-journaled fs (like ext2)
  • Due to balance of speed and safety, there is the possibility to lose data on aprupt power loss (outdated journal metadata), but not filesystem consistence
  • Currently there is no ability to shrink an xfs filesystem. However, there are some people working on this issue
  • GRUB-Support is in an early stage
  • The journal is currently not 32-bit / 64-bit portable. Before mounting a xfs prevously used on 32-bit linux in 64-bit linux (or different arch) the journal has to be emptied using xfs_repair.

Tools for the ext2,ext3,ext4 and XFS file systems in Ubuntu.
The available tools for file system information and modifications, are separated ion this paper in two parts. The one where the only the graphical tools are explained, and the second one where the commands in terminal are briefly explained.
GUI tools
A fast overview of the available file systems can be found in the system monitor, in the file systems tab.

Further actions and information for the available storage devices can be found in the Disk Utility in the Control Center.
Disk Utility provide us with an overview of the storage devices currently connected in the system. It also provides us with an easy graphical interface to:
  • get hardware information
  • mount devices
  • unmount devices
  • check the filesystem in the device
  • edit partitions
  • delete partitions
  • format volumes
Graphical tools for fie systems, as for any other administrative actions, are poorer comparing to the equivalents in terminal. They addressed to normal users and available actions are limited, e.g define block size, cannot be processed thought them.
Terminal Commands
They are addressed to advanced users and the can modify any available option for a file system. This is why they must be used as superuser, or by borrowing su privileges. Additional information, about parameters and syntax, are provided in the links below.
  • blkid, used to find the Universally Unique Identifier (UUID) for a device, which is prefered from Ubuntu over the deveice path, eg /dev/sda, since it can change during boot.

  • mkfs, used to create a file system
  • fsck, used to check and repair file system
  • fdisk
  • df, used to report file system usage

  • badblocks, used to check a device for bad blocks
  • umount and mount, used to unmount and mount devices

References

  1. Operating System Concepts, 8th Edition, Abraham Silberschatz, Peter B. Galvin, Greg Gange, 2010, Jonh Wiley and Sons
  2. http://www.raiden.net/node/576, as found on June 15th 2011
  3. Pro Ubuntu Server Administration, Sander van Vugt, 2009, Apress
  4. http://kernelnewbies.org/Ext4, as found on June 14th 2011
  5. https://wiki.ubuntu.com/XFS , as found on June 15th 2011
  6. http://oss.sgi.com/projects/xfs , as found on June 15th 2011
  7. https://help.ubuntu.com/community/Mount, as found on June 15th 2011


Comments

Popular posts from this blog

Ubuntu 11.04 - Process management (2/6)

Ubuntu 11.04 - Memory management (3/6)