We all use files or files every day, and we all use it too folders or directories, whether in the file systems of the desktop PC, on laptops, mobile devices, etc. Therefore, in this article I will explain everything you need to know about it, so that you can understand what information is and how it is stored in memory of.
What is a files system?
Un File System File System (FS) is a method and data structure that the operating system uses to control how information is stored and retrieved. Without a file system, data stored on a storage medium would be a single data entity with no way to distinguish where one data set ends and the next begins, or where any data set is located when it needs to be retrieved. This would be what is known as raw data.
When you format a storage drive using NTFS, FAT, ext4, btrfs, HFS+, etc., you're actually creating a file system to manage the data on that drive. In other words, what we call formats are actually FS.
By separating the data into chunks and naming each chunk, lThe data can be easily isolated and identifiedTo make all this possible, a file system consists of two or three layers. Sometimes these layers are explicitly separated, and other times, functions are combined. The layers we're talking about are:
- Logical file system: is responsible for interaction with the user application. It provides an API for file operations, such as Open, Close, Read, etc., and forwards the requested operation to the layer below it for processing. The logical file system manages open file table entries and file descriptors per process. This layer provides file access, directory operations, security, and protection.
- Virtual file system: This interface allows support for multiple concurrent instances of physical file systems. This layer is optional and is not always present.
- Physical file system: Handles the physical operation of the storage device. It processes the physical blocks being read or written. It handles buffering, memory management, and is responsible for the physical placement of blocks in specific locations on the storage medium. The physical file system interacts with storage hardware device drivers or the channel to direct the storage device.
FS Features
Each and every file system in existence has its limitations and characteristics that you should know, like:
- Space management or allocation: File systems allocate space granularly, usually across multiple physical drives on the device. File systems are responsible for organizing files and directories, and keeping track of which areas of the media belong to which files and which are unused.
- Fragmentation: This occurs when the same file is stored in different parts of the storage drive, meaning the individual file parts are not contiguous. As files are created, modified, and deleted, this can result in areas of used and unused space of varying sizes. Not only that, this affects some FS (e.g., Microsoft's FAT and NTFS), slowing down file reading.
- File names: File systems manage length restrictions, case sensitivity, and the use of special characters in file names.
- Directories: They often have directories to organize files into separate collections. They can be flat or hierarchical. I'll discuss this in more detail later, as this is what it's all about...
- Metadata: It has associated metadata information, which includes details such as data length, timestamps, access permissions, and file attributes.
- Utilities and access: They are used to initialize, modify, and delete file system instances, as well as to create, rename, and delete directories and files, encryption, quotas, migration, conversion, backup, etc.
- Integrity and fault management: The FS must maintain the integrity of file system structures on secondary storage or external drives, even in the event of operating system failures or power outages.
- User Data: must allow management of user data, including storage, retrieval and updating of data.
- Multiple file systems: It is possible to have multiple file systems on a single system.
- Design limitations: File systems have functional limitations that define the maximum storage capacity within that system, the maximum size of files, number of files, etc.
What is a partition?
An partition A partition is a logical section or subdivision of a data storage device, such as an HDD, SSD, flash drive, etc. This partition serves several purposes, allowing for better organization and management of stored data. For example, each partition could be used for a specific purpose, multiple operating systems could be installed on it, etc.
It is important to note that the creation, deletion, and resizing Partition changes can lead to data loss, so it is important to perform these operations with caution and back up critical data before making major changes to the partition structure.
On the other hand, in summary, you should also understand that there are various types of partitions fundamental in the MBR system (the possibilities are expanded in the GPT, with up to 128 possible partitions on the same unit, although this is another topic…):
- Primary Partition: Each storage drive can have up to four primary partitions. These are the main partitions on which operating systems are installed and where data resides. One of the primary partitions can be designated as an active or boot partition, from which the operating system boots.
- Extended Partition: An extended partition is a special primary partition used to create additional logical partitions within it. It doesn't directly contain data, but acts as a container for logical partitions. You can have multiple logical partitions within an extended partition. This allows you to overcome the limitation of four primary partitions on a hard drive.
- Logical Partition: These are created within an extended partition. They are not used to boot the operating system and are generally used to organize data or files. You can have multiple logical partitions within an extended partition, which makes data organization easier.
What is partition table?
La partition table It is a data structure located at the beginning of a hard drive and contains information about how the drive is divided into partitions. It is typically stored in the master boot sector (MBR) on BIOS-based systems or in GPT (GUID Partition Table) on more modern UEFI-based systems. The partition table includes details such as the size of each partition, its type (e.g., NTFS, FAT32, ext4, etc.), and its location on the storage medium.
Therefore, the relationship between the partition table and the file system lies in the fact that the partition table indicates where each partition starts and ends on the physical disk. Each partition is formatted with a specific file system. When a partition is accessed to read or write files, the operating system uses the information in the partition table to locate and understand how the file system is structured on that particular partition.
File system types
Among the different file systems, the following types should be highlighted:
- Disk File Systems: These systems take advantage of the ability of disk storage media to access data randomly and quickly. They allow multiple users or processes to access data on the disk regardless of its sequential location. Examples include FAT, exFAT, NTFS, Reiser FS, HFS, HFS+, HPFS, APFS, UFS, ext2, ext3, ext4, XFS, btrfs, VMFS, ZFS, ReiserFS, NSS, and ScoutFS. In addition to these, the following could also be considered:
- Optical Discs: common formats such as ISO 9660 and Universal Disk Format (UDF) used on CD, DVD and Blu-ray.
- Flash File Systems: These systems are designed specifically for flash memory devices and take into account their characteristics and restrictions. It is recommended to use file systems designed for flash devices instead of adapting disk systems. Some examples are JFFS, JFFS2, YAFFS, UBIFS, LogFS, and F2FS.
- Magnetic Tape File Systems: These systems manage storage on tapes, which have longer random access times than disks. They differ in directory management and emphasize the need to avoid frequent linear movements on tapes. An example would be IBM's LTFS.
- Database File Systems: Database-based file systems, where files are identified by characteristics such as type, author, or metadata. Examples include IBM DB2, among others.
- Transactional File Systems: These systems guarantee the atomicity and isolation of operations on a file. Examples include NTFS in Microsoft Windows and other prototype transactional file systems for UNIX/Linux, such as LFS, ext3, etc.
- Network File Systems: Network file systems that allow access to files on remote servers via protocols such as NFS, AFS, SMB, FTP, and WebDAV. That is, they are designed for distributed computing.
- Shared File Systems: These systems allow multiple servers to securely access the same disk subsystem. Examples include GFS2, GPFS, SFS, CXFS, StorNext, and ScoutFS.
- Special File Systems: They are somewhat specific, and don't have file elements as such, but they can be accessed via APIs. For example, we define Device File System as devfs, used in Linux, etc. On the other hand, we have other special ones like configfs, sysfs, and procfs, also known in the Linux world.
- Minimal File System / Audio-Cassette Storage: Audio cassettes were used as data storage systems to power some microcomputer models of the time, such as the Commodore PET.
- Flat File Systems (Flat File System): These systems have no subdirectories and store all directory entries in a single main directory. Examples of these FS are the one used in the CP/M system and the Macintosh File System for Apple's classic Macs.
FS Technologies
Each file system has its own particularities, and they support different technologies. Some of the most important are:
- Self-Healing: Refers to a file system's ability to automatically detect and correct errors and problems that may arise in the storage system. These errors can include bad sectors on a hard drive or other types of data corruption. When an error is detected, the file system can attempt to recover the affected data by restoring from backups or by repairing corrupted data. This helps maintain the integrity of the data stored in the file system.
- Compression: It is a feature that allows you to reduce the size of files and data stored in the file system. Compressing data can save disk space and speed up file transfers. When compression is enabled, the file system automatically compresses files written to it and decompresses files when they are read. This can be useful on resource-constrained storage systems, although it may slow down access.
- Encryption: It is a technique used to protect the confidentiality of data stored in the file system. Data is encrypted before being written to disk and decrypted when it is read. Encryption ensures that the data is unreadable to anyone without the appropriate decryption key. This is essential for protecting the privacy and security of sensitive data. Like compression, it can also slow down access.
- Journaling (Transaction Log): It is a feature that keeps a record of all operations performed on the file system, such as the creation, modification, or deletion of files. This record is called a "journal" or "transaction log." In the event of a system failure, such as an unexpected power outage, the file system can use the transaction log to recover to a consistent state. This prevents data corruption and ensures the integrity of the file system.
- Snapshot Capacity: Snapshots are point-in-time copies of the file system at a specific point in time. These copies are snapshots and are stored alongside the current data in the file system. Snapshots allow the file system to be restored to a previous state if errors or unwanted changes occur. This is useful for backup, data recovery, and testing changes before implementing them permanently on the system.
What is a directory or folder?
Now that we know how data is stored in storage units and the necessary data structures, it is time to move on to explain what a folder or directory.
A directory or folder is a structure for cataloging other files within the file system, giving it a more intuitive hierarchy or organization for the user or the operating system. To do this, it contains references to other files and possibly other directories or folders, in this case called subdirectories or subfolders.
Some operating systems with hierarchical file systems, such as Unix, typically have a directory cache with recent paths stored in a portion of RAM. In Unix, this portion is known as DNLC (Directory Name Lookup Cache), while in Linux, it is called dcache. This portion of memory is updated with the most recently accessed paths, while in network file systems, a mechanism is required to ensure consistency by managing entries that have been invalidated or created by users.
In a hierarchical file system Storage is tree-like. The terms "parent" and "child" are often used to describe the relationship between a subdirectory and the directory in which it is cataloged, the latter being the parent. The highest-ranking directory in such a file system, which doesn't have its own parent, is called the root directory. This can be best seen in Unix or Linux, where you have a particular hierarchy and the root directory is root or /, from which all other directories hang, even if they're not on the same partition.
A virtual directory is a type of file organization that doesn't rely on location in a hierarchical directory tree. Instead, it gathers results from a data source, such as a database or a custom index, and presents them visually in the same format as folder views.
These folders or directories can be managed using tools, both GUI and CLI. For example, we can use commands or the file manager itself to create, delete, rename, move, copy, etc. Although some operating systems may have certain restrictions on the permissions and actions you can perform on certain folders...
Details about a folder or directory on different operating systems
En cada file system and operating system, files and folders are treated differently:
- Root: This is the point from which the rest of the system's folder hierarchy hangs or where it resides. In Windows, this is usually C:\, while in Unix/Linux, it's the root partition or /.
- Path or route: This is the path where a folder or file is located within an FS. For example, in Windows we might have C:\Program Files\Office\Word.exe. In Unix/Linux, it would be something like /home/user/example.c. As we can see, in Windows, the backslash is used, while in the *nix world, the traditional slash is used.
- Filename or file name: This is the name given to each file within a specific FS. Each file system will have its own limitations regarding the maximum character limit for a filename. Furthermore, a distinction must be made between the base name and the extension. Some early systems also had limitations on the length of extensions, generally three. This is why you'll sometimes see .htm instead of .html, to ensure compatibility with DOS systems. An example of a filename might be example.txt, where "example" is the name and "txt" is the extension, indicating in this case that it is text.
The particular case of Unix/Linux
What is the superblock?
Un superblock It is a fundamental data structure found at the beginning of a file system. The super block contains critical information about the file system itself and is used to manage and access data stored on that file system. Every Unix file system has its own super block, and its specific format can vary depending on the file system used (e.g., ext4, XFS, UFS, etc.).
La information that is stored in the superblock may include:
- File system size: indicates the total size of the file system.
- Number of inodes: Specifies the number of inodes available on the file system. Inodes are data structures that represent files and directories.
- Free blocks: Records the number of free data blocks remaining in the file system for storing information.
- Mounting point: indicates the directory in which the file system is mounted on the operating system.
- File system identification: a unique identifier for the file system that distinguishes it from other file systems on the same system.
- Date and time of assembly: records when the file system was last mounted.
- Inode and block counters used: keeps track of how many inodes and data blocks are being used on the file system.
- Checksum and other integrity metadata: Some modern superblocks may include additional information to verify the integrity of data stored in the file system.
When a file system, i.e., a formatted partition, is mounted, the operating system accesses the superblock to obtain vital information about the file system's structure and state. This information is essential for reading and writing data to the file system, as well as for ensuring the integrity and consistency of the stored data.
Inodes and directories
En Unix/Linux, Everything is a file, such as partitions, device drives, etc. This isn't the case in other systems, such as Windows. However, when everything is a file, even folders or directories are considered special file types.
Therefore, a directory is a special file that contains file (and subdirectory) names and inode numbers for files on the same file system or names of symbolic links to files or directories on the same or different file systems. In the case of hard links, the inode keeps track of how many directory entries are listed in it and deletes file blocks when the link count reaches zero. This is different in the case of a soft link or symbolic link. Deleting a symbolic link does not affect the target file. However, if a file with symbolic links to it is deleted, the links become unusable.
In Unix/Linux file systems, a file can have several names, with a directory entry for each name in the same or different directories, all pointing to the same inode structure that maintains the list of disk blocks where the data is stored.
To better understand all of this, it's important to remember what an inode (index node) is. It's a fundamental data structure in *nix file systems. Represents and stores essential information on a file or directory within the file system. Therefore, both a folder and a file will be seen as an inode by the system.
For this to be possible, each inode points to the next insights:
- Inode number: a unique identifier for each file or directory.
- Type of file: Indicates whether the inode refers to a regular file (f), a directory (d), a symbolic link (l), or another type of file.
- Tamaño the archive: the current size of the file in bytes.
- Permissions and owner: Information about who has permission to access (read, write, execute) and modify the file, as well as the file's owner (user or root). The owner could even be a user who doesn't exist as such, but could be a device, etc.
- Creation/modification date and time: Records when the inode was first created and when it was last modified. Other metadata or extended permissions may also be included.
- Number of links: Indicates how many file or directory names are associated with this inode. Directories have at least two links: one to themselves and one to their parent directory.
- Pointers to data blocks: Contains references to data blocks on the storage unit that store the actual contents of the file. These blocks can be direct, indirect, or doubly indirect addresses, depending on the specific file system implementation and the size of the file.
Inodes are crucial to the functioning of the file system, as they allow the operating system to keep track of the physical location and information associated with each file and directory. When a file or directory is accessed, the operating system consults the corresponding inode number to find the location of the data and other information related to that file.