Memory Management & Virtual memory

tl;dr

You will delve into memory management techniques, including contiguous memory allocation, paging, segmentation, and page table structures. They will explore swapping mechanisms and how operating systems efficiently allocate memory. The unit also covers virtual memory concepts such as demand paging, Copy-on-Write, page replacement strategies, frame allocation, and thrashing, equipping students with a deep understanding of how modern systems optimize memory usage for performance and efficiency.

Table of Contents

A file is a logical storage unit that stores data, programs, or system information. The Operating System (OS) manages files by organizing, protecting, and providing access methods to ensure efficient storage and retrieval of information.

This blog will cover:

  • File Concept
  • Access Methods
  • Directory and Disk Structure
  • File Protection

What is a File?

A file is a collection of related information stored on a storage device. Files can contain:

  • Text files (documents, code).
  • Binary files (executables, images, videos).
  • System files (OS-related data).

File Attributes

Each file has attributes that provide metadata about the file.

AttributeDescription
NameHuman-readable name (e.g., report.docx).
TypeFile format (e.g., .txt, .mp4).
SizeFile length in bytes.
LocationAddress of file on disk.
PermissionsAccess rights (read, write, execute).
TimestampsCreation, modification, and last access times.

Example: A file named data.txt has a size of 2 MB, is located in /home/user/, and is readable by all users but writable only by the owner.

File Operations

An OS provides various file operations:

OperationDescription
CreateCreate a new file
OpenLoad a file for reading/writing.
ReadRetrieve data from a file.
WriteModify or append data.
CloseRelease file resources.
DeleteRemove the file from storage.
RenameChange the file’s name.

Example: A media player opens a video file, reads data, and displays it on the screen.

Access methods determine how data is retrieved from files.

Sequential Access

  • Reads or writes data in order, from start to end.
  • Used in text files, logs, and audio/video streams.

Example: Reading a novel page by page.

Direct (Random) Access

  • Allows access to any part of a file instantly.
  • Used in databases, executable files, and large datasets.

Example: Jumping to page 50 of a PDF directly.

Indexed Access

  • Uses an index table to locate file blocks.
  • Faster than sequential access for large files.

Example: A search engine database uses indexed access to retrieve data quickly.

Comparison of Access Methods

Access TypeSpeedUse CaseExample
SequentialSlowLogs, Media filesReading a movie file
DirectFastDatabases, Code executionJumping to a specific row in a database
IndexedFastestLarge datasetsSearching in a digital library

Directory Structure

A directory is a file system component that organizes files in a hierarchical manner.

Types of Directory Structures

StructureDescriptionExample
Single-LevelAll files stored in one directoryEarly operating systems (MS-DOS).
Two-LevelSeparate user directories under a master directory.UNIX home directories (/home/user).
Tree-StructuredHierarchical, allows subdirectories.Windows file system (C:\Users\Admin\Documents).
Acyclic GraphAllows file sharing across directories.Symbolic links in UNIX.
General GraphAllows multiple links to files but may create loops.Used in some distributed file systems.

Example:

/home/

  â”œâ”€â”€ user1/

  â”‚   ├── documents/

  â”‚   ├── pictures/

  â”œâ”€â”€ user2/

      â”œâ”€â”€ downloads/

      â”œâ”€â”€ projects/

Disk Structure

A disk is divided into blocks where files are stored. The OS manages files using:

  1. File Allocation Table (FAT) – Uses a table to map files to disk blocks.
  2. Inode-Based File System – Used in UNIX/Linux to store file metadata in inode tables.

Example: Windows uses NTFS, while Linux uses ext4 for managing disk structures.

File protection ensures that unauthorized users cannot modify or delete files.

Access Control Mechanisms

MethodDescriptionExample
User Groups & PermissionsFiles have Owner, Group, Others permissions.Linux (chmod 755 file.txt).
Access Control Lists (ACLs)Specifies detailed permissions for users.Windows NTFS file security settings.
EncryptionConverts file data into an unreadable format.Secure document storage.

Example:

In Linux, file permissions are set using:

chmod 644 myfile.txt  # Read & write for owner, read-only for others

Common File Security Threats

ThreatDescriptionSolution
Unauthorized AccessHackers or users accessing restricted files.Use strong passwords & ACLs.
Data CorruptionFiles get damaged due to software/hardware failures.Use backups & checksums.
Malware & RansomwareMalicious software modifies or encrypts files.Use antivirus & secure OS settings.

Example: Ransomware attacks encrypt user files and demand payment for access.

Unauthorized AccessSequential AccessDirect AccessIndexed Access
Use CaseLog files, videosDatabases, executablesSearch engines, large records
SpeedSlowFastVery Fast
ExampleReading a CSV fileJumping to a specific page in a bookSearching a contact in a phonebook
Directory StructureProsCons
Single-LevelSimple, fastNo organization
Tree-BasedOrganized, scalableMore complex
Graph-BasedSupports file sharingCan create loops
Protection MethodPurpose
Permissions (RWX)Restricts user access
EncryptionSecures sensitive files
Backup & RecoveryPrevents data loss

File management is a critical function of an OS that ensures efficient file storage, access, security, and organization. The OS provides various access methods, directory structures, and protection mechanisms to keep data safe and easily accessible.

Key Takeaways
  Files store programs, data, and system information.
  Access methods include sequential, direct, and indexed access.
  Directory structures help organize files efficiently.
  File protection mechanisms include permissions, ACLs, and encryption.

Mass storage refers to large-scale storage devices that hold data persistently, such as hard drives (HDDs), solid-state drives (SSDs), magnetic tapes, and optical disks. The Operating System (OS) manages these devices using scheduling algorithms and swap space management techniques to optimize performance and ensure efficient data access.

This blog covers:

  • Overview of Mass-Storage Structure
  • Disk Scheduling
  • Swap Space Management

What is Mass Storage?

Mass storage refers to non-volatile memory used to store large amounts of data for long-term use. Unlike RAM, which is temporary, mass storage devices retain data even after power is turned off.

Types of Mass-Storage Devices

TypeDescriptionExample
Hard Disk Drive (HDD)Uses spinning disks and magnetic heads to read/write data.Laptop, desktops.
Solid-State Drive (SSD)Uses flash memory for high-speed storageGaming PCs, data centers.
Magnetic TapeUsed for long-term backups and archivesEnterprise storage systems.
Optical DisksCD/DVDs used for media storage.Movies, games.
Cloud StorageRemote storage over the internet.Google Drive, Dropbox.

Example: Your laptop’s SSD stores the OS, applications, and user files, while cloud storage keeps backups online.

Structure of a Hard Disk

A hard disk consists of:

  • Platters – Circular disks where data is stored magnetically.
  • Tracks – Concentric circles on the platters.
  • Sectors – Smallest storage unit (usually 512 bytes).
  • Read/Write Head – Reads and writes data onto the disk.

Example: When you open a file, the OS locates it using the track and sector number and retrieves the data.

SSD vs. HDD

FeatureHDDSSD
SpeedSlower (100 MB/s)Faster (500+ MB/s)
DurabilityMechanical parts can failNo moving parts (more reliable)
CostCheaper per GBMore expensive per GB
Power ConsumptionHigherLower

Example: SSDs are preferred for gaming due to faster load times, while HDDs are used for bulk storage.

What is Disk Scheduling?

  • Disk scheduling optimizes how read/write requests are handled to minimize seek time and improve performance.
  • When multiple I/O requests arrive, the OS decides which request to process first.

Disk Scheduling Algorithms

AlgorithmDescriptionProsCons
First-Come First-Serve (FCFS)Processes requests in order of arrival.Simple, fair.Slow for random access.
Shortest Seek Time First (SSTF)Serves the request closest to the read/write head.Reduces seek time.Can cause starvation.
SCAN (Elevator Algorithm)Moves head from one end to the other, serving requests along the way.Avoids starvationLong wait times for some requests.
C-SCAN (Circular SCAN)Like SCAN but resets at the end instead of reversing direction.More uniform waiting time.Some requests wait longer.
LOOK & C-LOOKOptimized SCAN/C-SCAN by stopping at the last request before reversing/resetting.Faster than SCAN.Still causes delay for far requests.

Example: If requests arrive at Tracks 10, 55, 15, and 90,

  • FCFS serves them in order: 10 → 55 → 15 → 90 (long seek times).
  • SSTF serves closest first: 10 → 15 → 55 → 90 (shorter seek times).

Performance Comparison

AlgorithmAverage Seek TimeBest for
FCFSHighFairness (no starvation).
SSTFLowFast response time.
SCANModerateHeavy workloads.
C-SCANLowLarge disk queues.

Real-World Use:

  • FCFS is used in small systems with low disk usage.
  • SSTF & SCAN are used in high-performance databases.
  • C-SCAN is preferred for large-scale servers.

What is Swap Space?

  • Swap space is virtual memory that acts as an extension of RAM, stored on disk.
  • When RAM is full, inactive pages are moved to swap space, allowing more processes to run.

Example: If a system has 8 GB RAM and needs 10 GB, the extra 2 GB is temporarily stored in swap space.

Swap vs. Virtual Memory

FeatureSwap SpaceVirtual Memory
DefinitionDedicated disk space for memory overflow.Logical memory larger than physical memory.
UsageExtends RAM capacity.Uses paging/swapping to optimize memory.
PerformanceSlower than RAM, faster than diskManages memory dynamically

Example: Windows uses “pagefile.sys” for swapping, while Linux uses a swap partition or swap file.

How Swap Space Works?

  • OS detects high memory usage.
  • Moves inactive pages from RAM to swap space.
  • Loads pages back into RAM when needed.

Swap Space Allocation

Swap space can be allocated in two ways:

MethodDescriptionExample
Fixed Swap SpacePredefined partition for swapping.Linux swap partition
Dynamic Swap FileExpands swap space as needed.Windows pagefile.sys.

Example: Windows automatically manages swap space, while Linux users manually set swap partitions.

Thrashing and Swap Space

  • Thrashing occurs when excessive swapping slows down the system.
  • If a system spends more time swapping pages than executing processes, performance drops significantly.

Solution:

  • Increase RAM.
  • Optimize swap space usage.
  • Use better memory management techniques (working set model, paging optimization).
FeatureDisk SchedulingSwap Space Management
PurposeOptimizes disk read/write requests.Extends memory using disk storage.
ImprovesData access speed.System multitasking.
ExampleSCAN algorithm for hard drivesLinux swap partition.

Mass storage is essential for data retention, memory management, and system performance. The OS optimizes storage using disk scheduling algorithms and swap space management techniques to prevent slowdowns and ensure smooth operations.

Key Takeaways
  HDDs are slower but cheaper, SSDs are faster but expensive.
  Disk scheduling (SSTF, SCAN, C-SCAN) optimizes disk performance.
  Swap space extends RAM and prevents memory overflow.
  Too much swapping causes thrashing and slows down performance.

System protection is a crucial feature of an Operating System (OS) that ensures resources like memory, files, and CPU time are accessed only by authorized users and processes. It prevents accidental damage, malicious attacks, and security breaches, ensuring system stability and data integrity.

This blog covers:

  • Goals of Protection
  • Principles of Protection
  • Domain of Protection
  • Access Matrix

The primary goal of system protection is to prevent unauthorized access and misuse of system resources.

Key Objectives

GoalDescriptionExample
ConfidentialityPrevents unauthorized access to sensitive information.Encryption of user files.
IntegrityEnsures data is accurate and not modified by unauthorized users.Digital signatures in documents.
AvailabilityEnsures system resources are available when needed.Preventing Denial-of-Service (DoS) attacks.
AuthenticationVerifies the identity of users and processes.Passwords, biometrics.
Access ControlRestricts user actions based on permissions.Read-only access to sensitive files.

Example: Banking applications use authentication and encryption to protect financial data from unauthorized access.

System protection follows key design principles to enforce security and access control.

The Least Privilege Principle

  • Users and processes should only have the minimum necessary permissions.
  • Reduces security risks by limiting access to critical files.

Example: A normal user should not have access to system files like /etc/passwd in Linux.

Fail-Safe Defaults

  • Default access should be restricted, only granting access when explicitly allowed.
  • Prevents accidental exposure of sensitive data.

Example: New user accounts should not have administrator privileges by default.

Economy of Mechanism

  • Security mechanisms should be simple and easy to manage.
  • Complex systems are harder to secure and maintain.

Example: Using a single authentication system for all users instead of multiple fragmented systems.

Open Design

  • Security should not rely on secrecy of the design, but on well-tested algorithms and mechanisms.
  • Avoids security through obscurity, which is unreliable.

Example: Open-source encryption algorithms (AES, RSA) are preferred over proprietary, closed-source methods.

Complete Mediation

  • Every access request should be validated against security policies.
  • Prevents unauthorized access by bypassing checks.

Example: File permissions should be checked each time a file is accessed, not just when it is opened.

Separation of Privilege

  • Requires multiple conditions to grant access, reducing security risks.
  • Multi-factor authentication (MFA) enforces this principle.

Example: A banking app requires both a password and an OTP for transactions.

Least Common Mechanism

  • Resources should be shared minimally to avoid security risks.
  • Reduces the impact of security vulnerabilities.

Example: Each user should have a separate login session, rather than sharing a common session.

Psychological Acceptability

  • Security mechanisms should be easy for users to understand and use.
  • Complex security measures often lead to user errors.

Example: Auto-locking computers after inactivity instead of requiring users to manually log out.

What is a Protection Domain?

A protection domain is a set of permissions that determine what resources a user or process can access.

Types of Protection Domains

TypeDescriptionExample
User-Level ProtectionAccess control based on user identity.Read/Write/Execute permissions.
Process-Level ProtectionDefines what processes can do in the system.Process isolation in memory.
Role-Based Access Control (RBAC)Assigns permissions based on user roles.Admins can modify system settings, users cannot.

🔹 Example: A web server should only access web-related files, not sensitive system files.

Switching Between Domains

Processes may need to switch between domains when performing system tasks.

Example:

  • A user process requests administrator privileges using sudo in Linux.
  • The OS verifies the request and temporarily grants higher privileges.

What is an Access Matrix?

An Access Matrix is a table that defines permissions for different users (subjects) on system resources (objects).

User/ProcessFile A (R)File B (W)Printer (Execute)
User1ReadNo AccessExecute
User2No AccessWriteNo Access

How Access Matrix Works

  • Rows represent users or processes (subjects).
  • Columns represent files, devices, or resources (objects).
  • Cells define access rights (read, write, execute, no access).

Implementation of Access Matrix

MethodDescriptionExample
Access Control Lists (ACLs)Stores access rights with each object.Windows NTFS file permissions.
Capability ListsEach user/process stores its allowed actions.Linux security tokens.

Example: In Linux, file permissions (chmod) implement an Access Matrix:

chmod 755 myfile.txt  # Owner: RWX, Group: R-X, Others: R-X

Advantages & Disadvantages of Access Matrix

 Advantages

  • Flexible and secure access control.
  • Allows fine-grained permissions for different users.

 Disadvantages

  • Can be complex to manage in large systems.
  • Large matrices consume memory, requiring optimizations.
FeatureAccess MatrixACLsCapability Lists
FlexibilityHighMediumMedium
EfficiencyLow (Large size)MediumHigh
SecurityHighHighMedium
ExampleUNIX file permissionsWindows NTFSLinux security tokens

System protection is essential for securing resources, preventing unauthorized access, and ensuring data integrity. The OS enforces protection using principles like least privilege, access control, and authentication mechanisms.

Key Takeaways
  Protection prevents unauthorized access and system misuse.
  Least privilege and complete mediation enhance security.
  Protection domains define access rights for users and processes.
  Access Matrix, ACLs, and Capability Lists implement security policies.

more from