QOS is stripe-group centric; all configuration and operations act on a specific stripe group. This allows stripe groups with differing access characteristics to use different real-time settings. When a client requests a certain amount of real-time I/O, the FSM informs all connected clients of the amount remaining and available for non-real-time I/O accesses.
This information is encapsulated in a token. Think of the token as a capability that enables the client to perform I/O without contacting the FSM. When the amount of real-time I/O on a stripe group changes, the FSM informs all connected clients about the change via a callback.
In both the client and the server, QOS is implemented as a state machine. Each state has a set of events and allowable actions. The term state thusly refers to the internal state machine.
Active versus Passive
QOS is a passive, not active implementation of real-time I/O. In an active implementation, guaranteed rate I/O known as GRIO), the scheduler is tightly coupled with the I/O subsystem. The qualities of the disk subsystem are well known so the scheduler can guarantee that a process will be scheduled such that it will receive the required amount of bandwidth. Since SNFS is a cross-platform file system that does not have hooks in the operating system scheduler, it cannot provide such a guarantee.
In a passive implementation, a real-time process gates its I/O according to some outside metric (such as a frame rate for specific video formats). The file system then gates all other non-real-time I/O so they do not interfere.
It is very important that the differences between Active and Passive OOS operation are understood by the user. It is a misconception to think that QOS, despite its name, guarantees a specific amount of real-time I/O to a process.
Supported Platforms
QOS has been tested on Windows XP, Windows 2008, Linux, and Solaris. In Windows, an application gets a handle to a file to perform I/O, usually via the Win32 CreateFile() API. In UNIX, an application receives a file descriptor (fd) via the open(2) system call. The term “handle” is synonymous with fd.
Non-Realtime Operation
QoS reserves bandwidth for non-realtime I/O clients. Differentiating between realtime and non-realtime I/O clients should result in better bandwidth utilization and overall performance.
During the mounting process, there is now provision to specify the amount of bandwidth to be reserved. Applications that do not have awareness of the external API can leverage the bandwidth amount. Also, clients with different load characteristics can be differentiated.
Non-realtime I/O support is provided through a central configuration file on the FSM server side, which defines the bandwidth reservation for non-realtime I/O requests on certain clients. This bandwidth reservation (RVIO) has higher priority than normal non-realtime clients, but lower priority over the realtime application (RTIO). FSM will always attempt to first satisfy realtime requests, and then follow the bandwidth reservation request. The remaining bandwidth is evenly distributed to all active non-realtime clients (excluding RVIO reserved clients).
To summarize, bandwidth priority is allocated in the following order:
- Realtime applications (RTIO)
- Bandwidth reservation (RVIO)
- Non-realtime I/O requests
Note: If there is not sufficient bandwidth to accommodate the bandwidth reservation request (RVIO), the client receives whatever bandwidth remains after RTIO requests have been granted. If there are multiple RVIO clients, then each client is allocated a RVIO bandwidth directly proportional to their requested bandwidth.
