Synchronous I/O (direct I/O) means wait for each I/O to complete.
Asynchronous I/O means issuing I/O then wait for them to complete or issuing an I/O without waiting and being allowed to perform other processing and then ask or wait for the previously issued I/O completion. There are ways for one thread to wait on another thread's I/O completion. The way the Asynchronous I/O is performed is platform dependent.
Progress does not do any Asynchronous I/O per se, as aio_read() or aio_write() system calls are not used. Progress implements its own Asynchronous I/O "processing" through its file (ai, bi and db) buffering and I/O by page writers. While the process that makes a change to a block in a buffer does not actually perform the I/O, it continues its processing while the I/O is waiting in a buffer pool (-aibufs, -bibufs or -B) to be written out by another process (the AIW, BIW, APW).
Progress uses two flavors of synchronous I/O:
- Buffered Synchronous I/O and
- Unbuffered Synchronous I/O (direct I/O). Some consider buffered I/O to be Asynchronous but it truly is not when Asynchronous I/O is defined as allowing other operations to execute by the same thread before an issued I/O completes
To guarantee crash recovery, a
write ahead logging (WAL) rule is enforced which basically states that we will always
write to the bi file first and then to the data files. To follow this protocol, not only do we need to issue the writes in the proper order, but we must ensure that the physical changes to the disk are done in the proper order.
Therefore, the current implementation of Progress I/O is as follows:1. Unbuffered synchronous I/O (O_RDWR|O_DSYNC) to the bi, crash recovery guaranteed unless running with -i (no-integrity) -R (non-raw) startup flags:
- When running with the -R (non-raw) startup flag - we do Buffered Synchronous I/O (O_RDWR) to the bi file. This allows rollback and possibly crash recovery. If the database were to crash, crash recovery should work as all the data needed should still be in the file system's cache if it hasn't made it to physical disk yet. However, if the system were to crash, it is not guaranteed that crash recovery will work - it most likely will not.
- When running with the -i (no-integrity) startup flag (-i forces -R) - we do Buffered Synchronous I/O (O_RDWR) to the bi file and no bi/data order synchronization. This means that rollback should work, but if the database were to crash or a user get killed even if the system were not to crash, there is no guarantee that crash recovery will complete. In this case, you are prevented from connecting to the database - we don't even try to recover.
** The last session was run with the no integrity (-i) parameter. (509)
** Your database cannot be repaired. You must restore a backup copy. (510)
2. When running with
-directio startup flag - we do Unbuffered Synchronous I/O (O_RDWR|O_DSYNC) to the data files. This means direct writes to disk for data files and allows us to avoid needing fdatasync() calls at checkpoints. Some customers see improvements running this way, some don't. It all depends on the application. The idea is that since Progress maintains a Buffer Pool, why do we need to use the file system's "buffer pool" as well? This would mean we have a cache on top of a cache. There are cases where double caching is better but that discussion is not the scope of this Article.
3. When not running with -directio - we do Buffered Synchronous I/O (O_RDWR) to the datafiles which means that we must issue an fdatasync() call when we want to guarantee that the data I/O has made it out of the file system's cache and has been written to disk (usually only at checkpoint time).
The difference between Buffered I/O and Unbuffered Synchronous I/O is that an Unbuffered Synchronous I/O is guaranteed to be on disk when returned from the write() system call. A Buffered Synchronous I/O is guaranteed to exist, either in the file system's cache or possibly on disk when returned from the write() system call.
In the above discussion,
fdatasync() on UNIX,
FileFlushBuffers() is used on Windows
