This Article outlines the UNIX kernel parameters that might require tuning as a result of installing a Progress application.
In most cases the default values for most of the UNIX kernel are sufficient. As more users or the number of processes executed increases over time, the Progress runtime environment will start to return system errors that indicate a shortage of some kernel resource (memory, number of files that can be opened by a user ...) at which time kernel parameters need to be reviewed.
Where applicable, this Article indicates whether a particular parameter is no longer required, when the value for the parameter is dynamically allocated in some UNIX or Linux Systems, or where parameter in a Version of UNIX is superseded by another UNIX Version kernel parameter.
This Article is only for informative purposes. Kernel parameter names vary from system to system. This Article provides an overview so that the correct kernel parameter on the particular system can be highlighted. If in doubt, consult your UNIX system Documentation.
What is the kernel?The kernel is the heart of the UNIX operating system. It manages memory, schedules processes, manages I/O, and does all other lower-level tasks that make the system work. The kernel is not a process (like the programs that run from a terminal). It pre-exists all processes and is responsible for starting them. Because it is so important, the kernel has key privileges. It is always resident in the processor's physical memory. While other programs can be swapped or paged to disk, but not the kernel.
Reviewing current kernel parameter values:Kernel parameters are defined in files semaphores (sem.h), shared memory (shm.h) and messages (msg.h) or similar files depending on the Operating System. Process limit related kernel parameters are handled differently by each Operating System. The name of the kernel configuration file and its location varies from system to system. Progress does not use parameters within msg.h (message queues).
A brief discussion on where current kernel settings can be found and changed on major supported Unix Operating systems follows with the caveat: Please consult your system documentation.
AIX
Upper limits are set for shared memory related parameters and their usage is dynamically allocated and de-allocated up to these upper limits, there’s not much to tune / set due to the set hard limit for shared memory related parameters.
The menu driven System Administration shell, SMIT can be used to view or modify current tunable kernel parameters,
Or from the command line, tunable process limit parameters can be displayed with: lsattr -EHl sys0
And changed with chdev, For example: Changing the maximum number of processes for user to 2048
$ chdev -l sys0 -a maxuproc=2048
Linux
Pre kernel version 2.4, semaphores (include/kernel/sem.h) and shared memory (include/kernel/shm.h) are hard coded in these files that are distributed with the Linux kernel tree.
A recompile of the kernel and reboot is required to changes made to sem.h/shm.h get populated.
Post kernel version 2.4 the current values are found under /proc/sys/kernel and can be dynamically changed on the fly where the session changes are valid until the next reboot.
For example,
To dynamically change SEMMSL, SEMMNS, SEMOPM, and SEMMNI:
$ echo 500 32000 200 2048 > /proc/sys/kernel/sem
To make the change permanent at the next reboot add the change to the file /etc/sysctl.conf file which is used during the boot process
$ echo "kernel.sem=500 32000 200 2048" >> /etc/sysctl.conf
HPUX
A recompile of the kernel and reboot is required once changes are made.
The menu driven System Administration Manager, SAM can be used to view or modify current tunable kernel parameters:
Kernel Configuration -> Configurable Parameters
Once changes are complete: “Process New Kernel”, “Move Kernel Into Place” and “Shutdown/Reboot System Now”
or manually using commands kmtune (query, set, reset); mk_kernel, kmupdate, reboot
SOLARIS
The ‘/etc/sysdef’ command provides the values for:
IPC Semaphores (sem.h),
IPC Shared Memory (shm.h),
Tunable Parameters (process limits).
To change values adjust the value for the appropriate tunable kernel parameter in: /etc/system
General UNIX Kernel Parameters:
- NPROC: The maximum number of simultaneous processes that the system can run.
- MAXPROC, MAXUP: The maximum number of processes that any user (any user ID number) can have at one time.
- NFILE: The maximum number of files (system-wide) that can be open at any time.
- NOFILES or NFILE: The maximum number of open files the process might have. When this limit is exceeded, attempts to open files fails and errno is set to EMFILE.
- ULIMIT: The size of the largest file that a process can create. There are user-level ways of overriding this kernel limit, such as the ulimit command and the ULIMIT line in /etc/default/login.
Memory Management Parameters:
- MAXUMEM: The maximum amount of memory that is used to run a process.
- MAXPMEM: Specifies the amount of physical memory in use ages. The value 0 means "use all of physical memory".
- SVMMLIM: The soft limit that specifies the maximum address space that can be mapped to a process. Attempts to increase a process's address space beyond this value fails with an ENOMEM error.
- HVMMLIM: The maximum value of SVMMLIM. While any user process can change its soft limits up to the corresponding hard limit, only root can increase the hard limits.
Streams Parameters:Progress only requires STREAMS to be installed. It does not require specific values for the streams parameters. For example when error
“Select returned -1 , errno=11. (793)” the first next step is to ensure that streams are installed
Inter-Process Communication (IPC) Parameters:
- SHMMNI: The maximum number of shared memory identifiers allowed for the system.
- SHMALL: The maximum number of in-use shared memory segments allowed for the system. Most systems do not have this parameter. Some Systems increase this value dynamically.
- SHMMAX: The maximum size of a single shared memory segment. Some systems call this parameter SHMSIZE.
- SHMSEG: The maximum number of shared memory segments a single process can attach to. Some systems do not have this parameter. Some Systems increase this value dynamically.
- SEMMNI: The maximum number of semaphore identifiers that are allowed for the system.
- SEMMSL: The maximum number of semaphores allowed per semaphore identifier (semaphore set).
- SEMMNS: The total number of semaphores allowed for the system.
- SEMMNU: The maximum number of semaphore undo structures allowed for the system.
Progress and General Kernel Parameters:As more processes are executed simultaneously on a machine, NPROC may need increasing. Progress statements OUTPUT THROUGH and INPUT THROUGH open pipes to UNIX that create additional processes on the system. If the problem is related to NSWAP, the local buffer (-l) or edit buffer (-e) start up parameters would need reviewing. When the NPROC or NSWAP parameter is exceeded, Progress returns error messages such as:
Too many subprocesses, cannot fork. Errno=<number>. (358)
Unable to fork process.
Additional processes may need to be executed by a particular use which would require MAXPROC, MAXUP parameters to be increased.
If additional files are opened as a result of increasing the number of processes that are executed corresponding NFILE, NFILES parameters would need attention.
Similarly, NFILE, NFILES parameters would also need increasing when more databases or many more extents are added to existing databases.
Blocks in Database Buffers (-B/-B2), Lock-table Entries (-L), and Number of Users (-n) startup parameters all affect Shared-Memory usage.
Number of Users (-n) and Maximum Servers (-Mn) parameters affect semaphore usage (each user or server process uses one semaphore).
Before re-configuring the kernel to increase shared memory or semaphore allocation, ensure these startup values are properly configured to current requirements.
Progress and UNIX Shared Memory:Progress uses Shared Memory to allow multiple remote server processes and self-service client processes concurrent access to a database in order to coordinate their activities.
The Shared Memory area contains the Buffer Pool, Lock Table, Log Buffers, and other shared data structures set by database startup parameters. When the database is started multi-user, the Database Broker creates and initializes the Shared Memory area to the size required by the values of database startup parameters which determine the sizes of the data structures.
On UNIX systems, Shared Memory consists of one or more segments. Several kernel parameters limit the number and size of shared memory segments. Each segment is identified by a system-wide unique number assigned by the process that creates the segment. Progress derives the identifier from the inode of the .db file. Additionally, Progress has an internal limit on the size of a single shared memory segment. If the shared memory size computed by the broker is larger than the maximum segment size allowed, the broker creates multiple segments to reach the required total. In OpenEdge 10.1B a new database configuration parameter, -shmsegsize, allows the shared-memory segment size to be set explicitly.
The total amount of shared memory that can be allocated is:
SHMALL (or SHMMAX * SHMMNI).
The combined requirements of Progress and any other software that also uses shared memory need to be taken into account.
Recommended Shared Memory settings:The following recommended settings are for Progress applications. The values required by Progress must be added to the current kernel parameter values. In other words, the following recommendations do not take into account the requirements of the UNIX Operating System and/or any other application that uses shared memory and semaphores. Consult your system documentation to determine UNIX's requirement.
SHMALL: (Total size of Shared Memory Segments System wide)
SHMALL = SHMMAX * SHMMNI
SHMALL will always be less than the Physical Memory (excluding Cache/Swap) on the System by at least the amount of memory required by the Operating System.
SHMMAX: (Maximum size of a segment)
SHMMAX limits the size of a single shared memory segment. Progress creates a smaller segment if less shared memory is needed than the maximum size. When SHMMAX is set too low, Progress needs to create many small segments (limited by the maximum size) to satisfy shared memory requirements. As a result, the limits SHMALL, SHMMNI or SHMSEG may overflow. The amount of Kernel Memory required for Shared Memory descriptions, not the shared memory itself, is relatively small, so setting SHMSEG, SHMMNI, higher than current needs probably won't affect performance.
Shared memory is not used for single-user sessions. Clients that connect to a remote database on a server machine do not use shared memory on the client machine, they attach to shared memory on the server machine.
The units used to specify a value for SHMMAX vary from system to system. Some units use bytes, others use "clicks" (pages). 128 MB for 32-bit and 1GB for 64-bit are ballpark values. In OpenEdge 10.1B a new database configuration parameter, -shmsegsize, allows the shared-memory segment size to be set explicitly in order to be able to control the number of SHMMAX segments that are used.
SHMMNI: (Maximum number of shared memory segments per process) and
SHMSEG: (The maximum number of shared memory identifiers system wide)
SHMMNI and SHMSEG depend on the number of active databases, the values of database startup parameters, and the current value of SHMMAX. Typically, the larger SHMMAX, the fewer segments Progress creates.
When SHMMNI is exceeded typically the following errors result:
Maximum number of shared-memory identifiers exceeded. (1137)
No more available shared-memory. (1138)
When self-service clients connect to multiple databases, shared memory segments for the database are attached. When SHMSEG are exceeded typically the following errors result:
Maximum number of shared-memory segments per process exceeded. (1175)
Unable to create shared memory <shmname>, error <n>. (1719)
Progress and UNIX Semaphores:Progress uses UNIX Semaphores to synchronize the activities of server processes and self-service client processes that are connected to a database. Semaphores are created and initialized by the Database broker when the database is started.
UNIX semaphores are grouped into semaphore sets. Each semaphore set has a unique identifying number called a semid. Within a semaphore set, individual semaphores are identified by an integer that ranges from 0 to one less than the size of the semaphore set.
The Database Broker creates a semaphore set when the database is started. The original recommended size of the semaphore set is the number of users (-n parameter) plus the number of servers (-Mn parameter) plus 4, however, the newer versions have added more auxiliary processes.
The number of semaphores within a semaphore set (SEMMSL) is determined by the -n and -Mn startup parameter for each database. Progress auxiliary process need to be included in the Maximum Users (-n) count, i.e. APWs, AIW,BIW, WDOG, replication process, online backup, promon sessions, etc..
The amount of kernel memory required for semaphores is relatively small, so setting the limits higher than current needs probably won't affect performance.
Semaphores are not used for single user sessions. Clients that connect to a remote database on a server machine do not use semaphores on the client machine.
Recommended Semaphores Settings:SEMMNI: (Maximum number of semaphore identifiers)
SEMMNI =Three semsets per active Enterprise database, otherwise one. The multiple semaphore sets feature is only available with the Enterprise Database Server product.
When SEMMNI is exceeded errors such as “
Progress Error Semaphore limit exceeded (1131)” arise.
SEMMSL: (Maximum number of semaphores per set)
SEMMSL = Maximum number of users on any database plus (+) the maximum number of remote servers for any database plus (+) 4 plus (+) the number auxiliary process.
When SEMMSL is exceeded errors such as
Warning: only <num> wait semaphores are available. (1093)
Less than minimum number of semaphores available (1130)
SEMMNS: (Maximum number of semaphores system-wide)
SEMMNS = SEMMSL multiply by (*) the number of active databases.
When SEMMNS is exceeded the following errors arise:
Warning: only <num> wait semaphores are available. (1093)
Semaphore limit exceeded (1131)
No available wait semaphore. (1195)
SEMMNU:
SEMMNU = SEMMNS
When SEMMNS is exceeded the following errors arise:
Too many users requested semaphore undo, increase SEMMNU. (1081)
MAXUMEM: Maximum amount of memory that will be used to run a process
Typically needs increasing when SHMMAX, SHMSEG are raised.
