Semaphore id's locked when working with multiple DBs

When working with multiple databases, the semaphore id's get locked by the process.
This can occur even if the database is shut down.

Conflict with the use of system resources.

When the first database was started, its before image writer sent the error (1075), followed by the server disconnection error (723) and finally the error (1132) showing the invalid semaphore id by the watchdog process:

-- Database 1
 

10:26:20 APW     5: Started. (2518)
10:26:20 APW     6: Started. (2518)
10:26:20 APW     7: Started. (2518)
10:26:30 BIW     3: Semaphore id 2 was removed (1075)
10:26:30 AIW     2: Semaphore id 2 was removed (1075)
10:26:30 AIW     2: ** The server has disconnected. (723)
10:26:30 BIW     3: ** The server has disconnected. (723)
10:27:01 WDOG    1: Disconnecting dead user 2. (2527)
10:27:01 WDOG    1: Invalid semaphore id (1132)
10:27:01 WDOG    1: SYSTEM ERROR: unable to get value of semaphore set :°, errno 22. (551)
10:31:13 BROKER  0: Broker could not spawn a server. (1157)

Almost simultaneously, the second running database starts sending the message (1132), an invalid semaphore id, followed by the error (551), unable to get value of semaphore set:

-- Database 2
 

09:44:08 Usr     4: Login by reh76555 on /dev/ttyp11. (452)
09:44:16 Usr     4: Userid is now reh76555. (708)
10:26:28 Usr     8: Login by reh76559 on /dev/ttyp0. (452)
10:32:03 Usr     8: Invalid semaphore id (1132)
10:32:03 Usr     8: SYSTEM ERROR: unable to set semaphore set :Ì, errno 22. (551)
10:32:03 Usr     8: Logout by  on /dev/ttyp0. (453)
10:32:03 Invalid semaphore id (1132)
10:32:03 SYSTEM ERROR: unable to get value of semaphore set :Ì, errno 22. (551)
10:55:13 Usr     9: Login by reh76095 on /dev/ttyp1. (452)
10:55:17 Usr     9: Semaphore id 2 was removed (1075)
11:03:05 Usr        Logout by  on /dev/ttyp1. (453)

Initial analysis seems to infer that there is a lack of system resources at the time. However, those databases were previously working with the same kernel parameters and there were no changes. There is a conflict in the use of the semaphores between these two databases due to semaphores remaining after the database is shutdown.

OPTION 1: Shutdown both databases and truncate their bi files. Usually this truncation completes normally, but if it becomes necessary you may need to use the -F truncate option. Please note that this may corrupt your database, as explained in the following Progress Articles: 

• 000020275: Consequences of using the -F startup option
• 000021932: Consequences of skipping crash recovery using the Force option (-F)

OPTION 2: Use ipcs -s to locate the semaphores those databases are using. Then shutdown both databases and make sure that semaphore's id's aren't still attached to any process. If this is the case, remove them with : ipcrm -s id.

OPTION 3: Review the OS System logs for kernel depletion messages. Increase kernel parameters for semaphores.

Progress Articles:

 Shared Memory and Semaphore Ids stored in Master Block error 1260   
 Using ipcs & ipcrm: Checking active message q's,memory, semaphores   
 Linux Database crashes when semaphores get removed