Das Problem
12.1 RAC-Datenbankfehler beim Starten über srvctl, Warnprotokoll zeigt den Datenbankstart und geht weiter nach unten:
alert_ORCL2.log in [Oracle Base]/diag/rdbms/[DB-Name]/[SID-Name]/trace:
Fri Nov 03 15:06:25 2017 Adjusting the default value of parameter parallel_max_servers from 960 to 486 due to the value of parameter processes (600) Starting ORACLE instance (normal) (OS id: 19684) . . Fri Nov 03 15:10:45 2017 Process startup failed, error stack: Fri Nov 03 15:10:45 2017 Errors in file /u01/app/oracle/diag/rdbms/ORCL/ORCL2/trace/ORCL2_psp0_19706.trc: ORA-27300: OS system dependent operation:fork failed with status: 11 ORA-27301: OS failure message: Resource temporarily unavailable ORA-27302: failure occurred at: skgpspawn3 Fri Nov 03 15:10:46 2017 Shutting down instance (abort) License high water mark = 2 Fri Nov 03 15:10:46 2017 USER (ospid: 22067): terminating the instance Fri Nov 03 15:10:47 2017 Instance terminated by USER, pid = 22067 Fri Nov 03 15:10:47 2017 Instance shutdown complete Errors in file //u01/app/oracle/diag/rdbms/ORCL/ORCL2/trace/ORCL2_mmon_19853.trc (incident=14681): ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], [] ORA-27300: OS system dependent operation:semop failed with status: 43 ORA-27301: OS failure message: Identifier removed ORA-27302: failure occurred at: sskgpwwait1 Incident details in: /u01/app/oracle/diag/rdbms/ORCL/ORCL2/incident/incdir_14681/ORCL2_mmon_19853_i14681.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details.
ORCL2_psp0_19706.trc:
*** 2017-11-03 15:10:44.989 Process startup failed, error stack: ORA-27300: OS system dependent operation:fork failed with status: 11 ORA-27301: OS failure message: Resource temporarily unavailable ORA-27302: failure occurred at: skgpspawn3 *** 2017-11-03 15:10:45.993 Process startup failed, error stack: ORA-27300: OS system dependent operation:fork failed with status: 11 ORA-27301: OS failure message: Resource temporarily unavailable ORA-27302: failure occurred at: skgpspawn3
ORCL2_mmon_19853.trc:
*** 2017-11-03 15:09:57.908 ***KELR Apply Log: unable to schedule MMON Slave, error 3 *** 2017-11-03 15:10:47.847 Incident 14681 created, dump file: /u01/app/oracle/diag/rdbms/ORCL/ORCL2/incident/incdir_14681/ORCL2_mmon_19853_i14681.trc ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], [] ORA-27300: OS system dependent operation:semop failed with status: 43 ORA-27301: OS failure message: Identifier removed ORA-27302: failure occurred at: sskgpwwait1 KEBM: MMON action policy violation. 'PQ: Adjust Slave Pool' viol=0; err=600 error 0 detected in background process kgxgnsdr: clssgsshdereg: warning: return status 26 (-558242808 ) OPIRIP: Uncaught error 447. Error stack: ORA-00447: fatal error in background process ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], [] ORA-27300: OS system dependent operation:semop failed with status: 43 ORA-27301: OS failure message: Identifier removed ORA-27302: failure occurred at: sskgpwwait1 kgxgnsdr: clssgsshdereg: warning: return status 26 (-813323584 )
Folgender Fehler wird in Betriebssystemprotokollen gemeldet
Nachrichten:
2017-11-03T15:10:11.690393+01:00 rachost1 Oracle Audit[7888]: LENGTH : '200' ACTION :[52] 'ALTER DATABASE MOUNT /* db agent *//* {2:39656:2} */' DATABASE USER:[1] '/' PRIVILEGE :[6] 'SYSDBA' CLIENT USER:[6] 'oracle' CLIENT TERMINAL:[0] '' STATUS:[1] '0' DBID:[10] '2949004148'
2017-11-03T15:10:47.700094+01:00 rachost1 kernel: [ 173.788859] cgroup: fork rejected by pids controller in /system.slice/ohasd.service <<<<<<<<<<<<<<<<<<<<< Datenbank kann mit sqlplus gestartet werden.
Die Lösung
Dies liegt an dem mit SUSE12 eingeführten PIDs cgroup Controller.
Aus dem Abschnitt „Support for PIDs cgroup Controller“ auf https://www.suse.com/releasenotes/x86_64/SUSE-SLES/12-SP2/#fate-320358"Um die Standardeinstellung TasksMax=für Dienste und Bereiche zu steuern, die auf dem System ausgeführt werden, verwenden Sie die system.conf-Einstellung DefaultTasksMax=. Diese Einstellung ist standardmäßig 512, was bedeutet, dass Dienste, die ansonsten nicht explizit konfiguriert sind, nur 512 Prozesse erstellen können oder Threads maximal.
Für Thread- oder prozessintensive Dienste müssen Sie möglicherweise einen höheren TasksMax-Wert festlegen. Stellen Sie in solchen Fällen TasksMax direkt in den jeweiligen Unit-Dateien ein. Wählen Sie entweder einen numerischen Wert oder sogar unendlich."
Ab SLES12 wird systemd statt initd verwendet und der OHASD-Server darf nur noch maximal 512 Tasks öffnen. Konfigurieren Sie den Wert von DefaultTasksMax bis 65535 in der Datei /etc/systemd/system.conf oder setzen Sie den TasksMax-Wert richtig für den ohasd systemd-Dienst.
Zum Beispiel:
# cat /etc/systemd/system/ohasd.service.d/lunar.conf [Service] TasksMax=16384
# systemctl status ohasd ● ohasd.service - LSB: Start and Stop Oracle High Availability Service Loaded: loaded (/etc/init.d/ohasd; bad; vendor preset: disabled) Drop-In: /etc/systemd/system/ohasd.service.d └─lunar.conf Active: active (exited) since Mon 2017-11-13 14:29:23 CET; 3h 5min ago Docs: man:systemd-sysv-generator(8) Process: 4876 ExecStart=/etc/init.d/ohasd start (code=exited, status=0/SUCCESS) Tasks: 612 (limit: 16384) <<<<<<=================== here was the limit with 512