View Issue Details

IDProjectCategoryView StatusLast Update
0000527Apache 2.x Bugpublic2022-05-24 18:34
ReporterMichelPB Assigned Topsmedley  
PriorityhighSeveritycrashReproducibilityalways
Status closedResolutionfixed 
PlatformOs/2OSEcsOS Version2.0
Product Versionv2.2.1 
Summary0000527: The server the apache ceases to work after several hours of normal work
DescriptionStop of work of the server
To a broad gull of errors writes
LIBC PANIC!!
fmutex deadlock: Owner died!
0x0063013c: Owner=0x08be0005 Self=0x08be0001 fs=0x3 flags=0x0 hev=0x00010050
            Desc="LIBC Heap"
pid=0x08be ppid=0x0041 tid=0x0001 slot=0x00e5 pri=0x0200 mc=0x0000
D:\APPS\APACHE2\BIN\HTTPD.EXE
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.
TagsNo tags attached.
Attached Files
error_log (286,725 bytes)
error_log.php5.4 (13,194 bytes)
popuplog.os2.php5.4 (4,465 bytes)

Activities

MichelPB

2012-03-27 09:41

reporter   ~0002231

The Apache 2.22 server departure with php 5.4 proceeds
after 2 hours of normal work.
further normal work of 6 hours with average loading

MichelPB

2012-04-24 00:43

reporter   ~0002244

Last edited: 2012-04-24 00:44

Problem in libc064 library
arises and in other products using the last, for example mysql, samba.
At kickback of programs on the younger version and kickback of libraries on libc063 the problem doesn't arise.

[2012/04/19 09:13:12, 2] smbd/server.c:open_sockets_smbd(620)
  waiting for a connection

Killed by SIGSEGV
pid=0x0068 ppid=0x0000 tid=0x0001 slot=0x00a9 pri=0x0400 mc=0x0001
C:\PROGRAMS\SAMBA\SMBD.EXE
LIBC064 0:000f0242
cs:eip=005b:1dfd0242 ss:esp=a39f:0061d512 ebp=0061d568
 ds=a39f es=a39f fs=150b gs=0000 efl=00010216
eax=0000a39f ebx=0061d540 ecx=00000000 edx=0000150b edi=0061db80 esi=0061d600
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.

psmedley

2012-04-25 09:25

administrator   ~0002251

Use libc065 and the forwarder libc064.dll until I rebuild apache2 with libc065

MichelPB

2012-04-26 09:34

reporter   ~0002255

Last edited: 2012-04-26 09:37

At installation of Libc065 libraries continues Apache departures, and from the start of system.
It was rolled away on Libc063.
The departure ravine at start

LIBC PANIC!!
fmutex deadlock: Owner died!
0x0063013c: Owner=0x004a0004 Self=0x004a0001 fs=0x3 flags=0x0 hev=0x00010050
            Desc="LIBC Heap"
pid=0x004a ppid=0x0032 tid=0x0001 slot=0x0083 pri=0x0200 mc=0x0000
D:\APPS\APACHE2\BIN\HTTPD.EXE
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.

LIBC PANIC!!
fmutex deadlock: Owner died!
0x0063013c: Owner=0x004a0004 Self=0x004a0001 fs=0x3 flags=0x0 hev=0x00010050
            Desc="LIBC Heap"
pid=0x004a ppid=0x0032 tid=0x0001 slot=0x0083 pri=0x0200 mc=0x0000
D:\APPS\APACHE2\BIN\HTTPD.EXE
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.

Popuplog.os2

04-26-2012 14:59:59 SYS3175 PID 003e TID 0005 Slot 0089
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffcc354
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=20036e40 ECX=0270ff70 EDX=0270ffd4
ESI=0270ffd4 EDI=00000005
DS=0053 DSACC=d0f3 DSLIM=6fffffff
ES=0053 ESACC=d0f3 ESLIM=6fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffcc354 CSACC=d0df CSLIM=6fffffff
SS:ESP=0053:0270ff94 SSACC=d0f3 SSLIM=6fffffff
EBP=0270ff94 FLG=00010206

DOSCALL1.DLL 0002:0000c354

------------------------------------------------------------

04-26-2012 14:59:59 SYS3175 PID 003f TID 0011 Slot 0094
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffcc354
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=2003f500 ECX=028eff70 EDX=028effd4
ESI=028effd4 EDI=00000011
DS=0053 DSACC=d0f3 DSLIM=6fffffff
ES=0053 ESACC=d0f3 ESLIM=6fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffcc354 CSACC=d0df CSLIM=6fffffff
SS:ESP=0053:028eff94 SSACC=d0f3 SSLIM=6fffffff
EBP=028eff94 FLG=00010206

DOSCALL1.DLL 0002:0000c354

------------------------------------------------------------

LewisR

2012-12-23 05:15

developer   ~0002389

I've found some interesting behavior with this under Apache 2.2.23, while testing different settings in conf/extra/httpd-mpm.conf.

With the default settings of (I believe):

    ThreadStackSize 65536
    StartServers 2
    MinSpareThreads 5
    MaxSpareThreads 10
    MaxRequestsPerChild 0

I get *no* LIBC panic messages (crashes can be triggered by doing a specific number of graceful restarts of the running daemon; in my test case, that number, with the current mix of modules - including PHP 5.3.19 and its module set - is 21).

However, increasing the number of StartServers beyond 2 - without increasing the MinSpareThreads & MaxSpareThreads - will lead to a rapid accumulation of LIBC panic messages as referenced in this bug. It's hard to tell where the appropriate proportions lie between StartServers and the size of the thread pool (it is apparently not a linear relationship), but there is definitely a correlation between these directives.

So, to determine whether this is indeed an issue, either set the MPM values back to defaults or comment the line in httpd.conf to include the MPM conf. Then see if these LIBC panics continue to occur.

On this test system, I have the following LIBC dlls:

 4-14-04 16:37 356,330 0 libc05.dll
 3-23-12 4:32 48,142 0 libc06.dll
 3-23-12 4:32 48,142 0 libc061.dll
 3-23-12 4:32 157,124 0 libc062.dll
 3-23-12 4:32 157,124 0 libc063.dll
 3-23-12 4:32 157,176 0 libc064.dll
 3-23-12 4:32 1,353,208 0 libc065.dll

Igor Vaskov

2013-01-01 23:04

reporter   ~0002392

Do not set the default value!
After prolonged use, and relatively high load you will get 100% load on ALL SMP processors. Server is very slow, the WPS interface responds to 30 seconds at the click of a mouse.
When setting:

ThreadStackSize 262144
StartServers 2
MinSpareThreads 50
MaxSpareThreads 60

Everything works without 100% load.

psmedley

2013-03-29 08:12

administrator   ~0002419

http://smedley.id.au/tmp/httpd-2.2.24-os2-20130329.zip might help with some of this....

psmedley

2013-10-10 23:04

administrator   ~0002545

Please test with current (ie September 2013 or greater) builds of apache/php which are built with GCC 4.7.3

Igor Vaskov

2013-10-11 07:05

reporter   ~0002552

I have another, but similar problem.
My error log:
APACHE2: UnableToOpenConfigurAPACHE2: UnableToOpenConfigureFile `delegates.xml' @ warning/configureAPACHE2: UnableToOpenConfigurAPACHE2: UnableToOpenConfigureFile `delegates.xml' @ warning/configure.c/GetConfigureOptions/591.

LIBC PANIC!!
fmutex deadlock: Recursive mutex!
0x0063013c: Owner=0x00450001 Self=0x00450001 fs=0x3 flags=0x0 hev=0x00010050
            Desc="LIBC Heap"
pid=0x0045 ppid=0x003c tid=0x0001 slot=0x0096 pri=0x0200 mc=0x0001
E:\APPS\APACHE2\BIN\HTTPD.EXE
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.

LIBC PANIC!!
fmutex deadlock: Recursive mutex!
0x0063013c: Owner=0x00450001 Self=0x00450001 fs=0x3 flags=0x0 hev=0x00010050
            Desc="LIBC Heap"
pid=0x0045 ppid=0x003c tid=0x0001 slot=0x0096 pri=0x0200 mc=0x0001
E:\APPS\APACHE2\BIN\HTTPD.EXE
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.

LIBC PANIC!!
fmutex deadlock: Recursive mutex!
0x0063013c: Owner=0x00450001 Self=0x00450001 fs=0x3 flags=0x0 hev=0x00010050
            Desc="LIBC Heap"
pid=0x0045 ppid=0x003c tid=0x0001 slot=0x0096 pri=0x0200 mc=0x0001
E:\APPS\APACHE2\BIN\HTTPD.EXE
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.

----And many block records fmutex deadlock: Recursive mutex! --------

In result - memory leak and stoped service.
Maybe open a new ticket?

psmedley

2013-10-11 11:01

administrator   ~0002557

yes, please open a new ticket with as much detail as possible on how to reproduce

LewisR

2015-02-07 20:51

developer   ~0003022

Michel, are you still seeing this, or may we close this ticket and focus on the other related stability issues?

psmedley

2019-09-29 01:07

administrator   ~0003336

Closing this due to lack of feedback.

Issue History

Date Modified Username Field Change
2012-02-15 03:34 MichelPB New Issue
2012-02-15 03:34 MichelPB File Added: error_log
2012-03-27 09:36 MichelPB File Added: error_log.php5.4
2012-03-27 09:37 MichelPB File Added: popuplog.os2.php5.4
2012-03-27 09:41 MichelPB Note Added: 0002231
2012-04-24 00:43 MichelPB Note Added: 0002244
2012-04-24 00:44 MichelPB Note Edited: 0002244
2012-04-25 09:25 psmedley Note Added: 0002251
2012-04-26 09:34 MichelPB Note Added: 0002255
2012-04-26 09:37 MichelPB Note Edited: 0002255
2012-12-23 05:15 LewisR Note Added: 0002389
2013-01-01 23:04 Igor Vaskov Note Added: 0002392
2013-03-29 08:12 psmedley Note Added: 0002419
2013-10-10 23:04 psmedley Note Added: 0002545
2013-10-10 23:04 psmedley Assigned To => psmedley
2013-10-10 23:04 psmedley Status new => feedback
2013-10-11 07:05 Igor Vaskov Note Added: 0002552
2013-10-11 11:01 psmedley Note Added: 0002557
2015-02-07 20:51 LewisR Note Added: 0003022
2019-09-29 01:07 psmedley Status feedback => closed
2019-09-29 01:07 psmedley Resolution open => fixed
2019-09-29 01:07 psmedley Note Added: 0003336