View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000425Apache 2.x Bugpublic2010-05-28 19:152020-08-24 03:11
ReporterIgor Vaskov Assigned Topsmedley  
PrioritynormalSeveritymajorReproducibilityrandom
Status closedResolutionfixed 
OSECSOS Version1.2 
Product Versionv2.2.1 
Summary0000425: Unknown crashes with Apache 2.2.11
DescriptionApache process crashes with message in POPUPLOG.OS2:

05-28-2010 21:09:49 SYS3175 PID 8507 TID 0006 Slot 00e2
D:\APACHE22\BIN\HTTPD.EXE
c0000005
1ffc1b0d
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=20035860 ECX=0236ffd4 EDX=0236ffd4
ESI=0236ffd4 EDI=00000006
DS=0053 DSACC=f0f3 DSLIM=ffffffff
ES=0053 ESACC=f0f3 ESLIM=ffffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffc1b0d CSACC=f0df CSLIM=ffffffff
SS:ESP=0053:0236ff94 SSACC=f0f3 SSLIM=ffffffff
EBP=0236ff94 FLG=00012213

DOSCALL1.DLL 0002:00001b0d


Server do work stable, bat size POPUPLOG.OS2 very large.
Part POPUPLOG.OS2 in attache.
Any ideas?
TagsNo tags attached.
Attached Files
POPUPLOG.OS2 (77,438 bytes)
4B80_01.TRP (27,597 bytes)
4E26_01.TRP (22,326 bytes)
4E27_01.TRP (22,326 bytes)
4E2C_01.TRP (22,326 bytes)
4E2D_01.TRP (20,152 bytes)

Relationships

has duplicate 0000355 closedpsmedley Random crashes with Apache 2.2.11 
has duplicate 0000299 closedpsmedley Apache + PHP5 crashes regularly under light load 

Activities

psmedley

2010-07-11 03:12

administrator   ~0001637

Please try http://download.smedley.info/httpd-2.2.15-os2-20100711.zip

Igor Vaskov

2011-03-20 13:36

reporter   ~0001865

Apache version 2.2.17
Many of the same errors.

POPUPLOG.OS2:

01-28-2011 20:31:43 SYS3175 PID 0034 TID 0008 Slot 00e1
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffc1b65
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=20038fe0 ECX=0276ffd4 EDX=0276ffd4
ESI=0276ffd4 EDI=00000008
DS=0053 DSACC=d0f3 DSLIM=5fffffff
ES=0053 ESACC=d0f3 ESLIM=5fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffc1b65 CSACC=d0df CSLIM=5fffffff
SS:ESP=0053:0276ff94 SSACC=d0f3 SSLIM=5fffffff
EBP=0276ff94 FLG=00010213

DOSCALL1.DLL 0002:00001b65

------------------------------------------------------------

Igor Vaskov

2011-12-04 12:11

reporter   ~0002065

Apache v. 2.2.21
This problem is present now.

LewisR

2011-12-31 20:57

developer   ~0002116

This is consistent with what I see:

12-31-2011 15:40:11 SYS3175 PID 0bd1 TID 000e Slot 00f2
J:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffb1b65
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=2003b600 ECX=024fffd4 EDX=024fffd4
ESI=024fffd4 EDI=0000000e
DS=0053 DSACC=f0f3 DSLIM=ffffffff
ES=0053 ESACC=f0f3 ESLIM=ffffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffb1b65 CSACC=f0df CSLIM=ffffffff
SS:ESP=0053:024fff94 SSACC=f0f3 SSLIM=ffffffff
EBP=024fff94 FLG=00010213

DOSCALL1.DLL 0002:00001b65

I will soon be generating a dumpfile for analysis. Steve Levine is working with me on this.

Meanwhile, Igor, please have a look at http://tinyurl.com/7478bd6 for more discussion of this. Could you also post some details of your configuration, such as, number of virtual hosts, modules loaded (as well as versions, for things like PHP) so that we can try to gather some more common factors?

Thanks!

LewisR

2011-12-31 21:04

developer   ~0002117

Paul: in http://mantis.smedley.info/view.php?id=299#c1255 you suggested increasing file handles. FWIW, I have been running with SHELLHANDLESINC=200 for quite some time (43 vhosts presently, with separate access & error logs for each, aside from the other things running on the box).

Also, mpm conf has:

    ThreadStackSize 2097152
    StartServers 2
    MinSpareThreads 5
    MaxSpareThreads 10
    MaxRequestsPerChild 0

Igor Vaskov

2012-01-02 11:30

reporter   ~0002125

After applying the settings

SHELLHANDLESINC=200
and
ThreadStackSize 2097152

the number of errors increased.

There is a question. SHELLHANDLESINC=200 can be set from a batch file just before starting httpd or just the file system configuration (config.sys)?

I rollback ThreadStackSize to default settings.

LewisR

2012-01-02 13:53

developer   ~0002126

Very interesting, Igor.

SHELLHANDLESINC must be set in CONFIG.SYS. It cannot be set from a batch file, as far as I know.

Interesting about ThreadStackSize. I'll roll mine back to default and see what I get. I increased it some time ago, in an effort to keep Apache from overrunning the stack (apparently a common issue for some operating systems).

Thanks for checking and reporting. Happy New Year!

Igor Vaskov

2012-01-03 17:43

reporter   ~0002132

Result of my tests...
Before
    ThreadStackSize 2097152
    StartServers 2
    MinSpareThreads 5
    MaxSpareThreads 10
    MaxRequestsPerChild 0

Many errors from error_log:
Killed by SIGSEGV
pid=0x0e93 ppid=0x0e92 tid=0x0001 slot=0x00db pri=0x0200 mc=0x0001
D:\APPS\APACHE2\BIN\HTTPD.EXE
LIBC064 0:000f0242
cs:eip=005b:1df10242 ss:esp=b87f:0022fb9a ebp=0022fbf0
 ds=b87f es=b87f fs=150b gs=0000 efl=00210206
eax=0000b87f ebx=0022fbc8 ecx=00000646 edx=0000150b edi=02690fb4 esi=00666640
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.

I change Settings:
"set SHELLHANDLESINC=200" (config.sys)
httpd-mpm.conf
    ThreadStackSize 262144
    StartServers 2
    MinSpareThreads 50
    MaxSpareThreads 60
    MaxRequestsPerChild 0

In result change error from error_log:
[Tue Jan 03 15:03:34 2012] [error] (OS 343)OS/2 error 343: unable to open work queue, exiting
[Tue Jan 03 17:57:21 2012] [error] (OS 343)OS/2 error 343: unable to open work queue, exiting
[Tue Jan 03 19:41:46 2012] [error] (OS 343)OS/2 error 343: unable to open work queue, exiting
[Tue Jan 03 19:41:48 2012] [error] (OS 343)OS/2 error 343: unable to open work queue, exiting
[Tue Jan 03 19:55:36 2012] [error] (OS 343)OS/2 error 343: unable to open work queue, exiting
[Tue Jan 03 20:48:13 2012] [error] caught exception (XCPT_ACCESS_VIOLATION) in worker thread, initiating child shutdown pid=4626
[Tue Jan 03 20:48:13 2012] [error] caught exception (XCPT_ACCESS_VIOLATION) in worker thread, initiating child shutdown pid=4626
[Tue Jan 03 20:48:13 2012] [error] caught exception in worker thread, initiating child shutdown pid=4626
[Tue Jan 03 20:48:13 2012] [error] caught exception in worker thread, initiating child shutdown pid=4626

popuplog.os2
01-03-2012 19:55:35 SYS3175 PID 0ea7 TID 0021 Slot 0100
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffc1b65
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=2004a8a0 ECX=02e6ffd4 EDX=02e6ffd4
ESI=02e6ffd4 EDI=00000021
DS=0053 DSACC=d0f3 DSLIM=5fffffff
ES=0053 ESACC=d0f3 ESLIM=5fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffc1b65 CSACC=d0df CSLIM=5fffffff
SS:ESP=0053:02e6ff94 SSACC=d0f3 SSLIM=5fffffff
EBP=02e6ff94 FLG=00010213

DOSCALL1.DLL 0002:00001b65

------------------------------------------------------------

01-03-2012 19:55:36 SYS3175 PID 1217 TID 0027 Slot 0107
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffc1b65
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=2004eb60 ECX=0302ffd4 EDX=0302ffd4
ESI=0302ffd4 EDI=00000027
DS=0053 DSACC=d0f3 DSLIM=5fffffff
ES=0053 ESACC=d0f3 ESLIM=5fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffc1b65 CSACC=d0df CSLIM=5fffffff
SS:ESP=0053:0302ff94 SSACC=d0f3 SSLIM=5fffffff
EBP=0302ff94 FLG=00010213

DOSCALL1.DLL 0002:00001b65

------------------------------------------------------------

01-03-2012 20:06:35 SYS3175 PID 1218 TID 003d Slot 00e0
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffc1b65
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=2005e0a0 ECX=0790ffd4 EDX=0790ffd4
ESI=0790ffd4 EDI=0000003d
DS=0053 DSACC=d0f3 DSLIM=5fffffff
ES=0053 ESACC=d0f3 ESLIM=5fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffc1b65 CSACC=d0df CSLIM=5fffffff
SS:ESP=0053:0790ff94 SSACC=d0f3 SSLIM=5fffffff
EBP=0790ff94 FLG=00010213

DOSCALL1.DLL 0002:00001b65

------------------------------------------------------------

01-03-2012 20:48:14 SYS3175 PID 1212 TID 002c Slot 00da
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffc1b65
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=20052300 ECX=0317ffd4 EDX=0317ffd4
ESI=0317ffd4 EDI=0000002c
DS=0053 DSACC=d0f3 DSLIM=5fffffff
ES=0053 ESACC=d0f3 ESLIM=5fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffc1b65 CSACC=d0df CSLIM=5fffffff
SS:ESP=0053:0317ff94 SSACC=d0f3 SSLIM=5fffffff
EBP=0317ff94 FLG=00010213

DOSCALL1.DLL 0002:00001b65

In latest configuration sever become more stable.

Bachinskiy

2012-01-10 00:54

reporter   ~0002160

Last edited: 2012-01-10 00:56

 Apache/2.2.21 (OS/2) PHP/5.2.14

regularity error message in POPUPLOG.os2

12-30-2011 14:06:01 SYS3175 PID 7776 TID 0009 Slot 00ef
D:\APPS\APACHE2\BIN\HTTPD.EXE
c0000005
1ffcc354
P1=00000001 P2=00000b14 P3=XXXXXXXX P4=XXXXXXXX
EAX=00000b14 EBX=20039ac0 ECX=027aff70 EDX=027affd4
ESI=027affd4 EDI=00000009
DS=0053 DSACC=d0f3 DSLIM=7fffffff
ES=0053 ESACC=d0f3 ESLIM=7fffffff
FS=150b FSACC=00f3 FSLIM=00000030
GS=0000 GSACC=**** GSLIM=********
CS:EIP=005b:1ffcc354 CSACC=d0df CSLIM=7fffffff
SS:ESP=0053:027aff94 SSACC=d0f3 SSLIM=7fffffff
EBP=027aff94 FLG=00010206

DOSCALL1.DLL 0002:0000c354

LewisR

2012-01-10 02:56

developer   ~0002161

One of us is going to have to bite the bullet and get a process dump for analysis. At the moment, I'm tied up with a backup issue, and won't be able to get to it for another couple of weeks.

Meanwhile, are we all running PHP as module? The last I tried running as CGI, I could not get Apache to start. I ention PHP, as we may indeed be looking in the wrong place for the cause of this crash.

I have the following modules loaded in my config:

core
mpmt_os2
http_core
mod_so
mod_actions
mod_alias
mod_asis
mod_authn_file
mod_authz_host
mod_authz_user
mod_auth_basic
mod_autoindex
mod_cgi
mod_dir
mod_env
mod_headers
mod_include
mod_log_config
mod_mime
mod_negotiation
mod_proxym
od_proxy_ajp
mod_proxy_balancer
mod_proxy_connect
mod_proxy_ftp
mod_proxy_http
mod_rewrite
mod_setenvif
mod_status
mod_userdir
mod_php5

(The above list was taken from phpinfo().) I am running Apache 2.2.21 / PHP 5.2.11.

psmedley

2012-10-02 00:24

administrator   ~0002318

Please update to v2.2.23 and install exceptq.dll

LewisR

2012-10-02 02:03

developer   ~0002319

I can't believe I haven't gotten farther into this. Thanks for the new builds, Paul. So far, 2.2.23 & PHP 5.2.17 with high mem support seem to be running well (I had on false alarm which turned out to be a browser issue, but caused me to restart Apache in the interim).

I'm set to make the switch to the exceptq-enabled 2.2.23 later tonight (I see people working on some sites over here, and don't want to bump them).

psmedley

2012-10-02 03:48

administrator   ~0002320

It may also make sense for PHP to be built with the exceptq enabled APR/pthread libraries - let me rebuild PHP 5.2.17

The 5.3.17 & 5.4.7 builds I added yesterday already include this, I just ran out of time for 5.2.17

psmedley

2012-10-02 03:57

administrator   ~0002321

http://smedley.id.au/php-5.2.17-os2-20121002.zip for latest PHP 5.2.17 with highmem used for modphp, and exceptq enabled APR/pthreads

LewisR

2012-10-03 00:25

developer   ~0002322

Honestly, I was going to ask whether PHP should be exceptq-enabled, but thought again that as it was running as a module, it shouldn't be necessary. I see your mention of enabling APR/pthreads.

Apache 2.2.23 (httpd-2.2.23-os2-20120928.zip) didn't like this latest PHP. It was fine with the non-exceptq build of 5.2.17, however, and is currently running it. TRP file to be attached, next (you know, there's an open bug at Mantis for adding the ability - D'Oh! - to upload an attachment *while* posting a note...now, why nobody thought of that little "convenience" before, I'll never know). ;-)

LewisR

2012-10-03 00:35

developer   ~0002323

At least we know exceptq is working on the box.

To correlate the TRP file against the Apache error log, I have this:

Creating 4B80_01.TRP

Killed by SIGSEGV
pid=0x4b80 ppid=0x4b7f tid=0x0001 slot=0x011c pri=0x0200 mc=0x0001
J:\APPS\APACHE2\BIN\HTTPD.EXE
PHP5 0:00003613
cs:eip=005b:1d063613 ss:esp=0053:0022fa10 ebp=0022fa48
 ds=0053 es=0053 fs=150b gs=0000 efl=00010206
eax=00dd73ee ebx=1d0d0070 ecx=4b800002 edx=00000000 edi=200373e8 esi=002304bd
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.
Creating 4B81_01.TRP

(etc.; got a few of them before I killed Apache and rolled back PHP versions).

psmedley

2012-10-03 20:36

administrator   ~0002325

This build works fine for Massimo. You're sure you don't have a stray php5.dll in the libpath?

LewisR

2012-10-04 00:55

developer   ~0002326

Red herring...

I hadn't checked the module inventory against what I was loading in php.ini. Once I commented out the dlls which weren't present in the modules directory, all went well. Running now.

While I haven't copied any modules over from the previous 5.2.17, Paul, they *seem* to be identical, so I should be able to just fill in the missing one(s) (pdf.dll is the one I really need) from the non-exceptq 5.2.17 package, correct?

psmedley

2012-10-04 01:04

administrator   ~0002327

Now that you have it going, can you switch to http://smedley.id.au/php-5.2.17-os2-20121003.zip

Should be identical, other than i included php5.xqs and mysql.xqs that afaik should be placed in the same directory as httpd.dll

It _seems_ the crash may be coming from php's mysql.dll

The modules most likely are identical - as the threading code comes from php5.dll, so there shouldn't be an exceptq-ness to them :)

ie copying them over should do no harm

LewisR

2012-10-04 01:52

developer   ~0002328

:-)

Done, and pdf module copied.

FWIW, I'm loading *both* mysql.dll and mqsqli.dll (I have *no* pdo modules loaded).

Thanks, Paul.

LewisR

2012-10-06 15:51

developer   ~0002329

No trp files. Running since my last post (2012-10-04 11:22), and the usual entries collected in Apache error log and POPUPLOG.OS2.

LewisR

2012-10-07 23:20

developer   ~0002330

Apologies. the four trp files I just uploaded may also be red herrings (I seem to be having a run on those herrings, these days).

I restarted httpd at 16:33 via Ctrl-C, and again at 16:36 (via Ctrl-C). This was sometime after applying http://smedley.id.au/php-5.2.17-dll-20121007.zip.

So, the good news is that whereas stopping httpd before did not yield trp files, now it does.

LewisR

2012-10-07 23:50

developer   ~0002331

Paul, just re-reading notes here, in comment 2327, you said:

>i included php5.xqs and mysql.xqs that afaik should be placed in the same directory as httpd.dll

I neglected to copy these out of \php5\ into \apache2\bin\. I'll do that now. Sorry for not reading more carefully.

psmedley

2013-10-10 23:00

administrator   ~0002543

I believe this ticket can be closed based on latest builds?

Igor Vaskov

2013-10-11 06:56

reporter   ~0002551

Yes. Now all works correctly.

psmedley

2013-10-11 11:02

administrator   ~0002558

resolved by current public builds built with gcc 4.7

Issue History

Date Modified Username Field Change
2010-05-28 19:15 Igor Vaskov New Issue
2010-05-28 19:15 Igor Vaskov File Added: POPUPLOG.OS2
2010-07-11 03:12 psmedley Note Added: 0001637
2010-07-11 03:12 psmedley Assigned To => psmedley
2010-07-11 03:12 psmedley Status new => feedback
2011-03-20 13:36 Igor Vaskov Note Added: 0001865
2011-03-20 13:36 Igor Vaskov Status feedback => assigned
2011-12-04 12:11 Igor Vaskov Note Added: 0002065
2011-12-31 08:25 psmedley Relationship added has duplicate 0000355
2011-12-31 08:27 psmedley Relationship added has duplicate 0000299
2011-12-31 20:57 LewisR Note Added: 0002116
2011-12-31 21:04 LewisR Note Added: 0002117
2012-01-02 11:30 Igor Vaskov Note Added: 0002125
2012-01-02 13:53 LewisR Note Added: 0002126
2012-01-03 17:43 Igor Vaskov Note Added: 0002132
2012-01-10 00:54 Bachinskiy Note Added: 0002160
2012-01-10 00:56 Bachinskiy Note Edited: 0002160
2012-01-10 02:56 LewisR Note Added: 0002161
2012-10-02 00:24 psmedley Note Added: 0002318
2012-10-02 02:03 LewisR Note Added: 0002319
2012-10-02 03:48 psmedley Note Added: 0002320
2012-10-02 03:57 psmedley Note Added: 0002321
2012-10-03 00:25 LewisR Note Added: 0002322
2012-10-03 00:26 LewisR File Added: 4B80_01.TRP
2012-10-03 00:35 LewisR Note Added: 0002323
2012-10-03 20:36 psmedley Note Added: 0002325
2012-10-04 00:55 LewisR Note Added: 0002326
2012-10-04 01:04 psmedley Note Added: 0002327
2012-10-04 01:52 LewisR Note Added: 0002328
2012-10-06 15:51 LewisR Note Added: 0002329
2012-10-07 23:12 LewisR File Added: 4E26_01.TRP
2012-10-07 23:12 LewisR File Added: 4E27_01.TRP
2012-10-07 23:13 LewisR File Added: 4E2C_01.TRP
2012-10-07 23:14 LewisR File Added: 4E2D_01.TRP
2012-10-07 23:20 LewisR Note Added: 0002330
2012-10-07 23:50 LewisR Note Added: 0002331
2013-10-10 23:00 psmedley Note Added: 0002543
2013-10-11 06:56 Igor Vaskov Note Added: 0002551
2013-10-11 11:02 psmedley Note Added: 0002558
2013-10-11 11:02 psmedley Status assigned => resolved
2013-10-11 11:02 psmedley Resolution open => fixed
2020-08-24 03:11 psmedley Status resolved => closed