0000637: Apache 2.2.29 / PHP 5.4.34 issues - server: eCSVirtual

ID	Project	Category	View Status	Date Submitted	Last Update

0000637	Apache 2.x	Bug	public	2014-12-20 08:59	2022-05-25 04:06

Reporter	LewisR	Assigned To	LewisR
Priority	normal	Severity	major	Reproducibility	sometimes
Status	closed	Resolution	suspended

Summary	0000637: Apache 2.2.29 / PHP 5.4.34 issues - server: eCSVirtual
Description	This is a metabug for issues discovered/investigated running Apache 2.2.29 / PHP 5.4.34 under VirtualBox eCS 2.1 VM (eCSVirtual) hosted on openSUSE 13.1 x64.
Additional Information	This chronicles progress which started with an email thread dated 10/30/2014: Well, as we've been going through another round of "where's the server?" over here, I spent the entire day yesterday working with Apache in a VM. I can replicate the stall condition, and I can even make the VM lock up hard and even reboot. All of this can be rather easily controlled via manipulating the MPM settings. Some examples: Set the threadstacksize too low (65535), and Apache will blow the stack and restart itself. This isn't an immediately-fatal problem for the server, of course, but there is a time penalty for Apache to restart. To get an idea of the time it takes for Apache to get back on its feet, run sid2 -ig a few times while Apache is starting up; you can watch the thread count rise as the daemon is initializing. Also, after a few rounds of this, the server becomes unstable and usually reboots. Set the threadstacksize too high (over 524280, for example), and the server can - and will - consume all RAM and either hang or reboot. Set the minsparethreads too low (10), and the performance is dreadful, as we have to keep spawning new threads as the server gets busy. Too high, and we can run out of threads (need to watch the arithmetic, mindful of CONFIG.SYS limit, minsparethreads, maxsparethreads, and startservers - threads are per-server). Siege tells us that with PHP not loaded (as module or CGI), we get 100% hits (using three parallel sieges of 200 semi-random urls - i.e., three sets of urls from three different vhosts, all running PHP-based sites). The expected response, of course, is Apache advising that the script could not be executed. However, the net effect on Apache is that it handles the request and returns a result - every time. Apache does consume some memory during this exercise, but not nearly as much as when PHP is loaded (as would be expected). Loading PHP as module results in hits approximately 30-35% of the time, using the same URLs against the same VM, with all other Apache configuration the same. The important thing to note here is that this same hit % is seen with and without the MySQL module loaded. I was going to blame slow MySQL responses, until I saw that the response times in the siege log were nearly identical. PHP should have simply returned an error that the MySQL module was not loaded, and this should have been very quick. Still, this is not the crux of the resource usage issue. Running as module, it is not uncommon to see Apache (and modules) continue to consume (consume, not just "not release") RAM after the siege is over, and the server should be idle. More on this, below. The PHP docs recommend a pre-fork server vs threaded when running PHP as module: http://php.net/manual/en/faq.installation.php#faq.installation.apache2 So, I reconfigured for CGI... Siege response is a little less than it is when running PHP as module (20-25%), however, what this did reveal was the following: 1. Apache does not get busy. 2. Apache uses only slightly more RAM than when sitting idle. 3. Spawned PHP.EXE processes either take a long time to exit or do not exit at all, and continue to consume more memory as they linger. So, run a siege with 200 clients, and a mess of PHP sessions are spawned. Unfortunately, even after the siege is over, they never go away. The majority of them may be killed manually (I had about a dozen left over which would not kill). Unfortunately, running a second siege compounds the problem, leaving us with another batch of PHP.EXEs running. Apache becomes unkillable, and memory continues to be consumed. I started with about 200MB RAM in use in a 2GB VM with a 2MB swap. About two minutes after the siege ended, I was using about 1GB (not unusual), but then it steadily grew until I had consumed all of the 2GB and the swap file grew to almost 300MB before I started killing off PHP sessions. Ultimately, I got back down to about 300MB in use, but the last dozen PHP sessions would not kill and neither would Apache (though I was able to remove it from the exit list). I confirmed this same behavior on the 2014-06-10 builds of Apache 2.2.27 and PHP 5.3.28. And it is not the MySQL module (or any other dynamic module), as I was able to replicate the condition with all modules commented out of php.ini. (That doesn't mean that none of the modules leak, of course; it just means that this issue is not due to a leaky dynamic module.) My conclusion at this point is that PHP is indeed the culprit and not Apache (threaded or forked, I think we're going to see the same issue until we get PHP under control). Tomorrow morning (10:40 EDT), we have an appointment with the oncologist to go over the lab results and the ultrasound. After that, my plan is to come back here, confirm this same behavior (or not) running PHP 5.2.17, then roll forward to the latest builds of both again to get some procdumps of a few of the PHP sessions and the stuck Apache session (the children exit when httpd -k restart is called, but not the parent), and I'll follow up with an ftp link to the files for us to share. Steve, I'll be sure to zip up the conf directory tree for you, as well. It's also interesting to bear in mind that this is a VM configured for one CPU, so while the SMP kernel with ACPI is being loaded (ACPI 3.22.03), there is still only one CPU in use.
Tags	No tags attached.
Attached Files	testing-http-php-with-os2trace-20141108.txt (8,431 bytes) How to use Dave Blaschke's os2trace to test apache httpd and php (cgi) 2014-11-07 SHL Baseline 2014-11-07 SHL Rework to use alternate ServerRoot directory. 2014-11-08 SHL Document logging level 2014-11-08 SHL Preset DOSCALLS APIs selections. == Prerequisites == - 4OS2 - Dave Blaschke's os2trace package. Available from http://www.angelfire.com/tx4/blaschke/OS2Trace/ - My os2traceenv.cmd script. Available from www.warpcave.com/betas - A working apache2 setup. - A working php CGI setup. == Files that will be modified == Os2trace modifies executables to insert forwarders to the tracing routines. The following files will need to be modified httpd.dll php.exe (cgi version) php5.dll libc066.dll == Install os2traceenv.cmd script == - Install os2traceenv.cmd to a directory in the PATH. == Install os2trace == - Create a directory for os2trace and unzip the distribution. I use d:\devtools\os2trace, but you can use a directory of your choice. == Setup a mirror of your production apache2/php installation == - Mirror your apache2 ServerRoot tree to a work directory. This makes setup simplifier and leaves the production apache's setup unsullied. I used d:\Internet\apache22-trc as the root, but you can use a directory of your choice. - Copy php5.dll to the mirror's bin directory. This makes BEGINLIBPATH setup simpler. - If your production cgi-bin is not apache2\cgi-bin, create a cgi-bin directory in the mirror and copy content of the production cgi-bin to the mirror's cgi-bin. - Edit the mirror's httpd.conf to match the new directory locations. Update ServerRoot to point to the mirror. Update ScriptAlias to point to the mirror's cgi-bin. Update absolute paths that should not point at production directories. Update log paths to leave the production logs unsullied. Update Directory sections as needed. Etc. - Copy libc066 DLLs to bin directory. This makes BEGINLIBPATH setup simpler. - Open a 4OS2 VIO session - Change to your os2trace directory and run - Start apache from the bin directory with httpd -d.. to test your edits. - Apache should start a serve content. - Stop apache. - Do not close the VIO session. You will continue to need it. == Modify files for os2trace == - Change to your os2trace directory and run os2traceenv to set up the os2trace environment. os2traceenv updates PATH and BEGINLIBPATH. - Change back to your mirror's bin directory - Run pmos2trc - Set the logging level to level 3 From the Logging Level dialog, select the Log API parameter contents (level 3) information item. Click OK to confirm the settings. The settings are stored in os2.ini in the os2trace application and will persist until changed. - Customize the DOSCALLS APIs selections. Use the Customize -> DOSCALLS APIs menu option to tune the settings. From the DOSCALLS APIs dialog, turn on FILE PIPE and PROC and turn off all the other selections. Click OK to confirm the settings. The settings are stored in os2.ini in the os2trace application and will persist until changed. - Enable tracing for httpd.dll php.exe (cgi version) php5.dll libc066.dll For each file, use the Enable -> Open file menu option to select the file to modify. From the Enablement dialog turn on DOSCALLS and disable all others Click OK to confirm the settings. - After all the files have been enabled for tracing, close pmos2trc leaving the VIO session open. == Adjust PATHs == - Edit BEGINLIBPATH and the mirror's bin directory so that apache2 and php can find the modified php5.dll and the modified libc066.dll when running LIBPATHSTRICT. - On my setup this results in BEGINLIBPATH=d:\internet\apache22-trc\bin;d:\devtools\os2trace; - Run set LIBPATHSTRICT=T to ensure that the modified libc066.dll is used. == Start apache and verify os2trace operation == - Start apache from the mirror's bin directory with httpd -d.. - If all goes well, apache2 will start and os2trace will report that it generated unique .trc file names for the 2nd and 3rd httpd.exe instances. - Try a simple php test. - Verify that os2trace creates a php.trc file in the mirror's cgi-bin directory. - Review the content of the generated .trc files. You should be able to see the pipe IO and file IO operations. You should also be able to see the CGI data that was read and written. == Testing with siege == - When running siege, os2trace will generate unique .trc file names for secondary php.exe instances. The primary php.trc file will contain reports for the php.exe instances that happened to be started when no other php.exe instances were active. - The size of the .trc files will depend on the amount of IO done by the PHP code. - It is recommended that you turn off as much logging as possible. This will reduce the amount of IO logged to the .trc files. The siege test should not need any CustomLogs and LogLevel Error should be sufficient for our purposes. - The httpd.trc file can get large, as can the php.trc file because they will contain the output for multiple siege transactions. - A siege run will create a large number of php .trc files unless the siege dies quickly. Each GET request for PHP content will spawn a new PHP instance. - It's probably a good idea to stop httpd between siege runs and clean up any left over .trc files. If a siege run fails, the .trc files should be moved to uniquely name archive directory for post-mortem analysis. Otherwise, the .trc files can be deleted. == Post mortem analysis == - Once a siege run produces some hung php.exe instances, we are ready to analyze. - Find the php .trc files that are missing the final "stopping at" message. These correspond to the hung instances. Verify that the PIDs match what pstat /c \| grep PHP reports. - For each hung php instance, examine the httpd .trc files and find the Dos32ExecPgm invocation that started this php instances. - Note the PID/TID combo and search for API failures or unexpected operations on this thread. - more later ... ============================================================ ==== Old setup method using saved copies of executables === ============================================================ == Save copies to files that will be modified by os2trace == - You will be instrumenting httpd.dll php.exe (cgi version) php5.dll libc066.dll os2trace can remove the modification, but I prefer to restore saved copies == Modify files for os2trace == - Open a 4OS2 VIO session - Change to your os2trace directory and run os2traceenv to set up an os2trace environment. os2traceenv updates PATH and BEGINLIBPATH. - Run pmos2trc and enable tracing for httpd.dll php.exe (cgi version) php5.dll libc066.dll Use Enable -> Open file to select the file to modify In the Enablement dialog turn on DOSCALLS and disable all others Click OK to confirm the settings. Use Customize -> DOSCALLS APIs to tune the settings In the DOSCALLS APIs dialog, turn on FILE PIPE PROC and turn off all the other selections. Click OK to confirm the settings. - Close pmos2trc and leave the VIO session open. == Prepare to start httpd == - Change to your apache2 bin directory. - Edit BEGINLIBPATH so that apache2 and php can find the modified libc DLLs. - Edit BEGINLIBPATH (if needed) so that php.exe can find php5.dll. - On my setup this results in BEGINLIBPATH=d:\internet\0;D:\Internet\php5;d:\devtools\os2trace; - Run set LIBPATHSTRICT=T to ensure that the modified libc DLLs are used. -- This email was Anti Virus checked by Astaro Security Gateway. http://www.astaro.com testing-http-php-with-os2trace-20141108.txt (8,431 bytes) readwrite.c.diff (11,703 bytes) Index: readwrite.c =================================================================== --- readwrite.c (revision 1629784) +++ readwrite.c (working copy) @@ -24,20 +24,35 @@ #include <malloc.h> -APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t nbytes) +APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t len) { ULONG rc = 0; - ULONG bytesread; + ULONG bytes_read = 0; - if (!thefile->isopen) { - nbytes = 0; - return APR_EBADF; + if (len <= 0) { + len = 0; + return APR_SUCCESS; } +#if 1 + / Handle the ungetchar if there is one / + / copied from win32/readwrite.c / + if (thefile->ungetchar != -1) { + bytes_read = 1; + (char )buf = (char)thefile->ungetchar; + buf = (char )buf + 1; + (len)--; + thefile->ungetchar = -1; + if (len == 0) { + len = bytes_read; + return APR_SUCCESS; + } + } +#endif if (thefile->buffered) { char pos = (char )buf; ULONG blocksize; - ULONG size = nbytes; + ULONG size = len; apr_thread_mutex_lock(thefile->mutex); @@ -56,20 +71,20 @@ while (rc == 0 && size > 0) { if (thefile->bufpos >= thefile->dataRead) { - ULONG bytesread; + ULONG bytes_read; rc = DosRead(thefile->filedes, thefile->buffer, - thefile->bufsize, &bytesread); - - if (bytesread == 0) { + thefile->bufsize, &bytes_read); + if (bytes_read == 0) { if (rc == 0) thefile->eof_hit = TRUE; break; } - - thefile->dataRead = bytesread; + else { + thefile->dataRead = bytes_read; thefile->filePtr += thefile->dataRead; thefile->bufpos = 0; } + } blocksize = size > thefile->dataRead - thefile->bufpos ? thefile->dataRead - thefile->bufpos : size; memcpy(pos, thefile->buffer + thefile->bufpos, blocksize); @@ -78,40 +93,60 @@ size -= blocksize; } - nbytes = rc == 0 ? pos - (char )buf : 0; + len = rc == 0 ? pos - (char )buf : 0; apr_thread_mutex_unlock(thefile->mutex); - if (nbytes == 0 && rc == 0 && thefile->eof_hit) { + if (len == 0 && rc == 0 && thefile->eof_hit) { return APR_EOF; } return APR_FROM_OS_ERROR(rc); } else { + / Unbuffered i/o / + apr_size_t nbytes; + +#if 1 // 2014-11-11 SHL + for (;;) { + int post_count; + rc = DosRead(thefile->filedes, buf, len, &nbytes); + if (!thefile->pipe \|\| rc != ERROR_NO_DATA \|\| thefile->timeout == 0) + break; + rc = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); + DosResetEventSem(thefile->pipeSem, &post_count); + if (rc == ERROR_TIMEOUT) { + len = 0; + return APR_TIMEUP; + } + } // for +#endif + +#if 0 // Old way if (thefile->pipe) DosResetEventSem(thefile->pipeSem, &rc); - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); + rc = DosRead(thefile->filedes, buf, len, &nbytes); if (rc == ERROR_NO_DATA && thefile->timeout != 0) { int rcwait = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); if (rcwait == 0) { - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); + rc = DosRead(thefile->filedes, buf, len, &nbytes); } else if (rcwait == ERROR_TIMEOUT) { - nbytes = 0; + len = 0; return APR_TIMEUP; } } +#endif if (rc) { - nbytes = 0; + len = 0; return APR_FROM_OS_ERROR(rc); } - nbytes = bytesread; + len = nbytes; - if (bytesread == 0) { + if (nbytes == 0) { thefile->eof_hit = TRUE; return APR_EOF; } @@ -126,17 +161,14 @@ { ULONG rc = 0; ULONG byteswritten; - if (!thefile->isopen) { nbytes = 0; return APR_EBADF; } - if (thefile->buffered) { char pos = (char )buf; int blocksize; int size = nbytes; - apr_thread_mutex_lock(thefile->mutex); if ( thefile->direction == 0 ) { @@ -147,19 +179,18 @@ thefile->bufpos = thefile->dataRead = 0; thefile->direction = 1; } - while (rc == 0 && size > 0) { - if (thefile->bufpos == thefile->bufsize) / write buffer is full / + if (thefile->bufpos == thefile->bufsize){ / write buffer is full / / XXX bug; - rc is double-transformed os->apr below / rc = apr_file_flush(thefile); - - blocksize = size > thefile->bufsize - thefile->bufpos ? thefile->bufsize - thefile->bufpos : size; + } + blocksize = size > thefile->bufsize - thefile->bufpos ? + thefile->bufsize - thefile->bufpos : size; memcpy(thefile->buffer + thefile->bufpos, pos, blocksize); thefile->bufpos += blocksize; pos += blocksize; size -= blocksize; } - apr_thread_mutex_unlock(thefile->mutex); return APR_FROM_OS_ERROR(rc); } else { @@ -167,20 +198,16 @@ FILELOCK all = { 0, 0x7fffffff }; ULONG newpos; rc = DosSetFileLocks(thefile->filedes, NULL, &all, -1, 0); - if (rc == 0) { rc = DosSetFilePtr(thefile->filedes, 0, FILE_END, &newpos); - if (rc == 0) { - rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); + rc = DosWrite(thefile->filedes, (PVOID)buf, nbytes, &byteswritten); } - DosSetFileLocks(thefile->filedes, &all, NULL, -1, 0); } } else { rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); } - if (rc) { nbytes = 0; return APR_FROM_OS_ERROR(rc); @@ -193,94 +220,84 @@ -#ifdef HAVE_WRITEV - APR_DECLARE(apr_status_t) apr_file_writev(apr_file_t thefile, const struct iovec vec, apr_size_t nvec, apr_size_t nbytes) { - int bytes; + apr_status_t rv = APR_SUCCESS; + apr_size_t i; + apr_size_t bwrote = 0; + char buf; - if (thefile->buffered) { - apr_status_t rv = apr_file_flush(thefile); + nbytes = 0; + for (i = 0; i < nvec; i++) { + buf = vec[i].iov_base; + bwrote = vec[i].iov_len; + rv = apr_file_write(thefile, buf, &bwrote); + nbytes += bwrote; if (rv != APR_SUCCESS) { - return rv; + break; } } - - if ((bytes = writev(thefile->filedes, vec, nvec)) < 0) { - nbytes = 0; - return errno; + return rv; } - else { - nbytes = bytes; - return APR_SUCCESS; - } -} -#endif APR_DECLARE(apr_status_t) apr_file_putc(char ch, apr_file_t thefile) { - ULONG rc; - ULONG byteswritten; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_write(thefile, &ch, &nbytes); } - rc = DosWrite(thefile->filedes, &ch, 1, &byteswritten); - if (rc) { - return APR_FROM_OS_ERROR(rc); - } - return APR_SUCCESS; -} - - - APR_DECLARE(apr_status_t) apr_file_ungetc(char ch, apr_file_t thefile) { - apr_off_t offset = -1; - return apr_file_seek(thefile, APR_CUR, &offset); + thefile->ungetchar = (unsigned char)ch; + return APR_SUCCESS; } APR_DECLARE(apr_status_t) apr_file_getc(char ch, apr_file_t thefile) { - ULONG rc; - apr_size_t bytesread; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_read(thefile, ch, &nbytes); + } - bytesread = 1; - rc = apr_file_read(thefile, ch, &bytesread); - if (rc) { - return rc; - } - if (bytesread == 0) { - thefile->eof_hit = TRUE; - return APR_EOF; +APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +{ + return apr_file_write_full(thefile, str, strlen(str), NULL); } - return APR_SUCCESS; -} - -APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +apr_status_t apr_file_flush_locked(apr_file_t thefile) { - apr_size_t len; + apr_status_t rv = APR_SUCCESS; - len = strlen(str); - return apr_file_write(thefile, str, &len); + if (thefile->direction == 1 && thefile->bufpos) { + apr_ssize_t written; + + do { + written = write(thefile->filedes, thefile->buffer, thefile->bufpos); + } while (written == -1 && errno == EINTR); + if (written == -1) { + rv = errno; + } else { + thefile->filePtr += written; + thefile->bufpos = 0; } + } + return rv; +} + + APR_DECLARE(apr_status_t) apr_file_flush(apr_file_t thefile) { if (thefile->buffered) { @@ -349,10 +366,61 @@ } +/ Pull from unix code to start a sync up / +#if (defined(__INNOTEK_LIBC__) \|\| defined(__WATCOMC__) ) +struct apr_file_printf_data { + apr_vformatter_buff_t vbuff; + apr_file_t fptr; + char buf; +}; + +static int file_printf_flush(apr_vformatter_buff_t buff) +{ + struct apr_file_printf_data data = (struct apr_file_printf_data )buff; + + if (apr_file_write_full(data->fptr, data->buf, + data->vbuff.curpos - data->buf, NULL)) { + return -1; + } + + data->vbuff.curpos = data->buf; + return 0; +} + APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, const char format, ...) { + struct apr_file_printf_data data; + va_list ap; + int count; + + /* don't really need a HUGE_STRING_LEN anymore / + data.buf = _lmalloc(HUGE_STRING_LEN); + if (data.buf == NULL) { + return -1; + } + data.vbuff.curpos = data.buf; + data.vbuff.endpos = data.buf + HUGE_STRING_LEN; + data.fptr = fptr; + va_start(ap, format); + count = apr_vformatter(file_printf_flush, + (apr_vformatter_buff_t )&data, format, ap); + /* apr_vformatter does not call flush for the last bits / + if (count >= 0) file_printf_flush((apr_vformatter_buff_t )&data); + + va_end(ap); + + free(data.buf); + + return count; +} + +#else + +APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, + const char format, ...) +{ int cc; va_list ap; char buf; @@ -369,9 +437,9 @@ free(buf); return (cc == APR_SUCCESS) ? len : -1; } +#endif - apr_status_t apr_file_check_read(apr_file_t fd) { int rc; readwrite.c.diff (11,703 bytes) readwrite2.c.diff (11,753 bytes) Index: readwrite.c =================================================================== --- readwrite.c (revision 1629784) +++ readwrite.c (working copy) @@ -24,20 +24,35 @@ #include <malloc.h> -APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t nbytes) +APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t len) { ULONG rc = 0; - ULONG bytesread; + ULONG bytes_read = 0; - if (!thefile->isopen) { - nbytes = 0; - return APR_EBADF; + if (len <= 0) { + len = 0; + return APR_SUCCESS; } +#if 1 + / Handle the ungetchar if there is one / + / copied from win32/readwrite.c / + if (thefile->ungetchar != -1) { + bytes_read = 1; + (char )buf = (char)thefile->ungetchar; + buf = (char )buf + 1; + (len)--; + thefile->ungetchar = -1; + if (len == 0) { + len = bytes_read; + return APR_SUCCESS; + } + } +#endif if (thefile->buffered) { char pos = (char )buf; ULONG blocksize; - ULONG size = nbytes; + ULONG size = len; apr_thread_mutex_lock(thefile->mutex); @@ -56,20 +71,20 @@ while (rc == 0 && size > 0) { if (thefile->bufpos >= thefile->dataRead) { - ULONG bytesread; + ULONG bytes_read; rc = DosRead(thefile->filedes, thefile->buffer, - thefile->bufsize, &bytesread); - - if (bytesread == 0) { + thefile->bufsize, &bytes_read); + if (bytes_read == 0) { if (rc == 0) thefile->eof_hit = TRUE; break; } - - thefile->dataRead = bytesread; + else { + thefile->dataRead = bytes_read; thefile->filePtr += thefile->dataRead; thefile->bufpos = 0; } + } blocksize = size > thefile->dataRead - thefile->bufpos ? thefile->dataRead - thefile->bufpos : size; memcpy(pos, thefile->buffer + thefile->bufpos, blocksize); @@ -78,40 +93,74 @@ size -= blocksize; } - nbytes = rc == 0 ? pos - (char )buf : 0; + len = rc == 0 ? pos - (char )buf : 0; apr_thread_mutex_unlock(thefile->mutex); - if (nbytes == 0 && rc == 0 && thefile->eof_hit) { + if (len == 0 && rc == 0 && thefile->eof_hit) { return APR_EOF; } return APR_FROM_OS_ERROR(rc); } else { + / Unbuffered i/o / + apr_size_t nbytes; + +#if 1 // 2014-11-11 SHL + for (;;) { + int post_count; if (thefile->pipe) + DosResetEventSem(thefile->pipeSem, &post_count); + rc = DosRead(thefile->filedes, buf, len, &nbytes); + if (!thefile->pipe) + break; // Blocking i/o - we are done + if (rc == ERROR_MORE_DATA) { + // Buffer is full - caller will read more eventually + rc = NO_ERROR; + break; + } + if (thefile->timeout == 0) + break; + if (rc == NO_ERROR) { + if (nbytes != 0) + break; + } + else if (rc != ERROR_NO_DATA) + break; + rc = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); + if (rc == ERROR_TIMEOUT) { + len = 0; + return APR_TIMEUP; + } + } // for +#endif + +#if 0 // Old way + if (thefile->pipe) DosResetEventSem(thefile->pipeSem, &rc); - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); + rc = DosRead(thefile->filedes, buf, len, &nbytes); if (rc == ERROR_NO_DATA && thefile->timeout != 0) { int rcwait = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); if (rcwait == 0) { - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); + rc = DosRead(thefile->filedes, buf, len, &nbytes); } else if (rcwait == ERROR_TIMEOUT) { - nbytes = 0; + len = 0; return APR_TIMEUP; } } +#endif if (rc) { - nbytes = 0; + len = 0; return APR_FROM_OS_ERROR(rc); } - nbytes = bytesread; + len = nbytes; - if (bytesread == 0) { + if (nbytes == 0) { thefile->eof_hit = TRUE; return APR_EOF; } @@ -126,17 +175,14 @@ { ULONG rc = 0; ULONG byteswritten; - if (!thefile->isopen) { nbytes = 0; return APR_EBADF; } - if (thefile->buffered) { char pos = (char )buf; int blocksize; int size = nbytes; - apr_thread_mutex_lock(thefile->mutex); if ( thefile->direction == 0 ) { @@ -147,19 +193,18 @@ thefile->bufpos = thefile->dataRead = 0; thefile->direction = 1; } - while (rc == 0 && size > 0) { - if (thefile->bufpos == thefile->bufsize) / write buffer is full / + if (thefile->bufpos == thefile->bufsize){ / write buffer is full / / XXX bug; - rc is double-transformed os->apr below / rc = apr_file_flush(thefile); - - blocksize = size > thefile->bufsize - thefile->bufpos ? thefile->bufsize - thefile->bufpos : size; + } + blocksize = size > thefile->bufsize - thefile->bufpos ? + thefile->bufsize - thefile->bufpos : size; memcpy(thefile->buffer + thefile->bufpos, pos, blocksize); thefile->bufpos += blocksize; pos += blocksize; size -= blocksize; } - apr_thread_mutex_unlock(thefile->mutex); return APR_FROM_OS_ERROR(rc); } else { @@ -167,20 +212,16 @@ FILELOCK all = { 0, 0x7fffffff }; ULONG newpos; rc = DosSetFileLocks(thefile->filedes, NULL, &all, -1, 0); - if (rc == 0) { rc = DosSetFilePtr(thefile->filedes, 0, FILE_END, &newpos); - if (rc == 0) { - rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); + rc = DosWrite(thefile->filedes, (PVOID)buf, nbytes, &byteswritten); } - DosSetFileLocks(thefile->filedes, &all, NULL, -1, 0); } } else { rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); } - if (rc) { nbytes = 0; return APR_FROM_OS_ERROR(rc); @@ -193,94 +234,84 @@ -#ifdef HAVE_WRITEV - APR_DECLARE(apr_status_t) apr_file_writev(apr_file_t thefile, const struct iovec vec, apr_size_t nvec, apr_size_t nbytes) { - int bytes; + apr_status_t rv = APR_SUCCESS; + apr_size_t i; + apr_size_t bwrote = 0; + char buf; - if (thefile->buffered) { - apr_status_t rv = apr_file_flush(thefile); + nbytes = 0; + for (i = 0; i < nvec; i++) { + buf = vec[i].iov_base; + bwrote = vec[i].iov_len; + rv = apr_file_write(thefile, buf, &bwrote); + nbytes += bwrote; if (rv != APR_SUCCESS) { - return rv; + break; } } - - if ((bytes = writev(thefile->filedes, vec, nvec)) < 0) { - nbytes = 0; - return errno; + return rv; } - else { - nbytes = bytes; - return APR_SUCCESS; - } -} -#endif APR_DECLARE(apr_status_t) apr_file_putc(char ch, apr_file_t thefile) { - ULONG rc; - ULONG byteswritten; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_write(thefile, &ch, &nbytes); } - rc = DosWrite(thefile->filedes, &ch, 1, &byteswritten); - if (rc) { - return APR_FROM_OS_ERROR(rc); - } - return APR_SUCCESS; -} - - - APR_DECLARE(apr_status_t) apr_file_ungetc(char ch, apr_file_t thefile) { - apr_off_t offset = -1; - return apr_file_seek(thefile, APR_CUR, &offset); + thefile->ungetchar = (unsigned char)ch; + return APR_SUCCESS; } APR_DECLARE(apr_status_t) apr_file_getc(char ch, apr_file_t thefile) { - ULONG rc; - apr_size_t bytesread; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_read(thefile, ch, &nbytes); + } - bytesread = 1; - rc = apr_file_read(thefile, ch, &bytesread); - if (rc) { - return rc; - } - if (bytesread == 0) { - thefile->eof_hit = TRUE; - return APR_EOF; +APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +{ + return apr_file_write_full(thefile, str, strlen(str), NULL); } - return APR_SUCCESS; -} - -APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +apr_status_t apr_file_flush_locked(apr_file_t thefile) { - apr_size_t len; + apr_status_t rv = APR_SUCCESS; - len = strlen(str); - return apr_file_write(thefile, str, &len); + if (thefile->direction == 1 && thefile->bufpos) { + apr_ssize_t written; + + do { + written = write(thefile->filedes, thefile->buffer, thefile->bufpos); + } while (written == -1 && errno == EINTR); + if (written == -1) { + rv = errno; + } else { + thefile->filePtr += written; + thefile->bufpos = 0; } + } + return rv; +} + + APR_DECLARE(apr_status_t) apr_file_flush(apr_file_t thefile) { if (thefile->buffered) { @@ -349,10 +380,61 @@ } +/ Pull from unix code to start a sync up / +#if (defined(__INNOTEK_LIBC__) \|\| defined(__WATCOMC__) ) +struct apr_file_printf_data { + apr_vformatter_buff_t vbuff; + apr_file_t fptr; + char buf; +}; + +static int file_printf_flush(apr_vformatter_buff_t buff) +{ + struct apr_file_printf_data data = (struct apr_file_printf_data )buff; + + if (apr_file_write_full(data->fptr, data->buf, + data->vbuff.curpos - data->buf, NULL)) { + return -1; + } + + data->vbuff.curpos = data->buf; + return 0; +} + APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, const char format, ...) { + struct apr_file_printf_data data; + va_list ap; + int count; + + /* don't really need a HUGE_STRING_LEN anymore / + data.buf = _lmalloc(HUGE_STRING_LEN); + if (data.buf == NULL) { + return -1; + } + data.vbuff.curpos = data.buf; + data.vbuff.endpos = data.buf + HUGE_STRING_LEN; + data.fptr = fptr; + va_start(ap, format); + count = apr_vformatter(file_printf_flush, + (apr_vformatter_buff_t )&data, format, ap); + /* apr_vformatter does not call flush for the last bits / + if (count >= 0) file_printf_flush((apr_vformatter_buff_t )&data); + + va_end(ap); + + free(data.buf); + + return count; +} + +#else + +APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, + const char format, ...) +{ int cc; va_list ap; char buf; @@ -369,9 +451,9 @@ free(buf); return (cc == APR_SUCCESS) ? len : -1; } +#endif - apr_status_t apr_file_check_read(apr_file_t fd) { int rc; readwrite2.c.diff (11,753 bytes) readwrite3.c.diff (11,964 bytes) Index: readwrite.c =================================================================== --- readwrite.c (revision 1629784) +++ readwrite.c (working copy) @@ -24,20 +24,34 @@ #include <malloc.h> -APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t nbytes) +APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t len) { ULONG rc = 0; - ULONG bytesread; + ULONG bytes_read = 0; - if (!thefile->isopen) { - nbytes = 0; - return APR_EBADF; + if (len <= 0) { + len = 0; + return APR_SUCCESS; } - +#if 1 + / Handle the ungetchar if there is one / + / copied from win32/readwrite.c / + if (thefile->ungetchar != -1) { + bytes_read = 1; + (char )buf = (char)thefile->ungetchar; + buf = (char )buf + 1; + (len)--; + thefile->ungetchar = -1; + if (len == 0) { + len = bytes_read; + return APR_SUCCESS; + } + } +#endif if (thefile->buffered) { char pos = (char )buf; ULONG blocksize; - ULONG size = nbytes; + ULONG size = len; apr_thread_mutex_lock(thefile->mutex); @@ -56,20 +70,20 @@ while (rc == 0 && size > 0) { if (thefile->bufpos >= thefile->dataRead) { - ULONG bytesread; + ULONG bytes_read; rc = DosRead(thefile->filedes, thefile->buffer, - thefile->bufsize, &bytesread); - - if (bytesread == 0) { + thefile->bufsize, &bytes_read); + if (bytes_read == 0) { if (rc == 0) thefile->eof_hit = TRUE; break; } - - thefile->dataRead = bytesread; + else { + thefile->dataRead = bytes_read; thefile->filePtr += thefile->dataRead; thefile->bufpos = 0; } + } blocksize = size > thefile->dataRead - thefile->bufpos ? thefile->dataRead - thefile->bufpos : size; memcpy(pos, thefile->buffer + thefile->bufpos, blocksize); @@ -78,40 +92,80 @@ size -= blocksize; } - nbytes = rc == 0 ? pos - (char )buf : 0; + len = rc == 0 ? pos - (char )buf : 0; apr_thread_mutex_unlock(thefile->mutex); - if (nbytes == 0 && rc == 0 && thefile->eof_hit) { + if (len == 0 && rc == 0 && thefile->eof_hit) { return APR_EOF; } return APR_FROM_OS_ERROR(rc); } else { + / Unbuffered i/o / + apr_size_t nbytes; + +#if 1 // 2014-11-11 SHL + for (;;) { + int post_count; if (thefile->pipe) + DosResetEventSem(thefile->pipeSem, &post_count); + rc = DosRead(thefile->filedes, buf, len, &nbytes); + if (!thefile->pipe) + break; // Blocking i/o - we are done + if (rc == ERROR_MORE_DATA) { + // Buffer is full - caller will read more eventually + rc = NO_ERROR; + break; + } + if (thefile->timeout == 0) + break; +#if 0 // 2014-11-12 SHL assumes eos can return NO_ERROR or ERROR_NO_DATA + if (rc == NO_ERROR) { + if (nbytes != 0) + break; + } + else if (rc != ERROR_NO_DATA) + break; +#endif +#if 1 // 2014-11-12 SHL assumes eos can only return NO_ERROR and count 0 + if (rc != ERROR_NO_DATA) + break; +#endif + rc = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); + if (rc == ERROR_TIMEOUT) { + len = 0; + return APR_TIMEUP; + } + } // for +#endif + +#if 0 // Old way + if (thefile->pipe) DosResetEventSem(thefile->pipeSem, &rc); - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); + rc = DosRead(thefile->filedes, buf, len, &nbytes); if (rc == ERROR_NO_DATA && thefile->timeout != 0) { int rcwait = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); if (rcwait == 0) { - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); + rc = DosRead(thefile->filedes, buf, len, &nbytes); } else if (rcwait == ERROR_TIMEOUT) { - nbytes = 0; + len = 0; return APR_TIMEUP; } } +#endif if (rc) { - nbytes = 0; + len = 0; return APR_FROM_OS_ERROR(rc); } - nbytes = bytesread; + len = nbytes; - if (bytesread == 0) { + if (nbytes == 0) { thefile->eof_hit = TRUE; return APR_EOF; } @@ -126,17 +180,14 @@ { ULONG rc = 0; ULONG byteswritten; - if (!thefile->isopen) { nbytes = 0; return APR_EBADF; } - if (thefile->buffered) { char pos = (char )buf; int blocksize; int size = nbytes; - apr_thread_mutex_lock(thefile->mutex); if ( thefile->direction == 0 ) { @@ -147,19 +198,18 @@ thefile->bufpos = thefile->dataRead = 0; thefile->direction = 1; } - while (rc == 0 && size > 0) { - if (thefile->bufpos == thefile->bufsize) / write buffer is full / + if (thefile->bufpos == thefile->bufsize){ / write buffer is full / / XXX bug; - rc is double-transformed os->apr below / rc = apr_file_flush(thefile); - - blocksize = size > thefile->bufsize - thefile->bufpos ? thefile->bufsize - thefile->bufpos : size; + } + blocksize = size > thefile->bufsize - thefile->bufpos ? + thefile->bufsize - thefile->bufpos : size; memcpy(thefile->buffer + thefile->bufpos, pos, blocksize); thefile->bufpos += blocksize; pos += blocksize; size -= blocksize; } - apr_thread_mutex_unlock(thefile->mutex); return APR_FROM_OS_ERROR(rc); } else { @@ -167,20 +217,16 @@ FILELOCK all = { 0, 0x7fffffff }; ULONG newpos; rc = DosSetFileLocks(thefile->filedes, NULL, &all, -1, 0); - if (rc == 0) { rc = DosSetFilePtr(thefile->filedes, 0, FILE_END, &newpos); - if (rc == 0) { - rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); + rc = DosWrite(thefile->filedes, (PVOID)buf, nbytes, &byteswritten); } - DosSetFileLocks(thefile->filedes, &all, NULL, -1, 0); } } else { rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); } - if (rc) { nbytes = 0; return APR_FROM_OS_ERROR(rc); @@ -193,94 +239,84 @@ -#ifdef HAVE_WRITEV - APR_DECLARE(apr_status_t) apr_file_writev(apr_file_t thefile, const struct iovec vec, apr_size_t nvec, apr_size_t nbytes) { - int bytes; + apr_status_t rv = APR_SUCCESS; + apr_size_t i; + apr_size_t bwrote = 0; + char buf; - if (thefile->buffered) { - apr_status_t rv = apr_file_flush(thefile); + nbytes = 0; + for (i = 0; i < nvec; i++) { + buf = vec[i].iov_base; + bwrote = vec[i].iov_len; + rv = apr_file_write(thefile, buf, &bwrote); + nbytes += bwrote; if (rv != APR_SUCCESS) { - return rv; + break; } } - - if ((bytes = writev(thefile->filedes, vec, nvec)) < 0) { - nbytes = 0; - return errno; + return rv; } - else { - nbytes = bytes; - return APR_SUCCESS; - } -} -#endif APR_DECLARE(apr_status_t) apr_file_putc(char ch, apr_file_t thefile) { - ULONG rc; - ULONG byteswritten; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_write(thefile, &ch, &nbytes); } - rc = DosWrite(thefile->filedes, &ch, 1, &byteswritten); - if (rc) { - return APR_FROM_OS_ERROR(rc); - } - return APR_SUCCESS; -} - - - APR_DECLARE(apr_status_t) apr_file_ungetc(char ch, apr_file_t thefile) { - apr_off_t offset = -1; - return apr_file_seek(thefile, APR_CUR, &offset); + thefile->ungetchar = (unsigned char)ch; + return APR_SUCCESS; } APR_DECLARE(apr_status_t) apr_file_getc(char ch, apr_file_t thefile) { - ULONG rc; - apr_size_t bytesread; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_read(thefile, ch, &nbytes); + } - bytesread = 1; - rc = apr_file_read(thefile, ch, &bytesread); - if (rc) { - return rc; - } - if (bytesread == 0) { - thefile->eof_hit = TRUE; - return APR_EOF; +APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +{ + return apr_file_write_full(thefile, str, strlen(str), NULL); } - return APR_SUCCESS; -} - -APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +apr_status_t apr_file_flush_locked(apr_file_t thefile) { - apr_size_t len; + apr_status_t rv = APR_SUCCESS; - len = strlen(str); - return apr_file_write(thefile, str, &len); + if (thefile->direction == 1 && thefile->bufpos) { + apr_ssize_t written; + + do { + written = write(thefile->filedes, thefile->buffer, thefile->bufpos); + } while (written == -1 && errno == EINTR); + if (written == -1) { + rv = errno; + } else { + thefile->filePtr += written; + thefile->bufpos = 0; } + } + return rv; +} + + APR_DECLARE(apr_status_t) apr_file_flush(apr_file_t thefile) { if (thefile->buffered) { @@ -349,10 +385,61 @@ } +/ Pull from unix code to start a sync up / +#if (defined(__INNOTEK_LIBC__) \|\| defined(__WATCOMC__) ) +struct apr_file_printf_data { + apr_vformatter_buff_t vbuff; + apr_file_t fptr; + char buf; +}; + +static int file_printf_flush(apr_vformatter_buff_t buff) +{ + struct apr_file_printf_data data = (struct apr_file_printf_data )buff; + + if (apr_file_write_full(data->fptr, data->buf, + data->vbuff.curpos - data->buf, NULL)) { + return -1; + } + + data->vbuff.curpos = data->buf; + return 0; +} + APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, const char format, ...) { + struct apr_file_printf_data data; + va_list ap; + int count; + + /* don't really need a HUGE_STRING_LEN anymore / + data.buf = _lmalloc(HUGE_STRING_LEN); + if (data.buf == NULL) { + return -1; + } + data.vbuff.curpos = data.buf; + data.vbuff.endpos = data.buf + HUGE_STRING_LEN; + data.fptr = fptr; + va_start(ap, format); + count = apr_vformatter(file_printf_flush, + (apr_vformatter_buff_t )&data, format, ap); + /* apr_vformatter does not call flush for the last bits / + if (count >= 0) file_printf_flush((apr_vformatter_buff_t )&data); + + va_end(ap); + + free(data.buf); + + return count; +} + +#else + +APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, + const char format, ...) +{ int cc; va_list ap; char buf; @@ -369,9 +456,9 @@ free(buf); return (cc == APR_SUCCESS) ? len : -1; } +#endif - apr_status_t apr_file_check_read(apr_file_t fd) { int rc; readwrite3.c.diff (11,964 bytes) readwrite4.c.diff (11,475 bytes) Index: readwrite.c =================================================================== --- readwrite.c (revision 1629784) +++ readwrite.c (working copy) @@ -24,20 +24,34 @@ #include <malloc.h> -APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t nbytes) +APR_DECLARE(apr_status_t) apr_file_read(apr_file_t thefile, void buf, apr_size_t len) { ULONG rc = 0; - ULONG bytesread; + ULONG bytes_read = 0; - if (!thefile->isopen) { - nbytes = 0; - return APR_EBADF; + if (len <= 0) { + len = 0; + return APR_SUCCESS; } - +#if 1 + / Handle the ungetchar if there is one / + / copied from win32/readwrite.c / + if (thefile->ungetchar != -1) { + bytes_read = 1; + (char )buf = (char)thefile->ungetchar; + buf = (char )buf + 1; + (len)--; + thefile->ungetchar = -1; + if (len == 0) { + len = bytes_read; + return APR_SUCCESS; + } + } +#endif if (thefile->buffered) { char pos = (char )buf; ULONG blocksize; - ULONG size = nbytes; + ULONG size = len; apr_thread_mutex_lock(thefile->mutex); @@ -56,20 +70,20 @@ while (rc == 0 && size > 0) { if (thefile->bufpos >= thefile->dataRead) { - ULONG bytesread; + ULONG bytes_read; rc = DosRead(thefile->filedes, thefile->buffer, - thefile->bufsize, &bytesread); - - if (bytesread == 0) { + thefile->bufsize, &bytes_read); + if (bytes_read == 0) { if (rc == 0) thefile->eof_hit = TRUE; break; } - - thefile->dataRead = bytesread; + else { + thefile->dataRead = bytes_read; thefile->filePtr += thefile->dataRead; thefile->bufpos = 0; } + } blocksize = size > thefile->dataRead - thefile->bufpos ? thefile->dataRead - thefile->bufpos : size; memcpy(pos, thefile->buffer + thefile->bufpos, blocksize); @@ -78,40 +92,50 @@ size -= blocksize; } - nbytes = rc == 0 ? pos - (char )buf : 0; + len = rc == 0 ? pos - (char )buf : 0; apr_thread_mutex_unlock(thefile->mutex); - if (nbytes == 0 && rc == 0 && thefile->eof_hit) { + if (len == 0 && rc == 0 && thefile->eof_hit) { return APR_EOF; } return APR_FROM_OS_ERROR(rc); } else { + / Unbuffered i/o / + apr_size_t nbytes; + + // 2014-11-11 SHL reworked to support ERROR_MORE_DATA etc. + for (;;) { + int post_count; if (thefile->pipe) - DosResetEventSem(thefile->pipeSem, &rc); - - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); - - if (rc == ERROR_NO_DATA && thefile->timeout != 0) { - int rcwait = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); - - if (rcwait == 0) { - rc = DosRead(thefile->filedes, buf, nbytes, &bytesread); + DosResetEventSem(thefile->pipeSem, &post_count); + rc = DosRead(thefile->filedes, buf, len, &nbytes); + if (!thefile->pipe) + break; // Blocking i/o - we are done + if (rc == ERROR_MORE_DATA) { + // Buffer is full - caller will read more eventually + rc = NO_ERROR; + break; } - else if (rcwait == ERROR_TIMEOUT) { - nbytes = 0; + if (thefile->timeout == 0) + break; + if (rc != ERROR_NO_DATA) + break; + rc = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); + if (rc == ERROR_TIMEOUT) { + len = 0; return APR_TIMEUP; } - } + } // for if (rc) { - nbytes = 0; + len = 0; return APR_FROM_OS_ERROR(rc); } - nbytes = bytesread; + len = nbytes; - if (bytesread == 0) { + if (nbytes == 0) { thefile->eof_hit = TRUE; return APR_EOF; } @@ -126,17 +150,14 @@ { ULONG rc = 0; ULONG byteswritten; - if (!thefile->isopen) { nbytes = 0; return APR_EBADF; } - if (thefile->buffered) { char pos = (char )buf; int blocksize; int size = nbytes; - apr_thread_mutex_lock(thefile->mutex); if ( thefile->direction == 0 ) { @@ -147,19 +168,18 @@ thefile->bufpos = thefile->dataRead = 0; thefile->direction = 1; } - while (rc == 0 && size > 0) { - if (thefile->bufpos == thefile->bufsize) /* write buffer is full / + if (thefile->bufpos == thefile->bufsize){ / write buffer is full / / XXX bug; - rc is double-transformed os->apr below / rc = apr_file_flush(thefile); - - blocksize = size > thefile->bufsize - thefile->bufpos ? thefile->bufsize - thefile->bufpos : size; + } + blocksize = size > thefile->bufsize - thefile->bufpos ? + thefile->bufsize - thefile->bufpos : size; memcpy(thefile->buffer + thefile->bufpos, pos, blocksize); thefile->bufpos += blocksize; pos += blocksize; size -= blocksize; } - apr_thread_mutex_unlock(thefile->mutex); return APR_FROM_OS_ERROR(rc); } else { @@ -167,20 +187,16 @@ FILELOCK all = { 0, 0x7fffffff }; ULONG newpos; rc = DosSetFileLocks(thefile->filedes, NULL, &all, -1, 0); - if (rc == 0) { rc = DosSetFilePtr(thefile->filedes, 0, FILE_END, &newpos); - if (rc == 0) { - rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); + rc = DosWrite(thefile->filedes, (PVOID)buf, nbytes, &byteswritten); } - DosSetFileLocks(thefile->filedes, &all, NULL, -1, 0); } } else { rc = DosWrite(thefile->filedes, buf, nbytes, &byteswritten); } - if (rc) { nbytes = 0; return APR_FROM_OS_ERROR(rc); @@ -193,94 +209,84 @@ -#ifdef HAVE_WRITEV - APR_DECLARE(apr_status_t) apr_file_writev(apr_file_t thefile, const struct iovec vec, apr_size_t nvec, apr_size_t nbytes) { - int bytes; + apr_status_t rv = APR_SUCCESS; + apr_size_t i; + apr_size_t bwrote = 0; + char buf; - if (thefile->buffered) { - apr_status_t rv = apr_file_flush(thefile); + nbytes = 0; + for (i = 0; i < nvec; i++) { + buf = vec[i].iov_base; + bwrote = vec[i].iov_len; + rv = apr_file_write(thefile, buf, &bwrote); + nbytes += bwrote; if (rv != APR_SUCCESS) { - return rv; + break; } } - - if ((bytes = writev(thefile->filedes, vec, nvec)) < 0) { - nbytes = 0; - return errno; + return rv; } - else { - nbytes = bytes; - return APR_SUCCESS; - } -} -#endif APR_DECLARE(apr_status_t) apr_file_putc(char ch, apr_file_t thefile) { - ULONG rc; - ULONG byteswritten; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_write(thefile, &ch, &nbytes); } - rc = DosWrite(thefile->filedes, &ch, 1, &byteswritten); - if (rc) { - return APR_FROM_OS_ERROR(rc); - } - return APR_SUCCESS; -} - - - APR_DECLARE(apr_status_t) apr_file_ungetc(char ch, apr_file_t thefile) { - apr_off_t offset = -1; - return apr_file_seek(thefile, APR_CUR, &offset); + thefile->ungetchar = (unsigned char)ch; + return APR_SUCCESS; } APR_DECLARE(apr_status_t) apr_file_getc(char ch, apr_file_t thefile) { - ULONG rc; - apr_size_t bytesread; + apr_size_t nbytes = 1; - if (!thefile->isopen) { - return APR_EBADF; + return apr_file_read(thefile, ch, &nbytes); + } - bytesread = 1; - rc = apr_file_read(thefile, ch, &bytesread); - if (rc) { - return rc; - } - if (bytesread == 0) { - thefile->eof_hit = TRUE; - return APR_EOF; +APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +{ + return apr_file_write_full(thefile, str, strlen(str), NULL); } - return APR_SUCCESS; -} - -APR_DECLARE(apr_status_t) apr_file_puts(const char str, apr_file_t thefile) +apr_status_t apr_file_flush_locked(apr_file_t thefile) { - apr_size_t len; + apr_status_t rv = APR_SUCCESS; - len = strlen(str); - return apr_file_write(thefile, str, &len); + if (thefile->direction == 1 && thefile->bufpos) { + apr_ssize_t written; + + do { + written = write(thefile->filedes, thefile->buffer, thefile->bufpos); + } while (written == -1 && errno == EINTR); + if (written == -1) { + rv = errno; + } else { + thefile->filePtr += written; + thefile->bufpos = 0; } + } + return rv; +} + + APR_DECLARE(apr_status_t) apr_file_flush(apr_file_t thefile) { if (thefile->buffered) { @@ -349,10 +355,61 @@ } +/ Pull from unix code to start a sync up / +#if (defined(__INNOTEK_LIBC__) \|\| defined(__WATCOMC__) ) +struct apr_file_printf_data { + apr_vformatter_buff_t vbuff; + apr_file_t fptr; + char buf; +}; + +static int file_printf_flush(apr_vformatter_buff_t buff) +{ + struct apr_file_printf_data data = (struct apr_file_printf_data )buff; + + if (apr_file_write_full(data->fptr, data->buf, + data->vbuff.curpos - data->buf, NULL)) { + return -1; + } + + data->vbuff.curpos = data->buf; + return 0; +} + APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, const char format, ...) { + struct apr_file_printf_data data; + va_list ap; + int count; + + /* don't really need a HUGE_STRING_LEN anymore / + data.buf = _lmalloc(HUGE_STRING_LEN); + if (data.buf == NULL) { + return -1; + } + data.vbuff.curpos = data.buf; + data.vbuff.endpos = data.buf + HUGE_STRING_LEN; + data.fptr = fptr; + va_start(ap, format); + count = apr_vformatter(file_printf_flush, + (apr_vformatter_buff_t )&data, format, ap); + /* apr_vformatter does not call flush for the last bits / + if (count >= 0) file_printf_flush((apr_vformatter_buff_t )&data); + + va_end(ap); + + free(data.buf); + + return count; +} + +#else + +APR_DECLARE_NONSTD(int) apr_file_printf(apr_file_t fptr, + const char format, ...) +{ int cc; va_list ap; char buf; @@ -369,9 +426,9 @@ free(buf); return (cc == APR_SUCCESS) ? len : -1; } +#endif - apr_status_t apr_file_check_read(apr_file_t fd) { int rc; readwrite4.c.diff (11,475 bytes) open.c.diff (4,577 bytes) Index: open.c =================================================================== --- open.c (revision 1629784) +++ open.c (working copy) @@ -28,14 +28,70 @@ return apr_file_close(file); } +#ifdef __INNOTEK_LIBC__ +static apr_status_t file_cleanup(apr_file_t file) +{ + apr_status_t rv = APR_SUCCESS; + if (close(file->filedes) == 0) { + file->filedes = -1; + if (file->flags & APR_DELONCLOSE) { + unlink(file->fname); + } +#if APR_HAS_THREADS + if (file->mutex) { + rv = apr_thread_mutex_destroy(file->mutex); + } +#endif + } + else { + / Are there any error conditions other than EINTR or EBADF? / + rv = errno; + } +#ifndef WAITIO_USES_POLL + if (file->pollset != NULL) { + int pollset_rv = apr_pollset_destroy(file->pollset); + / If the file close failed, return its error value, + * not apr_pollset_destroy()'s. + / + if (rv == APR_SUCCESS) { + rv = pollset_rv; + } + } +#endif / !WAITIO_USES_POLL / + return rv; +} +apr_status_t apr_unix_file_cleanup(void thefile) +{ + apr_file_t file = thefile; + apr_status_t flush_rv = APR_SUCCESS, rv = APR_SUCCESS; + + if (file->buffered) { + flush_rv = apr_file_flush(file); + } + + rv = file_cleanup(file); + + return rv != APR_SUCCESS ? rv : flush_rv; +} + +apr_status_t apr_unix_child_file_cleanup(void thefile) +{ + return file_cleanup(thefile); +} +#endif + + APR_DECLARE(apr_status_t) apr_file_open(apr_file_t *new, const char fname, apr_int32_t flag, apr_fileperms_t perm, apr_pool_t pool) { int oflags = 0; int mflags = OPEN_FLAGS_FAIL_ON_ERROR\|OPEN_SHARE_DENYNONE\|OPEN_FLAGS_NOINHERIT; int rv; ULONG action; + ULONG CurMaxFH = 0; / Current count of handles / + LONG ReqCount = 0; / Number to adjust file handles / + apr_file_t dafile = (apr_file_t )apr_palloc(pool, sizeof(apr_file_t)); if (flag & APR_FOPEN_NONBLOCK) { @@ -90,6 +146,15 @@ rv = DosOpen(fname, &(dafile->filedes), &action, 0, 0, oflags, mflags, NULL); + if (rv == ERROR_TOO_MANY_OPEN_FILES) { + rv = DosSetRelMaxFH(&ReqCount, / Using 0 here will return the / + &CurMaxFH); / current number of file handles / + ReqCount = 10L; / Want 10 more file handles / + rv = DosSetRelMaxFH(&ReqCount,&CurMaxFH); / Change handle maximum / + + rv = DosOpen(fname, &(dafile->filedes), &action, 0, 0, oflags, mflags, NULL); + } + if (rv == 0 && (flag & APR_FOPEN_APPEND)) { ULONG newptr; rv = DosSetFilePtr(dafile->filedes, 0, FILE_END, &newptr ); @@ -108,7 +173,9 @@ dafile->dataRead = 0; dafile->direction = 0; dafile->pipe = FALSE; + dafile->ungetchar = -1; + if (!(flag & APR_FOPEN_NOCLEANUP)) { apr_pool_cleanup_register(dafile->pool, dafile, apr_file_cleanup, apr_file_cleanup); } @@ -127,6 +194,11 @@ if (file && file->isopen) { / XXX: flush here is not mutex protected / status = apr_file_flush(file); + / 2014-11-18 SHL ensure client sees ERROR_BROKEN_PIPE / + if (file->pipe == 1) { + DosCloseEventSem(file->pipeSem); + DosDisconnectNPipe(file->filedes); + } rc = DosClose(file->filedes); if (rc == 0) { @@ -197,6 +269,7 @@ (file)->flags = flags; (file)->pipe = FALSE; (file)->buffered = (flags & APR_FOPEN_BUFFERED) > 0; + (file)->ungetchar = -1; if ((file)->buffered) { apr_status_t rv; @@ -279,11 +352,9 @@ ULONG state; rv = DosQueryFHState(thefile->filedes, &state); - if (rv == 0 && (state & OPEN_FLAGS_NOINHERIT) != 0) { rv = DosSetFHState(thefile->filedes, state & ~OPEN_FLAGS_NOINHERIT); } - return APR_FROM_OS_ERROR(rv); } @@ -295,10 +366,14 @@ ULONG state; rv = DosQueryFHState(thefile->filedes, &state); - if (rv == 0 && (state & OPEN_FLAGS_NOINHERIT) == 0) { - rv = DosSetFHState(thefile->filedes, state \| OPEN_FLAGS_NOINHERIT); +/* rv = DosSetFHState(thefile->filedes, state \| OPEN_FLAGS_NOINHERIT);/ } - return APR_FROM_OS_ERROR(rv); } + +APR_DECLARE(apr_status_t) apr_file_link(const char from_path, + const char to_path) +{ + return APR_ENOTIMPL;; +} open.c.diff (4,577 bytes) pipe.c.diff (3,450 bytes) Index: pipe.c =================================================================== --- pipe.c (revision 1629784) +++ pipe.c (working copy) @@ -30,6 +30,8 @@ ULONG rc, action; static int id = 0; char pipename[50]; + ULONG CurMaxFH = 0; / Current count of handles / + LONG ReqCount = 0; / Number to adjust file handles / sprintf(pipename, "/pipe/%d.%d", getpid(), id++); rc = DosCreateNPipe(pipename, filedes, NP_ACCESS_INBOUND, NP_NOWAIT\|1, 4096, 4096, 0); @@ -48,7 +50,19 @@ OPEN_ACTION_OPEN_IF_EXISTS \| OPEN_ACTION_FAIL_IF_NEW, OPEN_ACCESS_WRITEONLY \| OPEN_SHARE_DENYREADWRITE, NULL); + if (rc == ERROR_TOO_MANY_OPEN_FILES) { + rc = DosSetRelMaxFH(&ReqCount, / Using 0 here will return the / + &CurMaxFH); / current number of file handles / + ReqCount = 10L; / Want 10 more file handles / + rc = DosSetRelMaxFH(&ReqCount,&CurMaxFH); / Change handle maximum / + rc = DosOpen (pipename, filedes+1, &action, 0, FILE_NORMAL, + OPEN_ACTION_OPEN_IF_EXISTS \| OPEN_ACTION_FAIL_IF_NEW, + OPEN_ACCESS_WRITEONLY \| OPEN_SHARE_DENYREADWRITE, + NULL); + + } + if (rc) { DosClose(filedes[0]); return APR_FROM_OS_ERROR(rc); @@ -84,6 +98,7 @@ (in)->flags = 0; (in)->pipe = 1; (in)->timeout = -1; + (in)->ungetchar = -1; (in)->blocking = BLK_ON; apr_pool_cleanup_register(pool, in, apr_file_cleanup, apr_pool_cleanup_null); @@ -94,7 +109,8 @@ (out)->isopen = TRUE; (out)->buffered = FALSE; (out)->flags = 0; - (out)->pipe = 1; + (out)->pipe = 2; // 2014-11-17 SHL mark as client pipe + (out)->ungetchar = -1; (out)->timeout = -1; (out)->blocking = BLK_ON; apr_pool_cleanup_register(pool, out, apr_file_cleanup, apr_pool_cleanup_null); @@ -143,24 +159,25 @@ APR_DECLARE(apr_status_t) apr_file_pipe_timeout_set(apr_file_t thepipe, apr_interval_time_t timeout) { + int rc = 0; + if (thepipe->pipe == 1) { thepipe->timeout = timeout; - if (thepipe->timeout >= 0) { - if (thepipe->blocking != BLK_OFF) { + if ((thepipe->timeout >= 0) && (thepipe->blocking != BLK_OFF) ) { thepipe->blocking = BLK_OFF; - return APR_FROM_OS_ERROR(DosSetNPHState(thepipe->filedes, NP_NOWAIT)); + rc = DosSetNPHState(thepipe->filedes, NP_NOWAIT); } - } - else if (thepipe->timeout == -1) { - if (thepipe->blocking != BLK_ON) { + else if ( (thepipe->timeout == -1) && (thepipe->blocking != BLK_ON) ) { thepipe->blocking = BLK_ON; - return APR_FROM_OS_ERROR(DosSetNPHState(thepipe->filedes, NP_WAIT)); + rc = DosSetNPHState(thepipe->filedes, NP_WAIT); } + + return APR_FROM_OS_ERROR(rc); + + } else return APR_EINVAL; + } - } - return APR_EINVAL; -} @@ -186,6 +203,7 @@ (file)->pipe = 1; (file)->blocking = BLK_UNKNOWN; / app needs to make a timeout call / (file)->timeout = -1; + (file)->ungetchar = -1; (file)->filedes = *thefile; if (register_cleanup) { pipe.c.diff (3,450 bytes) siegeurls1 (78,190 bytes) http://www.2rosenthals.net/ http://www.2rosenthals.net/coppermine http://www.2rosenthals.net/coppermine/ http://www.2rosenthals.net/coppermine/albums/upload/BVC/thumb_img011.jpg http://www.2rosenthals.net/coppermine/albums/upload/BVC/thumb_img012.jpg http://www.2rosenthals.net/coppermine/albums/upload/BVC/thumb_img01.jpg http://www.2rosenthals.net/coppermine/albums/upload/BVC/thumb_img04.jpg http://www.2rosenthals.net/coppermine/albums/upload/mjones/thumb_dsc00197.jpg http://www.2rosenthals.net/coppermine/albums/upload/wallpapers/SuSE/9_3/thumb_suse-default.png http://www.2rosenthals.net/coppermine/albums/userpics/10001/thumb_5192245.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10001/thumb_Dscn4596.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10001/thumb_Dscn4597.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10001/thumb_IMG00285.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10002/thumb_71407_06.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10002/thumb_71407_07.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10002/thumb_71407_08.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10002/thumb_71407_09.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10002/thumb_71407_10.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10002/thumb_71407_11.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_1963_Bonneville.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_64477956741_3300.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_64479166213_3300.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_bcar001.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_bcar006.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_bealtonmap.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_DSCN2749.JPG http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_DSCN2751.JPG http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_DSCN2755.JPG http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_dscn2824.jpg http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_DSCN2846.JPG http://www.2rosenthals.net/coppermine/albums/userpics/10003/thumb_DSCN2847.JPG http://www.2rosenthals.net/coppermine/contact.php?referer=index.php http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D1 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D13 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D14 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D15 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D16 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D17 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D4 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D6 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Fcat%3D9 http://www.2rosenthals.net/coppermine/contact.php?referer=index.php%3Flang%3Dxxx http://www.2rosenthals.net/coppermine/contact.php?referer=index.php&lang=xxx http://www.2rosenthals.net/coppermine/contact.php?referer=login.php%3Freferer%3Dindex.php http://www.2rosenthals.net/coppermine/contact.php?referer=search.php http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D15 http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D18 http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D2 http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D22 http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D24 http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D5 http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D6 http://www.2rosenthals.net/coppermine/contact.php?referer=thumbnails.php%3Falbum%3D7 http://www.2rosenthals.net/coppermine/displayimage.php?album=15&pid=217 http://www.2rosenthals.net/coppermine/displayimage.php?album=15&pid=236 http://www.2rosenthals.net/coppermine/displayimage.php?album=15&pid=237 http://www.2rosenthals.net/coppermine/displayimage.php?album=15&pid=238 http://www.2rosenthals.net/coppermine/displayimage.php?album=15&pid=306 http://www.2rosenthals.net/coppermine/displayimage.php?album=15&pid=308 http://www.2rosenthals.net/coppermine/displayimage.php?album=15&pid=315 http://www.2rosenthals.net/coppermine/displayimage.php?album=16&pid=186 http://www.2rosenthals.net/coppermine/displayimage.php?album=18&pid=189 http://www.2rosenthals.net/coppermine/displayimage.php?album=18&pid=344 http://www.2rosenthals.net/coppermine/displayimage.php?album=18&pid=345 http://www.2rosenthals.net/coppermine/displayimage.php?album=18&pid=348 http://www.2rosenthals.net/coppermine/displayimage.php?album=18&pid=349 http://www.2rosenthals.net/coppermine/displayimage.php?album=18&pid=353 http://www.2rosenthals.net/coppermine/displayimage.php?album=22&pid=562 http://www.2rosenthals.net/coppermine/displayimage.php?album=7&pid=17 http://www.2rosenthals.net/coppermine/displayimage.php?album=7&pid=18 http://www.2rosenthals.net/coppermine/displayimage.php?album=7&pid=564 http://www.2rosenthals.net/coppermine/displayimage.php?album=7&pid=565 http://www.2rosenthals.net/coppermine/help.php?f=empty.htm&base=64&h=czoyOToiVmlzdWFsIGNvbmZpcm1hdGlvbiAoY2FwdGNoYSkiOw%3D%3D&t=czoxOTE6IlRvIGF2b2lkIHNwYW0sIHlvdSBoYXZlIHRvIGNvbmZpcm0gdGhhdCB5b3UgYXJlIGFuIGFjdHVhbCBodW1hbiBiZWluZyBhbmQgbm90IGp1c3QgYSBib3Qgc2NyaXB0IGJ5IGVudGVyaW5nIHRoZSBkaXNwbGF5ZWQgdGV4dC48YnIgLz5DYXBpdGFsaXphdGlvbiBkb2VzIG5vdCBtYXR0ZXIsIHlvdSBjYW4gdHlwZSBpbiBsb3dlcmNhc2UuIjs%3D http://www.2rosenthals.net/coppermine/images/flags/reset.png http://www.2rosenthals.net/coppermine/images/help.gif http://www.2rosenthals.net/coppermine/images/h_powered-mysql.gif http://www.2rosenthals.net/coppermine/images/h_powered-php.gif http://www.2rosenthals.net/coppermine/images/h_valid-css.gif http://www.2rosenthals.net/coppermine/images/h_valid-xhtml10.gif http://www.2rosenthals.net/coppermine/images/powered-mysql.gif http://www.2rosenthals.net/coppermine/images/powered-php.gif http://www.2rosenthals.net/coppermine/images/spacer.gif http://www.2rosenthals.net/coppermine/images/thumbs/thumb_avi.png http://www.2rosenthals.net/coppermine/images/thumbs/thumb_private.png http://www.2rosenthals.net/coppermine/images/valid-css.gif http://www.2rosenthals.net/coppermine/images/valid-xhtml10.gif http://www.2rosenthals.net/coppermine/index.php http://www.2rosenthals.net/coppermine/index.php?cat=0 http://www.2rosenthals.net/coppermine/index.php?cat=1 http://www.2rosenthals.net/coppermine/index.php?cat=10 http://www.2rosenthals.net/coppermine/index.php?cat=10007 http://www.2rosenthals.net/coppermine/index.php?cat=10&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=11 http://www.2rosenthals.net/coppermine/index.php?cat=11&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=13 http://www.2rosenthals.net/coppermine/index.php?cat=13&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=14 http://www.2rosenthals.net/coppermine/index.php?cat=14&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=15 http://www.2rosenthals.net/coppermine/index.php?cat=16 http://www.2rosenthals.net/coppermine/index.php?cat=17 http://www.2rosenthals.net/coppermine/index.php?cat=17&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=2 http://www.2rosenthals.net/coppermine/index.php?cat=2&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=3 http://www.2rosenthals.net/coppermine/index.php?cat=3&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=4 http://www.2rosenthals.net/coppermine/index.php?cat=4&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=5 http://www.2rosenthals.net/coppermine/index.php?cat=6 http://www.2rosenthals.net/coppermine/index.php?cat=6&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=7 http://www.2rosenthals.net/coppermine/index.php?cat=7&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=8 http://www.2rosenthals.net/coppermine/index.php?cat=8&lang=xxx http://www.2rosenthals.net/coppermine/index.php?cat=9 http://www.2rosenthals.net/coppermine/index.php?lang=xxx http://www.2rosenthals.net/coppermine/js/contact.js http://www.2rosenthals.net/coppermine/js/jquery-1.3.2.js http://www.2rosenthals.net/coppermine/js/jquery.elastic.js http://www.2rosenthals.net/coppermine/js/jquery.greybox.js http://www.2rosenthals.net/coppermine/js/scripts.js http://www.2rosenthals.net/coppermine/js/thumbnails.js http://www.2rosenthals.net/coppermine/login.php http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D179 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D180 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D217 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D236 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D237 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D238 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D306 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D307 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D314 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D16%26pid%3D185 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D16%26pid%3D186 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D18%26pid%3D189 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D18%26pid%3D203 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D18%26pid%3D344 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D18%26pid%3D349 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D18%26pid%3D353 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D22%26pid%3D562 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D16 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D17 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D20 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D21 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D563 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D564 http://www.2rosenthals.net/coppermine/login.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D566 http://www.2rosenthals.net/coppermine/login.php?referer=contact.php%3Freferer%3Dindex.php http://www.2rosenthals.net/coppermine/login.php?referer=index.php http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D0 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D1 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D10 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D11 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D14 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D16 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D2 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D4 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D5 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D8 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Fcat%3D9 http://www.2rosenthals.net/coppermine/login.php?referer=index.php%3Flang%3Dxxx http://www.2rosenthals.net/coppermine/login.php?referer=index.php&lang=xxx http://www.2rosenthals.net/coppermine/login.php?referer=thumbnails.php%3Falbum%3D16 http://www.2rosenthals.net/coppermine/login.php?referer=thumbnails.php%3Falbum%3D17 http://www.2rosenthals.net/coppermine/login.php?referer=thumbnails.php%3Falbum%3D2 http://www.2rosenthals.net/coppermine/login.php?referer=thumbnails.php%3Falbum%3D6 http://www.2rosenthals.net/coppermine/login.php?referer=thumbnails.php%3Falbum%3D7 http://www.2rosenthals.net/coppermine/plugins/light_box/slideshow/css/lightbox.css http://www.2rosenthals.net/coppermine/plugins/light_box/slideshow/images/blank.gif http://www.2rosenthals.net/coppermine/plugins/light_box/slideshow/images/nextlabel.gif http://www.2rosenthals.net/coppermine/plugins/light_box/slideshow/images/prevlabel.gif http://www.2rosenthals.net/coppermine/plugins/light_box/slideshow/js/lightbox_s.js http://www.2rosenthals.net/coppermine/plugins/light_box/slideshow/js/prototype.js http://www.2rosenthals.net/coppermine/plugins/light_box/slideshow/js/scriptaculous.js?load=effects http://www.2rosenthals.net/coppermine/profile.php?uid=7 http://www.2rosenthals.net/coppermine/js/scripts.js http://www.2rosenthals.net/coppermine/search.php http://www.2rosenthals.net/coppermine/send_activation.php http://www.2rosenthals.net/coppermine/themes/classic/images/button_bg_anim.gif http://www.2rosenthals.net/coppermine/themes/classic/images/button_bg.gif http://www.2rosenthals.net/coppermine/themes/classic/images/site_logo.png http://www.2rosenthals.net/coppermine/themes/classic/style.css http://www.2rosenthals.net/coppermine/thumbnails.php?album=1 http://www.2rosenthals.net/coppermine/thumbnails.php?album=10 http://www.2rosenthals.net/coppermine/thumbnails.php?album=11 http://www.2rosenthals.net/coppermine/thumbnails.php?album=12 http://www.2rosenthals.net/coppermine/thumbnails.php?album=13 http://www.2rosenthals.net/coppermine/thumbnails.php?album=13&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=14 http://www.2rosenthals.net/coppermine/thumbnails.php?album=15 http://www.2rosenthals.net/coppermine/thumbnails.php?album=15&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=15&page=2 http://www.2rosenthals.net/coppermine/thumbnails.php?album=15&page=6 http://www.2rosenthals.net/coppermine/thumbnails.php?album=16 http://www.2rosenthals.net/coppermine/thumbnails.php?album=17 http://www.2rosenthals.net/coppermine/thumbnails.php?album=17&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=18 http://www.2rosenthals.net/coppermine/thumbnails.php?album=19 http://www.2rosenthals.net/coppermine/thumbnails.php?album=2 http://www.2rosenthals.net/coppermine/thumbnails.php?album=21 http://www.2rosenthals.net/coppermine/thumbnails.php?album=22 http://www.2rosenthals.net/coppermine/thumbnails.php?album=22&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=24 http://www.2rosenthals.net/coppermine/thumbnails.php?album=24&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=2&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=4 http://www.2rosenthals.net/coppermine/thumbnails.php?album=5 http://www.2rosenthals.net/coppermine/thumbnails.php?album=5&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=6 http://www.2rosenthals.net/coppermine/thumbnails.php?album=6&lang=xxx http://www.2rosenthals.net/coppermine/thumbnails.php?album=7 http://www.2rosenthals.net/coppermine/thumbnails.php?album=favpics http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=0 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=10 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=13 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=-13 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=15 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=-15 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=-2 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=-24 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=5 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=-5 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=6 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=-6 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=7 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=8 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastcom&cat=9 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=0 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=10 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=13 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=14 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=15 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=-16 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=17 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=-18 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=2 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=-24 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=4 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=5 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=-7 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=8 http://www.2rosenthals.net/coppermine/thumbnails.php?album=lastup&cat=9 http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=2001 http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=3rd http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=ahoy http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=apron http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=art http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=awning http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=back http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=cadillac http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=cake http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=calvert http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=car http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=center http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=curriculum http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=dashboard http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=decklid http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=dog http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=door http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=drawing http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=dvd http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=fender http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=frame http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=gas http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=grade http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=grill http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=gutter http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=hills http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=indoors http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=inside http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=interior http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=issue http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=kitchen http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=left http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=lesson http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=lilac http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=luke http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=montage http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=moulding http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=new http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=oak http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=outside http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=panel http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=parts http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=passenger http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=present http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=profile http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=rachie http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=santa http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=site http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=spring http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=step http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=swing http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=tank http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=top http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=tree http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=tricycle http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=trim http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=windshield http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=wink http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=woodpile http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=yaya http://www.2rosenthals.net/coppermine/thumbnails.php?album=search&keywords=on&search=york http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=0 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=1 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=10 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=11 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=13 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=14 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=15 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=-15 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=16 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=-17 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=-18 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=2 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=-24 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=4 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=-5 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=6 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=-6 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=-7 http://www.2rosenthals.net/coppermine/thumbnails.php?album=topn&cat=8 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=0 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=1 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=10 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=13 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=-13 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=14 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=16 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=17 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=-17 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=-24 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=3 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=-5 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=6 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=7 http://www.2rosenthals.net/coppermine/thumbnails.php?album=toprated&cat=-7 http://www.2rosenthals.netlogin.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D237 http://www.2rosenthals.netlogin.php?force_login=1&referer=displayimage.php%3Falbum%3D15%26pid%3D315 http://www.2rosenthals.netlogin.php?force_login=1&referer=displayimage.php%3Falbum%3D18%26pid%3D189 http://www.2rosenthals.netlogin.php?force_login=1&referer=displayimage.php%3Falbum%3D18%26pid%3D349 http://www.2rosenthals.netlogin.php?force_login=1&referer=displayimage.php%3Falbum%3D7%26pid%3D564 http://www.2rosenthals.net/robots.txt http://www.2rosenthals.net/wordpress http://www.2rosenthals.net/wordpress/ http://www.2rosenthals.net/wordpress/abduction-save-as-image-mod-for-seamonkey-576/ http://www.2rosenthals.net/wordpress/abduction-save-as-image-mod-for-seamonkey-576/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/abduction-save-as-image-mod-for-seamonkey-576/?action=register&instance=1 http://www.2rosenthals.net/wordpress/abduction-save-as-image-mod-for-seamonkey-576/comment-page-1/ http://www.2rosenthals.net/wordpress/abduction-save-as-image-mod-for-seamonkey-576/feed/ http://www.2rosenthals.net/wordpress/abduction-save-as-image-mod-for-seamonkey-576/?format=pdf http://www.2rosenthals.net/wordpress/?action=login&instance=1 http://www.2rosenthals.net/wordpress/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/betterprivacy-mod-for-seamonkey-532/ http://www.2rosenthals.net/wordpress/betterprivacy-mod-for-seamonkey-532/?format=pdf http://www.2rosenthals.net/wordpress/cloud-adoption-brings-unexpected-costs-kpmg-survey-says-583/?action=register&instance=1 http://www.2rosenthals.net/wordpress/cloud-adoption-brings-unexpected-costs-kpmg-survey-says-583/feed/ http://www.2rosenthals.net/wordpress/cloud-adoption-brings-unexpected-costs-kpmg-survey-says-583/?format=pdf http://www.2rosenthals.net/wordpress/comments/feed/ http://www.2rosenthals.net/wordpress/configuring-squid-proxy-on-os2-path-adjustments-399/ http://www.2rosenthals.net/wordpress/contact/ http://www.2rosenthals.net/wordpress/core/core-images/backgrounds/low_contrast_linen.png http://www.2rosenthals.net/wordpress/crts-vs-lcds-in-20112012-388/ http://www.2rosenthals.net/wordpress/date/2013/02/ http://www.2rosenthals.net/wordpress/egad-why-do-people-do-their-own-web-development-384/ http://www.2rosenthals.net/wordpress/facebook-threatens-to-shut-down-conservative-site-fox-news-commentary-todd-starnes-582/ http://www.2rosenthals.net/wordpress/facebook-threatens-to-shut-down-conservative-site-fox-news-commentary-todd-starnes-582/?action=register&instance=1 http://www.2rosenthals.net/wordpress/facebook-threatens-to-shut-down-conservative-site-fox-news-commentary-todd-starnes-582/feed/ http://www.2rosenthals.net/wordpress/feed/ http://www.2rosenthals.net/wordpress/file/?action=register&instance=1 http://www.2rosenthals.net/wordpress/file/feed/ http://www.2rosenthals.net/wordpress/firesheep-not-on-a-hautspot-network-123/ http://www.2rosenthals.net/wordpress/group-chat-showdown-which-instant-messaging-service-is-best-for-your-business-pcworld-523/ http://www.2rosenthals.net/wordpress/help/ http://www.2rosenthals.net/wordpress/help/about/ http://www.2rosenthals.net/wordpress/help/about/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/help/about/?action=register&instance=1 http://www.2rosenthals.net/wordpress/help/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/index.php?=PHPE9568F34-D428-11d2-A769-00AA001ACF42 http://www.2rosenthals.net/wordpress/index.php?=PHPE9568F35-D428-11d2-A769-00AA001ACF42 http://www.2rosenthals.net/wordpress/installing-windows-server-2008-r2-x64-on-the-hp-proliant-dl380-g4-108/ http://www.2rosenthals.net/wordpress/login/ http://www.2rosenthals.net/wordpress/login/?action=logout&_wpnonce=64c145177b http://www.2rosenthals.net/wordpress/login/?action=profile http://www.2rosenthals.net/wordpress/login/?loggedout=true http://www.2rosenthals.net/wordpress/login/?redirect_to=http%3A%2F%2Fwww.2rosenthals.net%2Fwordpress%2Fcloud-adoption-brings-unexpected-costs-kpmg-survey-says-583%2F http://www.2rosenthals.net/wordpress/login/?redirect_to=http%3A%2F%2Fwww.2rosenthals.net%2Fwordpress%2Ffile%2F http://www.2rosenthals.net/wordpress/login/?redirect_to=http%3A%2F%2Fwww.2rosenthals.net%2Fwordpress%2Fremoving-the-wp-post-to-pdf-icon-in-wptouch-569%2F http://www.2rosenthals.net/wordpress/login/?redirect_to=http%3A%2F%2Fwww.2rosenthals.net%2Fwordpress%2Fwp-admin%2Fplugin-install.php&reauth=1 http://www.2rosenthals.net/wordpress/login/?redirect_to=http%3A%2F%2Fwww.2rosenthals.net%2Fwordpress%2Fwp-admin%2Fupdate-core.php&reauth=1 http://www.2rosenthals.net/wordpress/mantis-email-fun-part-2-362/ http://www.2rosenthals.net/wordpress/mass-editing-users-in-wordpress-528/ http://www.2rosenthals.net/wordpress/more-cloud-disasters-499/ http://www.2rosenthals.net/wordpress/more-cloud-disasters-499/?format=pdf http://www.2rosenthals.net/wordpress/mozilla-hardware-acceleration-and-vnc-sessions-186/ http://www.2rosenthals.net/wordpress/new-upgrade-yet-again-492/ http://www.2rosenthals.net/wordpress/on-the-ungrateful-nature-of-users-and-the-oss-development-community-438/ http://www.2rosenthals.net/wordpress/?p=569 http://www.2rosenthals.net/wordpress/?p=582 http://www.2rosenthals.net/wordpress/?p=584 http://www.2rosenthals.net/wordpress/?p=584&preview=true http://www.2rosenthals.net/wordpress/?p=596&preview=true http://www.2rosenthals.net/wordpress/page/2/ http://www.2rosenthals.net/wordpress/ramdom-thoughts-on-the-2011-and-beyond-firefox-release-schedule-223/ http://www.2rosenthals.net/wordpress/random-thoughts-on-thunderbirds-current-state-495/ http://www.2rosenthals.net/wordpress/removing-the-wp-post-to-pdf-icon-in-wptouch-569/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/removing-the-wp-post-to-pdf-icon-in-wptouch-569/feed/ http://www.2rosenthals.net/wordpress/removing-the-wp-post-to-pdf-icon-in-wptouch-569/?format=pdf http://www.2rosenthals.net/wordpress/rsync-error-5-may-be-exactly-what-it-says-165/ http://www.2rosenthals.net/wordpress/running-rconsolej-on-os2-483/ http://www.2rosenthals.net/wordpress/tag/64-bit/ http://www.2rosenthals.net/wordpress/tag/apache/ http://www.2rosenthals.net/wordpress/tag/arena/ http://www.2rosenthals.net/wordpress/tag/buzzword/ http://www.2rosenthals.net/wordpress/tag/chip/ http://www.2rosenthals.net/wordpress/tag/chrome/ http://www.2rosenthals.net/wordpress/tag/cloud/ http://www.2rosenthals.net/wordpress/tag/cloud/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/conf/ http://www.2rosenthals.net/wordpress/tag/consult/ http://www.2rosenthals.net/wordpress/tag/crt/ http://www.2rosenthals.net/wordpress/tag/ddos/ http://www.2rosenthals.net/wordpress/tag/directory/ http://www.2rosenthals.net/wordpress/tag/ecomstation/ http://www.2rosenthals.net/wordpress/tag/ecs-2/ http://www.2rosenthals.net/wordpress/tag/extended-attributes/ http://www.2rosenthals.net/wordpress/tag/facebook/ http://www.2rosenthals.net/wordpress/tag/facebook/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/files/ http://www.2rosenthals.net/wordpress/tag/files/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/tag/files/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/files/feed/ http://www.2rosenthals.net/wordpress/tag/files/three-good-ways-to-thwart-tracking-attempts-on-the-net-251/ http://www.2rosenthals.net/wordpress/tag/filesystem/ http://www.2rosenthals.net/wordpress/tag/firefox/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/fix/ http://www.2rosenthals.net/wordpress/tag/fix/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/foss/ http://www.2rosenthals.net/wordpress/tag/fragmentation/ http://www.2rosenthals.net/wordpress/tag/green/ http://www.2rosenthals.net/wordpress/tag/head-parking/ http://www.2rosenthals.net/wordpress/tag/heat/ http://www.2rosenthals.net/wordpress/tag/hosted/ http://www.2rosenthals.net/wordpress/tag/how-to/ http://www.2rosenthals.net/wordpress/tag/humor/ http://www.2rosenthals.net/wordpress/tag/hype/ http://www.2rosenthals.net/wordpress/tag/hype/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/hype/feed/ http://www.2rosenthals.net/wordpress/tag/images/ http://www.2rosenthals.net/wordpress/tag/images/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/install/ http://www.2rosenthals.net/wordpress/tag/internet-2/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/tag/internet-2/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/internet-2/feed/ http://www.2rosenthals.net/wordpress/tag/iproute2/ http://www.2rosenthals.net/wordpress/tag/jabber/ http://www.2rosenthals.net/wordpress/tag/jfs/ http://www.2rosenthals.net/wordpress/tag/lcd/ http://www.2rosenthals.net/wordpress/tag/linux/ http://www.2rosenthals.net/wordpress/tag/management/ http://www.2rosenthals.net/wordpress/tag/mantis-2/ http://www.2rosenthals.net/wordpress/tag/mozilla/ http://www.2rosenthals.net/wordpress/tag/mozilla/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/tag/mozilla/feed/ http://www.2rosenthals.net/wordpress/tag/msf/ http://www.2rosenthals.net/wordpress/tag/myspace/ http://www.2rosenthals.net/wordpress/tag/mysql-2/ http://www.2rosenthals.net/wordpress/tag/netscape/ http://www.2rosenthals.net/wordpress/tag/noise/ http://www.2rosenthals.net/wordpress/tag/nvidia/ http://www.2rosenthals.net/wordpress/tag/oss/ http://www.2rosenthals.net/wordpress/tag/perl/ http://www.2rosenthals.net/wordpress/tag/php-2/ http://www.2rosenthals.net/wordpress/tag/power-management/ http://www.2rosenthals.net/wordpress/tag/privacy/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/tag/proliant/ http://www.2rosenthals.net/wordpress/tag/quality/ http://www.2rosenthals.net/wordpress/tag/release-cycle/ http://www.2rosenthals.net/wordpress/tag/repair/ http://www.2rosenthals.net/wordpress/tag/rim/ http://www.2rosenthals.net/wordpress/tag/routing-2/ http://www.2rosenthals.net/wordpress/tag/script/ http://www.2rosenthals.net/wordpress/tag/seamonkey/ http://www.2rosenthals.net/wordpress/tag/seamonkey/?action=lostpassword&instance=1 http://www.2rosenthals.net/wordpress/tag/seamonkey/?action=register&instance=1 http://www.2rosenthals.net/wordpress/tag/security/ http://www.2rosenthals.net/wordpress/tag/server-2008/ http://www.2rosenthals.net/wordpress/tag/site/ http://www.2rosenthals.net/wordpress/tag/social-networking/feed/ http://www.2rosenthals.net/wordpress/tag/squid/ http://www.2rosenthals.net/wordpress/tag/symlink/ http://www.2rosenthals.net/wordpress/tag/technology/ http://www.2rosenthals.net/wordpress/tag/thinkpad/ http://www.2rosenthals.net/wordpress/tag/time-wasters/ http://www.2rosenthals.net/wordpress/tag/troubleshooting/ http://www.2rosenthals.net/wordpress/tag/tuning/ http://www.2rosenthals.net/wordpress/tag/twitter/ http://www.2rosenthals.net/wordpress/tag/update/ http://www.2rosenthals.net/wordpress/tag/upgrade/ http://www.2rosenthals.net/wordpress/tag/vnc/ http://www.2rosenthals.net/wordpress/tag/windows/ http://www.2rosenthals.net/wordpress/tag/wordpress/ http://www.2rosenthals.net/wordpress/tag/workplace-shell/ http://www.2rosenthals.net/wordpress/tag/x64/ http://www.2rosenthals.net/wordpress/tag/xmpp/ http://www.2rosenthals.net/wordpress/the-strangest-laptop-repair-ive-ever-done-127/ http://www.2rosenthals.net/wordpress/the-strangest-laptop-repair-ive-ever-done-127/comment-page-1/ http://www.2rosenthals.net/wordpress/the-telltale-hard-drive-173/ http://www.2rosenthals.net/wordpress/updating-firefox-on-linux-the-dreaded-couldnt-load-xpcom-190/ http://www.2rosenthals.net/wordpress/warpstock-2012-portland-oregon-usa-488/ http://www.2rosenthals.net/wordpress/welcome/ http://www.2rosenthals.net/wordpress/welcome/?action=register&instance=1 http://www.2rosenthals.net/wordpress/welcome/?page_id=44 http://www.2rosenthals.net/wordpress/why-i-still-use-os2-ecomstation-436/ http://www.2rosenthals.net/wordpress/with-the-coming-of-windows-8-there-has-never-been-a-better-time-to-switch-platforms-498/ http://www.2rosenthals.net/wordpress/workaround-multi-column-footers-in-openoffice-org-writer-540/ http://www.2rosenthals.net/wordpress/wp-admin/ http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=ajax-tag-search&tax=post_tag&q=clie http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=ajax-tag-search&tax=post_tag&q=client http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=ajax-tag-search&tax=post_tag&q=pr http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=ajax-tag-search&tax=post_tag&q=pract http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=ajax-tag-search&tax=post_tag&q=practi http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=ajax-tag-search&tax=post_tag&q=practice http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=ajax-tag-search&tax=post_tag&q=przc http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=dashboard-widgets&widget=dashboard_incoming_links http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=dashboard-widgets&widget=dashboard_plugins http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=dashboard-widgets&widget=dashboard_primary http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=dashboard-widgets&widget=dashboard_secondary http://www.2rosenthals.net/wordpress/wp-admin/admin-ajax.php?action=oembed-cache&post=569 http://www.2rosenthals.net/wordpress/wp-admin/admin.php?page=ckeditor_advanced_options http://www.2rosenthals.net/wordpress/wp-admin/admin.php?page=ckeditor_basic_options http://www.2rosenthals.net/wordpress/wp-admin/admin.php?page=ckeditor_settings http://www.2rosenthals.net/wordpress/wp-admin/admin.php?page=stats&noheader=1 http://www.2rosenthals.net/wordpress/wp-admin/css/colors-fresh.min.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/edit.php http://www.2rosenthals.net/wordpress/wp-admin/images/arrows.png http://www.2rosenthals.net/wordpress/wp-admin/images/bubble_bg.gif http://www.2rosenthals.net/wordpress/wp-admin/images/button-grad.png http://www.2rosenthals.net/wordpress/wp-admin/images/comment-grey-bubble.png http://www.2rosenthals.net/wordpress/wp-admin/images/date-button.gif http://www.2rosenthals.net/wordpress/wp-admin/images/icons32-2x.png?ver=20121105 http://www.2rosenthals.net/wordpress/wp-admin/images/icons32.png?ver=20111206 http://www.2rosenthals.net/wordpress/wp-admin/images/icons32.png?ver=20121105 http://www.2rosenthals.net/wordpress/wp-admin/images/list.png http://www.2rosenthals.net/wordpress/wp-admin/images/media-button-2x.png http://www.2rosenthals.net/wordpress/wp-admin/images/media-button.png http://www.2rosenthals.net/wordpress/wp-admin/images/menu.png?ver=20120201 http://www.2rosenthals.net/wordpress/wp-admin/images/menu.png?ver=20121105 http://www.2rosenthals.net/wordpress/wp-admin/images/menu-shadow.png http://www.2rosenthals.net/wordpress/wp-admin/images/press-this.png?v=20120502 http://www.2rosenthals.net/wordpress/wp-admin/images/resize.gif http://www.2rosenthals.net/wordpress/wp-admin/images/sort.gif http://www.2rosenthals.net/wordpress/wp-admin/images/stars.png?ver=20121108 http://www.2rosenthals.net/wordpress/wp-admin/images/white-grad-active.png http://www.2rosenthals.net/wordpress/wp-admin/images/white-grad.png http://www.2rosenthals.net/wordpress/wp-admin/images/wordpress-logo-2x.png?ver=20120412 http://www.2rosenthals.net/wordpress/wp-admin/images/wpspin_light.gif http://www.2rosenthals.net/wordpress/wp-admin/images/xit.gif http://www.2rosenthals.net/wordpress/wp-admin/index.php http://www.2rosenthals.net/wordpress/wp-admin/js/password-strength-meter.min.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/js/user-profile.min.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=0&load=admin-bar,hoverIntent,common,jquery-color,thickbox,plugin-install,jquery-ui-widget,jquery-ui-position,wp-pointer,jquery-ui-core,jquery-ui-datepicker&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=0&load=admin-bar,hoverIntent,common,jquery-color,thickbox,theme,customize-base,customize-loader,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=0&load=admin-bar,hoverIntent,common,jquery-color,thickbox,wp-ajax-response,wp-lists,quicktags,jquery-query,admin-comments,jquery-ui-core,jquery-ui-widget,jquery-ui-mouse,jquery-ui-sortable,postbox,dashboard,plugin-install,media-upload,jquery-ui-position,wp-pointer,editor&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=0&load[]=admin-bar,hoverIntent,common,thickbox,plugin-install,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=0&load=hoverIntent,common,jquery-color,thickbox,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=0&load=jquery,utils,json2&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=0&load=jquery,utils&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,jquery-ui-core,jquery-ui-widget,jquery-ui-mouse,jquery-ui-sortable,postbox,thickbox,jquery-ui-posit&load%5B%5D=ion,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,plugin-install,jquery-ui-widget,jquery-ui-position,wp-pointer,jquery-ui-core,jquery-ui-dat&load%5B%5D=epicker&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,plugin-install,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,schedule,wp-ajax-response,autosave,jquery-color,wp-lists,quicktags,jquery-query,admin-comm&load%5B%5D=ents,suggest,jquery-ui-core,jquery-ui-widget,jquery-ui-mouse,jquery-ui-sortable,postbox,post,underscore,shortcode,backbone,media&load%5B%5D=-models,wp-plupload,media-views,media-editor,jquery-ui-position,wp-pointer,editor&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,schedule,wp-ajax-response,autosave,jquery-color,wp-lists,quicktags,jquery-query,admin-comm&load%5B%5D=ents,suggest,jquery-ui-core,jquery-ui-widget,jquery-ui-mouse,jquery-ui-sortable,postbox,post,underscore,shortcode,backbone,media&load%5B%5D=-models,wp-plupload,media-views,media-editor,jquery-ui-position,wp-pointer,word-count,editor,jquery-ui-resizable,jquery-ui-dragg&load%5B%5D=able,jquery-ui-button,jquery-ui-dialog,wpdialogs,wplink,wpdialogs-popup,wp-fullscreen,media-upload&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,schedule,wp-ajax-response,autosave,suggest,jquery-color,wp-lists,jquery-ui-core,jquery-ui-&load%5B%5D=widget,jquery-ui-mouse,jquery-ui-sortable,postbox,post,underscore,shortcode,backbone,media-models,wp-plupload,media-views,media-&load%5B%5D=editor,jquery-ui-position,wp-pointer,word-count,editor,quicktags,jquery-ui-resizable,jquery-ui-draggable,jquery-ui-button,jquery&load%5B%5D=-ui-dialog,wpdialogs,wplink,wpdialogs-popup,wp-fullscreen,media-upload&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,suggest,inline-edit-post,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=admin-bar,hoverIntent,common,thickbox,wp-ajax-response,jquery-color,wp-lists,quicktags,jquery-query,admin-comments,jquery-ui-cor&load%5B%5D=e,jquery-ui-widget,jquery-ui-mouse,jquery-ui-sortable,postbox,dashboard,customize-base,customize-loader,plugin-install,underscor&load%5B%5D=e,shortcode,media-upload,jquery-ui-position,wp-pointer,editor,backbone,media-models,plupload,plupload-html5,plupload-flash,plupl&load%5B%5D=oad-silverlight,plupload-html4,wp-plupload,media-views,media-editor&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=hoverIntent,common,thickbox,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=jquery,utils,json2&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=jquery,utils,plupload,plupload-html5,plupload-flash,plupload-silverlight,plupload-html4,json2&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load%5B%5D=jquery,utils&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=admin-bar,hoverIntent,common,thickbox,plugin-install,jquery-ui-widget,jquery-ui-position,wp-pointer,jquery-ui-core,jquery-ui-dat&load[]=epicker&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=admin-bar,hoverIntent,common,thickbox,plugin-install,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=admin-bar,hoverIntent,common,thickbox,schedule,wp-ajax-response,autosave,jquery-color,wp-lists,quicktags,jquery-query,admin-comm&load[]=ents,suggest,jquery-ui-core,jquery-ui-widget,jquery-ui-mouse,jquery-ui-sortable,postbox,post,underscore,shortcode,backbone,media&load[]=-models,wp-plupload,media-views,media-editor,jquery-ui-position,wp-pointer,editor&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=admin-bar,hoverIntent,common,thickbox,schedule,wp-ajax-response,autosave,suggest,jquery-color,wp-lists,jquery-ui-core,jquery-ui-&load[]=widget,jquery-ui-mouse,jquery-ui-sortable,postbox,post,underscore,shortcode,backbone,media-models,wp-plupload,media-views,media-&load[]=editor,jquery-ui-position,wp-pointer,editor&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=admin-bar,hoverIntent,common,thickbox,suggest,inline-edit-post,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=admin-bar,hoverIntent,common,thickbox,wp-ajax-response,jquery-color,wp-lists,quicktags,jquery-query,admin-comments,jquery-ui-cor&load[]=e,jquery-ui-widget,jquery-ui-mouse,jquery-ui-sortable,postbox,dashboard,customize-base,customize-loader,plugin-install,underscor&load[]=e,shortcode,media-upload,jquery-ui-position,wp-pointer,editor,backbone,media-models,plupload,plupload-html5,plupload-flash,plupl&load[]=oad-silverlight,plupload-html4,wp-plupload,media-views,media-editor&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=hoverIntent,common,thickbox,jquery-ui-widget,jquery-ui-position,wp-pointer&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=jquery-ui-core,jquery-ui-widget,jquery-ui-mouse,jquery-ui-resizable,jquery-ui-draggable,jquery-ui-button,jquery-ui-position,jque&load[]=ry-ui-dialog&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=jquery,utils,json2&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=jquery,utils&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-scripts.php?c=1&load[]=jquery&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=0&dir=ltr&load=admin-bar,wp-admin,buttons,wp-pointer&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=0&dir=ltr&load=admin-bar,wp-admin,wp-pointer&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=0&dir=ltr&load=wp-admin,wp-pointer&ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=1&dir=ltr&load=admin-bar,buttons,media-views,wp-admin,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=1&dir=ltr&load=admin-bar,wp-admin,buttons,wp-pointer&ver=3.5 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=1&dir=ltr&load=admin-bar,wp-admin,buttons,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=1&dir=ltr&load=media-views&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=1&dir=ltr&load=wp-admin,buttons,wp-pointer&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/load-styles.php?c=1&dir=ltr&load=wp-jquery-ui-dialog&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-admin/options-general.php http://www.2rosenthals.net/wordpress/wp-admin/options-general.php?page=syntax-highlighter-compress.php http://www.2rosenthals.net/wordpress/wp-admin/options-general.php?page=wp-post-to-pdf/wp-post-to-pdf.php http://www.2rosenthals.net/wordpress/wp-admin/options.php http://www.2rosenthals.net/wordpress/wp-admin/plugin-editor.php?file=wptouch%2Fadmin-css%2Fbnc-compressed-global.css&plugin=wptouch%2Fwptouch.php http://www.2rosenthals.net/wordpress/wp-admin/plugin-editor.php?file=wptouch%2Fadmin-css%2Fbnc-global.css&plugin=wptouch%2Fadmin-css%2Fbnc-compressed-global.css http://www.2rosenthals.net/wordpress/wp-admin/plugin-install.php http://www.2rosenthals.net/wordpress/wp-admin/plugin-install.php?tab=plugin-information&plugin=print-me& http://www.2rosenthals.net/wordpress/wp-admin/plugin-install.php?tab=search&s=wp+post+to+pdf&plugin-search-input=Search+Plugins http://www.2rosenthals.net/wordpress/wp-admin/plugin-install.php?tab=search&type=term&s=email&plugin-search-input=Search+Plugins http://www.2rosenthals.net/wordpress/wp-admin/plugins.php http://www.2rosenthals.net/wordpress/wp-admin/plugins.php?action=activate&plugin=ckeditor-for-wordpress%2Fckeditor_wordpress.php&plugin_status=all&paged=1&s&_wpnonce=b0b02fa219 http://www.2rosenthals.net/wordpress/wp-admin/plugins.php?action=activate&plugin=wp-post-to-pdf%2Fwp-post-to-pdf.php&plugin_status=all&paged=1&s&_wpnonce=4cc7ee04d0 http://www.2rosenthals.net/wordpress/wp-admin/plugins.php?action=activate&plugin=wp-post-to-pdf%2Fwp-post-to-pdf.php&_wpnonce=4cc7ee04d0 http://www.2rosenthals.net/wordpress/wp-admin/plugins.php?action=deactivate&plugin=ckeditor-for-wordpress%2Fckeditor_wordpress.php&plugin_status=all&paged=1&s&_wpnonce=05d0ba2ffb http://www.2rosenthals.net/wordpress/wp-admin/plugins.php?action=deactivate&plugin=wp-post-to-pdf%2Fwp-post-to-pdf.php&plugin_status=all&paged=1&s&_wpnonce=dce3298d87 http://www.2rosenthals.net/wordpress/wp-admin/plugins.php?activate=true&plugin_status=all&paged=1&s= http://www.2rosenthals.net/wordpress/wp-admin/plugins.php?deactivate=true&plugin_status=all&paged=1&s= http://www.2rosenthals.net/wordpress/wp-admin/post-new.php http://www.2rosenthals.net/wordpress/wp-admin/post.php http://www.2rosenthals.net/wordpress/wp-admin/post.php?post=44&action=edit http://www.2rosenthals.net/wordpress/wp-admin/post.php?post=584&action=edit http://www.2rosenthals.net/wordpress/wp-admin/post.php?post=584&action=edit&message=10 http://www.2rosenthals.net/wordpress/wp-admin/post.php?post=584&action=edit&message=6 http://www.2rosenthals.net/wordpress/wp-admin/post.php?post=596&action=edit http://www.2rosenthals.net/wordpress/wp-admin/post.php?post=596&action=edit&message=10 http://www.2rosenthals.net/wordpress/wp-admin/theme-editor.php http://www.2rosenthals.net/wordpress/wp-admin/theme-editor.php?file=page.php&theme=lightword-2.0.0.6 http://www.2rosenthals.net/wordpress/wp-admin/theme-editor.php?file=yarpp-template-example.php&theme=lightword-child http://www.2rosenthals.net/wordpress/wp-admin/themes.php http://www.2rosenthals.net/wordpress/wp-admin/tools.php http://www.2rosenthals.net/wordpress/wp-admin/update-core.php http://www.2rosenthals.net/wordpress/wp-admin/update-core.php?action=do-core-upgrade http://www.2rosenthals.net/wordpress/wp-admin/update-core.php?action=do-plugin-upgrade http://www.2rosenthals.net/wordpress/wp-admin/update.php?action=install-plugin&plugin=wp-post-to-pdf&_wpnonce=99a20360a6 http://www.2rosenthals.net/wordpress/wp-admin/update.php?action=update-selected&plugins=akismet%2Fakismet.php&_wpnonce=e5c5f98ae2 http://www.2rosenthals.net/wordpress/wp-admin/update.php?action=update-selected&plugins=postie%2Fpostie.php&_wpnonce=9301f7883a http://www.2rosenthals.net/wordpress/wp-admin/update.php?action=update-selected&plugins=ultimate-tinymce%2Fmain.php%2Cyet-another-related-posts-plugin%2Fyarpp.php&_wpnonce=540719f73e http://www.2rosenthals.net/wordpress/wp-admin/update.php?action=update-selected&plugins=ultimate-tinymce%2Fmain.php&_wpnonce=5be9516d74 http://www.2rosenthals.net/wordpress/wp-admin/upgrade.php?step=1&backto=%2Fwordpress%2Fwp-admin%2Fupdate-core.php http://www.2rosenthals.net/wordpress/wp-admin/upgrade.php?_wp_http_referer=%2Fwordpress%2Fwp-admin%2Fupdate-core.php http://www.2rosenthals.net/wordpress/wp-content/plugins/akismet/akismet.css?ver=2.5.4.4 http://www.2rosenthals.net/wordpress/wp-content/plugins/akismet/akismet.js?ver=2.5.4.6 http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/SyntaxHighlighter/build/scripts/shBrushBash-min.js http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/SyntaxHighlighter/build/scripts/shBrushPlain-min.js http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/SyntaxHighlighter/build/scripts/shBrushXml-min.js http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/SyntaxHighlighter/build/styles/shThemeDefault-min.css?ver=3.0.83 http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/tinymce/ash/ash_editor.html?ver=2.3.3&ver=358-23224 http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/tinymce/ash/ash_fullscreen.css?ver=2.3.3 http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/tinymce/ash/editor_plugin.js?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/tinymce/ash/img/ash.gif http://www.2rosenthals.net/wordpress/wp-content/plugins/auto-syntaxhighlighter/tinymce/ash/js/ash.js?ver=2.3 http://www.2rosenthals.net/wordpress/wp-content/plugins/better-plugin-compatibility-control/img/info.gif http://www.2rosenthals.net/wordpress/wp-content/plugins/better-plugin-compatibility-control/js/bbpc_dom.3.1.js?ver=3.5 http://www.2rosenthals.net/wordpress/wp-content/plugins/better-plugin-compatibility-control/js/bbpc_dom.3.1.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/changelogger/css/style.css?ver=1.2.15 http://www.2rosenthals.net/wordpress/wp-content/plugins/changelogger/js/admin_scripts.js?ver=1.2.15 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/ckeditor.js?t=CBDD&ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/ckeditor.js?ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/ckeditor.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor.config.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/contents.css http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/lang/en.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/plugins/fakeobjects/images/spacer.gif?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/plugins/icons.png?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/plugins/smiley/images/whatchutalkingabout_smile.gif http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/plugins/smiley/images/wink_smile.gif http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/skins/kama/editor.css?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/skins/kama/icons.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor/skins/kama/skin.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/ckeditor.styles.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/ckeditor_ico16.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/ckeditor_ico32.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/compat_logos.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/configuration.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/documentation.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/exclamation.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/help.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/menuicon.gif http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/images/plugin.png http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/includes/basic.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/includes/ckeditor.comment-reply.js?ver=20100901 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/includes/ckeditor.utils.js?ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/includes/ckeditor.utils.js?ver=3.5 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/includes/ckeditor.utils.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/includes/overview.css?ver=4.0 http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/plugins/wpgallery/plugin.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/plugins/wpmore/images/more_bug.gif?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/plugins/wpmore/images/more.gif?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/ckeditor-for-wordpress/plugins/wpmore/plugin.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/contact-form-7/admin/images/menu-icon.png http://www.2rosenthals.net/wordpress/wp-content/plugins/contact-form-7/includes/css/styles.css?ver=3.3.3 http://www.2rosenthals.net/wordpress/wp-content/plugins/contact-form-7/includes/js/jquery.form.min.js?ver=3.25.0-2013.01.18 http://www.2rosenthals.net/wordpress/wp-content/plugins/contact-form-7/includes/js/scripts.js?ver=3.3.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/contact-form-7/includes/js/scripts.js?ver=3.3.2 http://www.2rosenthals.net/wordpress/wp-content/plugins/contact-form-7/includes/js/scripts.js?ver=3.3.3 http://www.2rosenthals.net/wordpress/wp-content/plugins/link-library/icons/folder-beige-internet-icon.png http://www.2rosenthals.net/wordpress/wp-content/plugins/mtc-ckeditor-link-page/ckeplugin/plugin.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/post-type-switcher/js/quickedit.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/subscribe2/include/email_edit.png http://www.2rosenthals.net/wordpress/wp-content/plugins/subscribe2/include/s2_button.png http://www.2rosenthals.net/wordpress/wp-content/plugins/subscribe2/tinymce3/css/content.css?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-content/plugins/subscribe2/tinymce3/editor_plugin.min.js?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-content/plugins/syntaxhighlighter-ckeditor-button/syntaxhighlight/dialogs/syntaxhighlight.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/syntaxhighlighter-ckeditor-button/syntaxhighlight/images/syntaxhighlight.gif?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/syntaxhighlighter-ckeditor-button/syntaxhighlight/lang/en.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/syntaxhighlighter-ckeditor-button/syntaxhighlight/plugin.js?t=CBDD http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/scripts/shAutoloader.js http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/scripts/shBrushCss.js http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/scripts/shBrushPhp.js http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/scripts/shBrushXml.js http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/scripts/shCore.js http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/styles/shCoreDefault.css http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/tinymce/editor_plugin.js?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/tinymce/shc.gif http://www.2rosenthals.net/wordpress/wp-content/plugins/syntax-highlighter-compress/tinymce/window.php?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-content/plugins/theme-my-login/modules/themed-profiles/themed-profiles.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/theme-my-login/theme-my-login.css?ver=6.2.3 http://www.2rosenthals.net/wordpress/wp-content/plugins/theme-my-login/theme-my-login.css?ver=6.3.8 http://www.2rosenthals.net/wordpress/wp-content/plugins/ultimate-tinymce/emotions/img/popobig/beat_ball.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wordpress-admin-bar-improved/wpabi.css?ver=2.0 http://www.2rosenthals.net/wordpress/wp-content/plugins/wordpress-admin-bar-improved/wpabi.js?ver=1.0 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-appstore/assets/jquery.tools.min.js?ver=20110322 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-appstore/assets/scrollable-buttons.css?ver=20110322 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-appstore/assets/scrollable-horizontal.css?ver=20110322 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-appstore/wp-appstore.css?ver=20110322 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/css/overcast/images/ui-bg_inset-soft_50_c9c9c9_1x100.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/css/overcast/jquery-ui-1.8.21.custom.css?ver=3.5 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/css/overcast/jquery-ui-1.8.21.custom.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/css/style.css?ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/css/style.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/js/jquery-ui-timepicker-addon.js?ver=02-22-2013 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/js/wp-maintenance-mode.js?ver=3.5 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-maintenance-mode/js/wp-maintenance-mode.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-post-to-pdf/asset/css/admin.css?ver=1.0 http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-post-to-pdf/asset/images/pdf.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wp-post-to-pdf/asset/images/toggle.gif http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/images/good.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/images/icon-pool/Contacts.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/images/icon-pool/Home.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/images/icon-pool/Mail.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/images/icon-pool/Notes.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/images/icon-pool/RSS.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/images/icon-pool/ToDo.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-css/wptouch-switch-link.css http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/backgrounds/low_contrast_linen.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/bookmarks/digg.jpg http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/bookmarks/magnolia.jpg http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/bookmarks/reddit.jpg http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/bookmarks/technorati.jpg http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/head-close.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/head-fade-bk.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/core/core-images/menu-sprite.png http://www.2rosenthals.net/wordpress/wp-content/plugins/wptouch/themes/default/style.css http://www.2rosenthals.net/wordpress/wp-content/plugins/yet-another-related-posts-plugin/js/metabox.js?ver=4.0.6 http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/css/admin.min.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/css/aye-admin.min.css?ver=3.5 http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/css/aye-main.min.css?ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/css/aye-main.min.css?ver=3.5 http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/css/main.min.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/images/admin_menu_icon.png http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/images/menu_icon.png http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/images/youtube_button_b&w.png http://www.2rosenthals.net/wordpress/wp-content/plugins/youtube-embed/js/mce-button.min.php?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-content/themes/carrington-mobile-1.0.2/screenshot.png http://www.2rosenthals.net/wordpress/wp-content/themes/classic/screenshot.png http://www.2rosenthals.net/wordpress/wp-content/themes/default/screenshot.png http://www.2rosenthals.net/wordpress/wp-content/themes/leathernote/screenshot.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword-2.0.0.6/screenshot.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword-child/screenshot.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword-child/style.css?ver=1.0.0.0 http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/date_comm_box.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/nav.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/older_newer.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/rss.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/searchbox.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/sidebar_h3.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/submit_btn.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/wider/content_bottom.gif http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/wider/content_bottom.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/wider/content_middle.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/wider/content_top.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/images/wider/tags_category.png http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/js/cufon-yui.js http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/js/menu.js http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/js/tabs.js http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/js/vera.font.js http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/style.css http://www.2rosenthals.net/wordpress/wp-content/themes/lightword/wider.css?ver=1.0.0.0 http://www.2rosenthals.net/wordpress/wp-content/themes/twentyeleven/screenshot.png http://www.2rosenthals.net/wordpress/wp-content/uploads/2011/01/net_wan_cloud.svg_.hi_-300x158.png http://www.2rosenthals.net/wordpress/wp-content/uploads/2011/01/net_wan_cloud.svg_.hi_.png http://www.2rosenthals.net/wordpress/wp-content/uploads/2011/11/LewisR_avatar-16x16.jpg http://www.2rosenthals.net/wordpress/wp-content/uploads/2011/11/LewisR_avatar-50x50.jpg http://www.2rosenthals.net/wordpress/wp-content/uploads/2011/11/LewisR_avatar-64x64.jpg http://www.2rosenthals.net/wordpress/wp-content/uploads/2011/11/LewisR_avatar.jpg http://www.2rosenthals.net/wordpress/wp-content/uploads/2012/05/PDXESES_Embassy_Suites_Portland-Airport_gallery_welcome-300x234.jpg http://www.2rosenthals.net/wordpress/wp-content/uploads/2012/07/WPTouch_ShareBar_modified.png http://www.2rosenthals.net/wordpress/wp-content/uploads/2012/10/BetterPrivacy-logo.png http://www.2rosenthals.net/wordpress/wp-content/uploads/2012/11/Writer-Columns-properties.jpg http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/1147162443.png http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/1343786788.png http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/154246084.png http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/1695641856.png http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/514117729.png http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/542904053.png http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/826095337.png http://www.2rosenthals.net/wordpress/wp-content/uploads/wpcf7_captcha/941254618.png http://www.2rosenthals.net/wordpress/wp-includes/css/admin-bar.min.css?ver=3.5 http://www.2rosenthals.net/wordpress/wp-includes/css/admin-bar.min.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-includes/css/buttons.min.css?ver=3.5 http://www.2rosenthals.net/wordpress/wp-includes/css/editor.css?ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-includes/css/editor.min.css?ver=3.5 http://www.2rosenthals.net/wordpress/wp-includes/css/editor.min.css?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-includes/images/admin-bar-sprite-2x.png?d=20120830 http://www.2rosenthals.net/wordpress/wp-includes/images/admin-bar-sprite.png?d=20120830 http://www.2rosenthals.net/wordpress/wp-includes/images/blank.gif http://www.2rosenthals.net/wordpress/wp-includes/images/down_arrow.gif http://www.2rosenthals.net/wordpress/wp-includes/images/smilies/icon_wink.gif http://www.2rosenthals.net/wordpress/wp-includes/images/wpicons.png?ver=20120720 http://www.2rosenthals.net/wordpress/wp-includes/js/admin-bar.min.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-includes/js/comment-reply.min.js?ver=3.5.1 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/jquery.js?ver=1.7.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/jquery.js?ver=1.8.3 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.button.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.core.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.dialog.min.js?ver=1.8.20 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.dialog.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.draggable.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.mouse.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.position.min.js?ver=1.8.20 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.position.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.resizable.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/jquery/ui/jquery.ui.widget.min.js?ver=1.9.2 http://www.2rosenthals.net/wordpress/wp-includes/js/thickbox/loadingAnimation.gif http://www.2rosenthals.net/wordpress/wp-includes/js/thickbox/thickbox.css?ver=20121105 http://www.2rosenthals.net/wordpress/wp-includes/js/thickbox/thickbox.css?ver=3.4.2 http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/langs/wp-langs-en.js?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/inlinepopups/skins/clearlooks2/img/buttons.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/inlinepopups/skins/clearlooks2/img/corners.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/inlinepopups/skins/clearlooks2/img/drag.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/inlinepopups/skins/clearlooks2/img/horizontal.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/inlinepopups/skins/clearlooks2/window.css?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/spellchecker/css/content.css http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/wordpress/img/trans.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/wpeditimage/img/delete.png http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/wpeditimage/img/image.png http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/wpgallery/img/delete.png http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/plugins/wpgallery/img/edit.png http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/themes/advanced/img/icons.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/themes/advanced/skins/wp_theme/content.css http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/themes/advanced/skins/wp_theme/dialog.css?ver=358-23224 http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/themes/advanced/skins/wp_theme/img/more_bug.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/themes/advanced/skins/wp_theme/img/tabs.gif http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/tiny_mce_popup.js http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/tiny_mce_popup.js?ver=345-20110908 http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/utils/form_utils.js http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/utils/form_utils.js?ver=345-20110908 http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/utils/mctabs.js http://www.2rosenthals.net/wordpress/wp-includes/js/tinymce/wp-tinymce.php?c=1&ver=358-23224 http://www.2rosenthals.net/wordpress/wp-includes/js/tw-sack.js?ver=1.6.1 http://www.2rosenthals.net/wordpress/wp-includes/js/tw-sack.min.js?ver=1.6.1 http://www.2rosenthals.net/wordpress/wp-login.php http://www.2rosenthals.net/wordpress/?wptouch_view=normal&wptouch_redirect_nonce=0d414bafc5&wptouch_redirect=%2Fwordpress%2Fbetterprivacy-mod-for-seamonkey-532%2F http://www.2rosenthals.net/wordpress/wrestling-with-apaches-mod_rewrite-374/ http://www.2rosenthals.net/wordpress/xmlrpc.php http://www.2rosenthals.net/wordpress/xmlrpc.php?rsd siegeurls1 (78,190 bytes) php.ini (69,783 bytes) httpd.conf (20,298 bytes) httpd-mpm.conf (4,606 bytes) httpd-php.conf (862 bytes)

LewisR 2014-12-20 09:01 developer ~0002940	Steve followed up: After some tweaking of config.sys to optimized what is getting loaded, we did a siege run. The result was a number of apparently stuck php.exe instances. Lewis captured a "full" dump with pdumpctl of one of the stuck instances. The dump shows that php.exe is stuck trying to write to stdout. At this point I am assuming that stdout is suppposed to be connected to a httpd thread and that the thread is not reading the pipe for an as yet unknown reason. What is a bit confusing is the content of the write buffer. It does not look right. I expected to see CGI output which tends to be text and what I see is binary. # %kx 005b:1e899864 00000001 0055a72c 00009e88 0012e7b0 [LIBC066 __write + 64] 005b:1e8c63ac 00000001 0055a72c 00009e88 03000000 [LIBC066 _std_write + 12c] 005b:00011067 00000001 0055a72c 00009e88 1d38b2df 005b:1d38c7b2 0055a72c 00009e88 00164160 00000000 [_php_output_write + 149] 005b:1d370120 00550898 00009e88 00164160 00478b14 [_tsrm_realpath + 29C] 005b:1d3f48ed 00550898 00009e88 0012e90c 1d36cb55 [_zend_print_zval_ex + 51] 005b:1d3f4944 1d3700f0 00478b14 00000000 1d470979 [_zend_print_zval + 21] 005b:1d3f1e52 00478b14 00000000 001906b8 0018f654 [_zend_print_variable + 19] 005b:1d441d9e 00478b14 0016da14 00191c28 00164160 [_execute + 2216A] 005b:1d41fe4b 0016dc14 00000000 00000004 1d401623 [_execute + 217] 005b:1d3f4ed1 0018a238 00164160 0000004b 0012ea3c [_zend_execute_scripts + 7A] 005b:1d375158 00000008 00164160 00000000 00000003 [_php_execute_script + 1E8] 005b:000148b0 0012fe6c 00164160 00000000 00000000 005b:00010021 00000001 0012ff9c 20030180 ffffffff 005b:1e88887b 00000000 1ffece38 0000115e 00000000 [LIBC066 __init_app + b] This says we are stuck at write(1, 0x55a72c, 0x9e88) and call gated to %1ffc6f84 DOSCALL1 DOS32IWRITE The buffer begins # dd 0055a72c 0053:0055a72c 00088b1f 00000000 fdd50600 48db7aeb 0053:0055a73c ff0c2d92 682afafe 69176f4e 27d1d441 0053:0055a74c 6cb714ca b69a55cb 7725b72d d2a4b775 0053:0055a75c c4a09103 08b00932 35128752 15aefed7 0053:0055a76c 80489991 9eeaa4a0 3dd9ef99 24011653 0053:0055a77c c8ce7912 fe2b1588 9dc69bd4 2b893f49 0053:0055a78c 5be4d1e7 a20f8d1a 9f1fa4c9 ace0c2d7 0053:0055a79c 8e197b7e 9e169d2b de060cec df56b4c3

LewisR 2014-12-20 09:09 developer ~0002941	Paul: one clue is I found a setmode(fd, O_BINARY) which is OS2 specific in main\streams\plain_wrapper.c A build of php 5.4.34 php5.dll that removes this is at http://smedley.id.au/tmp/php-5.4.34-dll-20141031.zip Steve: I guess it's worth a test, although I don't expect much difference. There's a slight performance boost when running O_BINARY because less data is received and sent. Where possible, I tend to run O_BINARY unless the handle is a console handle. Paul: http://smedley.id.au/tmp/php-5.4.34-20141101.zip is a full rebuild which also contains map files for cli and cgi php.exe Lewis: The new php5.4.34 build seemed to lock up on me to the point where I could not even get a dump file. Paul: http://smedley.id.au/tmp/php-5.4.34-20141101b.zip reverts the os2 specific implementation of socket non-blocking.... Lewis: pdf.dll needs to be recompiled for the difference in the API with this new PHP build. Paul: I would have expected the PHP 5.4 build of pdf.dll to work for all 5.4.x versions Anyhow, http://smedley.id.au/tmp/pdf.zip contains a rebuild Lewis: I have no idea where any of these things may lead us, but the issue is decidedly worse with the 20141101b build (got stuck twice, unable to dump). Paul: I've reverted the removal of setmode() in the following - which should get us back to the same state as the original build of 5.4.34 http://smedley.id.au/tmp/php-5.4.34-20141101c.zip

LewisR 2014-12-20 09:15 developer ~0002942 Last edited: 2014-12-20 09:20	Steve: When you get to the state where the php instances are hung let's see if we can figure out how the pipes are associated.. Run Theseus2 and use Process->General Process Information and look at the open files for one of the stuck instances. Note the system handle numbers for process handles 0, 1 and 2. Then view the process information for the httpd processes and figure out which process (if any) has the other side of the pipe open. If you have trouble with this, save the process information windows and the system file list window to files and let me take a look. This is somewhat non-trivial since each side of the pipe has a different system file handle so it's a process of elimination. [...] Looking at Theseus2 output for php-cgi we have: Process System Flags File name // Flags... 0000 0241 00 named pipe // 0001 01AF 00 named pipe // 0002 0242 00 named pipe // 0007 005D 00 named pipe // and for httpd we have Process System Flags File name // Flags... 0000 004F 08 \DEV\CON // console 0001 004F 08 \DEV\CON // console 0002 0120 00 J:\APPS\APACHE2\LOGS\ERROR_LOG // 0007 005D 00 named pipe // I need to think about this a bit, but it appears reasonable. I think that the php process should have closed handle 7, but this is not related to the problem at hand. I need to figure out how to extract the pipe name from the dump data. [...] The contents are as expected. The process is hung at the DosWrite. I haven't yet figured out how to derive the pipe name from the handle. It might be easier to add some debug code to httpd to log when it kicks off a CGI process. I need to poke about in the source code... --- We decided to use OS2TRACE to attempt to determine which pipes were not getting connected. See attached testing-http-php-with-os2trace-20141108.txt for procedures.

LewisR 2014-12-20 09:22 developer ~0002943	Steve: I think I found our problem: Take a look at PROC002E.TRC and find the entries for PID/TID 002E 0006 which is the thread that's talking to the fung php PID 0033. If you look at line 2031655, you will find the log message: 002E 0006 \| Dos32Write Entry, Return Address = 0x1DA6EAD3 (HTTPD 0001:0000EAD3) \| Parameter 1: HFILE = 0x00000002 \| Parameter 2: PVOID = 0x02A759EC \| 0x02A759EC: 5B 53 75 6E 20 4E 6F 76 20 30 39 20 30 31 3A 31 [Sun Nov 09 01:1 \| 0x02A759FC: 39 3A 35 32 20 32 30 31 34 5D 20 5B 65 72 72 6F 9:52 2014] [erro \| 0x02A75A0C: 72 5D 20 5B 63 6C 69 65 6E 74 20 31 39 32 2E 31 r] [client 192.1 \| 0x02A75A1C: 36 38 2E 31 30 30 2E 38 5D 20 53 63 72 69 70 74 68.100.8] Script \| 0x02A75A2C: 20 74 69 6D 65 64 20 6F 75 74 20 62 65 66 6F 72 timed out befor \| 0x02A75A3C: 65 20 72 65 74 75 72 6E 69 6E 67 20 68 65 61 64 e returning head \| 0x02A75A4C: 65 72 73 3A 20 70 68 70 2E 65 78 65 0D 0A ers: php.exe.. \| Parameter 3: ULONG = 0x0000006E \| Parameter 4: PULONG = 0x02A75934 Me thinks the timeouts are too low. FWIW, this message has been in the logs all along, but probably was easy to miss among the thousands of other messages. The failing code is probably near mod_ext_filter.c:829: rv = apr_file_read(ctx->proc->out, The message is coming from util_script.c:443 else if (rv == -1) { ap_log_rerror(APLOG_MARK, APLOG_ERR\|APLOG_TOCLIENT, 0, r, "Script timed out before returning headers: %s", apr_filepath_name_get(r->filename));

LewisR 2014-12-20 09:36 developer ~0002944 Last edited: 2014-12-20 09:38	Paul: I'd guess it could be the same error condition found in testpipe, failing when no data is read from a pipe..... [Ref: https://groups.google.com/forum/#!topic/apache2/nq7h4_kQnYE ] Steve: I'm pretty sure it is. The apr code is abusing the pipe semaphore. The was the semaphore is supposed to be used is as a signal that it's time to read from the pipe. The apr code is using the semaphore to wait until retrying the read. The problem is that this assumes a lot about the order that thing occur. What you might want to try is do the read if the read returns no data reset the event semaphore wait for the semaphore retry the read if there's data, good if the semaphore timed out fail if the semaphore wait was interrupted try again This way a lost event does can cause spurious errors. At worst the read will be delayed for a bit. It's not all that well documented what determines if an can get interrupted and return ERROR_INTERRUPT. I go by the rule of thumb that if the API can has a wait associated with it and can be safely restarted if interrupted, it can return ERROR_INTERRUPT. If an API lists ERROR_INTERRUPT as a possible return, it's a good idea to check for it. Paul: To be clear, this is the code I believe is being used (and failing): /* Unbuffered i/o / apr_size_t nbytes; if (thefile->pipe) DosResetEventSem(thefile->pipeSem, &rc); rc = DosRead(thefile->filedes, buf, len, &nbytes); if (rc == ERROR_NO_DATA && thefile->timeout != 0) { int rcwait = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); if (rcwait == 0) { rc = DosRead(thefile->filedes, buf, len, &nbytes); } else if (rcwait == ERROR_TIMEOUT) { len = 0; return APR_TIMEUP; } } I did try removing: if (thefile->pipe) DosResetEventSem(thefile->pipeSem, &rc); and it made no difference. Steve: It's my current guess that the timeouts [during siege tests and referenced in the above Apache/2 list thread with testall] both occurred for the same reason. A timing race in the apr_file_read code that's supposed to deal with non-blocking pipes. [...] Take a look at the attached [readwrite2.c.diff]. My change is near line 92 of the diff. The idea is that you read until the other side closes the pipe or somethingbad happens and there's a timeout.

LewisR 2014-12-20 09:42 developer ~0002945	Steve [upon revising original readwrite.c.diff]: It occurred to me that we need to map ERROR_MORE_DATA to NO_ERROR because, like ERROR_NO_DATA, it is a notice, not an error. It means that the caller's buffer is full and the pipe is not empty. I also added a bit more code that should ensure that we don't confuse no data currently available with end of data. What I'd like to know for sure is if NO_ERROR and count = 0 always means end of data, even in the non-blocking case. This is possible if the kernel ensures ERROR_NO_DATA is always returned if an open, non-blocking pipe has no data available. The docs are not 100% clear if this is the case. Could you instrument testpipe and readchild to trace as described in "testing-http-php-with-os2trace.txt" and see what happens. You don't need to instrument libc066 because apr is using native DOS APIs. Since we care about the semaphore activity, add SEM to the selected DOSCALLS API groups. Paul: Unfortunately no change. Steve: Not on the gross pass/fail level, but the .trc files show a lot. I think we are close to resolving the httpd issues. The testall failure is something else. Here's what's happening. Testall fails with 0CC1 0001 \| Dos32Close Entry, Return Address = 0x00040459 (TESTALL 0001:00030459) \| Parameter 1: HFILE = 0x00000006 0CC1 0001 \| Dos32Close Exit PASS \| Return code: 0 (NO_ERROR) 0CC1 0001 \| Dos32ResetEventSem Entry, Return Address = 0x00040B03 (TESTALL 0001:00030B03) \| Parameter 1: HEV = 0x80010187 \| Parameter 2: PULONG = 0x001CFDAC 0CC1 0001 \| Dos32ResetEventSem Exit FAIL \| Return code: 300 (ERROR_ALREADY_RESET) \| Parameter 2: PULONG = 0x001CFDAC [0x00000000] 0CC1 0001 \| Dos32Read Entry, Return Address = 0x00040B28 (TESTALL 0001:00030B28) \| Parameter 1: HFILE = 0x00000007 \| Parameter 2: PVOID = 0x001CFE28 \| Parameter 3: ULONG = 0x00000080 \| Parameter 4: PULONG = 0x001CFDB0 0CC1 0001 \| Dos32Read Exit FAIL \| Return code: 232 (ERROR_NO_DATA) \| Parameter 2: PVOID = 0x001CFE28 \| Parameter 4: PULONG = 0x001CFDB0 [0x00000000] 0CC1 0001 \| Dos32WaitEventSem Entry, Return Address = 0x00040BB3 (TESTALL 0001:00030BB3) \| Parameter 1: HEV = 0x80010187 \| Parameter 2: ULONG = 0x00002710 0CC1 0001 \| Dos32WaitEventSem Exit FAIL \| Return code: 640 (ERROR_TIMEOUT) \| API entry at 09:58:50.18, exit at 09:59:00.20 The code is doing what is supposed to - wait 10 seconds for semaphore to be posted when the read reports no data. The problem is on the other side, the readchild trace reports 0CC2 0001 \| Dos32Read Entry, Return Address = 0x00012928 (READCHIL 0001:00002928) \| Parameter 1: HFILE = 0x00000000 \| Parameter 2: PVOID = 0x0013FEF8 \| Parameter 3: ULONG = 0x00000080 \| Parameter 4: PULONG = 0x0013FEB0 0CC2 0001 \| Dos32Read Exit PASS \| Return code: 0 (NO_ERROR) \| Parameter 2: PVOID = 0x0013FEF8 \| 0x0013FEF8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| 0x0013FF08: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| 0x0013FF18: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| 0x0013FF28: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| 0x0013FF38: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| 0x0013FF48: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| 0x0013FF58: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| 0x0013FF68: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ \| Parameter 4: PULONG = 0x0013FEB0 [0x00000080] 0CC2 0001 \| Dos32Read Entry, Return Address = 0x00012928 (READCHIL 0001:00002928) \| Parameter 1: HFILE = 0x00000000 \| Parameter 2: PVOID = 0x0013FEF8 \| Parameter 3: ULONG = 0x00000080 \| Parameter 4: PULONG = 0x0013FEB0 READCHILD.EXE (0CC2) stopping at 09:59:05.07 on 11/12/2014 Readchild is still busy reading data when testall decides it has waited long enough for readchild to respond. It's almost as if readchild does not get informed that the testpipe has closed the other end of the pipe. I need to think about what, if anything, that the APR might be doing to cause this. Note that on the readchild side stdin looks like a file (i.e. the pipe flag is not set) so reads are normal blocking reads.

LewisR 2014-12-20 09:46 developer ~0002946	Paul: Given your comments I'll rebuild httpd.dll anyway and let Lewis run the siege tests. Steve: Good idea. If it works, I will tune my patch to avoid what I believe is an extra sem wait, which will improve performance. I found some additional references to ERROR_NO_DATA in the CPG&R docs that strenghten my belief that the statement found elsewhere that states <snip> If you attempt a read of a named pipe when the other end of the pipe is closed while the read is pending, DosRead returns: pcbActual equal to 0 ulrc equal to either 0 or ERROR_NO_DATA </snip> is probably not correct. We sould be able to depend on NO_ERROR with actual count = 0 meaning that the other side has closed the pipe. [Paul built test build of httpd.dll.] Steve: Assuming nothing really bad shows up with your last build, here's the next patch that I would like to test [readwrite3.c.diff]. It assumes that NO_ERROR and count = 0 indicates end of data and bypasses the semaphore wait.

LewisR 2014-12-20 09:53 developer ~0002947	Lewis [upon testing Paul's build before readwrite3.c.diff]: Fewer stuck ones [PHP processes] than before, but I have not looked to see how these compare to the last batch. Steve: Better, but not fixed. My testing indicates that the last patch I posted is needed to ensure that end of data is seen on the httpd side. Paul: Build with Steven's latest patch: http://smedley.id.au/tmp/httpd-dll-20141114.zip Steve: Ignoring Lewis's issues, my traces look exactly like I expect except for one thing. This from the httpd .trc: 0C27 0003 \| Dos32Read Entry at 19:41:05.88, Return Address = 0x0F4AE788 (HTTPD 0001:0000E788) \| Parameter 1: HFILE = 0x0000000E \| Parameter 2: PVOID = 0x006DAD68 \| Parameter 3: ULONG = 0x00001F40 \| Parameter 4: PULONG = 0x02829A6C 0C27 0003 \| Dos32Read Exit at 19:41:05.88 PASS \| Return code: 0 (NO_ERROR) \| Parameter 2: PVOID = 0x006DAD68 \| 0x006DAD68: 58 2D 50 6F 77 65 72 65 64 2D 42 79 3A 20 50 48 X-Powered-By: PH \| 0x006DAD78: 50 2F 35 2E 34 2E 33 34 0D 0A 43 6F 6E 74 65 6E P/5.4.34..Conten \| 0x006DAD88: 74 2D 74 79 70 65 3A 20 74 65 78 74 2F 68 74 6D t-type: text/htm \| 0x006DAD98: 6C 0D 0A 0D 0A 3C 68 74 6D 6C 3E 0D 0A 20 3C 68 l....<html>.. <h \| 0x006DADA8: 65 61 64 3E 0D 0A 20 20 3C 74 69 74 6C 65 3E 50 ead>.. <title>P \| 0x006DADB8: 48 50 20 54 65 73 74 20 2D 20 6F 73 32 74 72 61 HP Test - os2tra \| 0x006DADC8: 63 65 3C 2F 74 69 74 6C 65 3E 0D 0A 20 3C 2F 68 ce</title>.. </h \| 0x006DADD8: 65 61 64 3E 0D 0A 20 3C 62 6F 64 79 3E 0D 0A 20 ead>.. <body>.. \| 0x006DADE8: 20 20 48 65 6C 6C 6F 20 66 72 6F 6D 20 4D 72 2E Hello from Mr. \| 0x006DADF8: 20 68 65 6C 6C 6F 2E 70 68 70 27 73 20 68 74 6D hello.php's htm \| 0x006DAE08: 6C 20 63 6F 64 65 2E 0D 0A 20 20 20 3C 70 3E 48 l code... H \| 0x006DAE18: 65 6C 6C 6F 20 66 72 6F 6D 20 4D 72 2E 20 68 65 ello from Mr. he \| 0x006DAE28: 6C 6C 6F 2E 70 68 70 27 73 20 50 48 50 20 63 6F llo.php's PHP co \| 0x006DAE38: 64 65 2E 0A 3C 2F 70 3E 20 3C 2F 62 6F 64 79 3E de.. </body> \| 0x006DAE48: 0D 0A 3C 2F 68 74 6D 6C 3E 0D 0A ..</html>.. \| Parameter 4: PULONG = 0x02829A6C [0x000000EB] At this point the read apr_file_read() returns to the caller and the caller calls apr_file_read() for more data... 0C27 0003 \| Dos32ResetEventSem Entry at 19:41:05.88, Return Address = 0x0F4AE763 (HTTPD 0001:0000E763) \| Parameter 1: HEV = 0x8001019A \| Parameter 2: PULONG = 0x0282BA28 0C27 0003 \| Dos32ResetEventSem Exit at 19:41:05.88 FAIL \| Return code: 300 (ERROR_ALREADY_RESET) \| Parameter 2: PULONG = 0x0282BA28 [0x00000000] 0C27 0003 \| Dos32Read Entry at 19:41:05.88, Return Address = 0x0F4AE788 (HTTPD 0001:0000E788) \| Parameter 1: HFILE = 0x0000000E \| Parameter 2: PVOID = 0x006DEE28 \| Parameter 3: ULONG = 0x00001F40 \| Parameter 4: PULONG = 0x0282BA2C 0C27 0003 \| Dos32Read Exit at 19:41:05.88 FAIL \| Return code: 232 (ERROR_NO_DATA) \| Parameter 2: PVOID = 0x006DEE28 \| Parameter 4: PULONG = 0x0282BA2C [0x00000000] This is OK because php.exe has no more data to send and has not yet shut itself down. However, this is really wierd: 0C27 0003 \| Dos32RequestMutexSem Entry at 19:41:05.88, Return Address = 0x0F4C7CA9 (HTTPD 0001:00027CA9) \| Parameter 1: HMTX = 0x0001006C \| Parameter 2: ULONG = 0xFFFFFFFF 0C27 0003 \| Dos32RequestMutexSem Exit at 19:41:05.88 PASS \| Return code: 0 (NO_ERROR) \| API entry at 19:41:05.88, exit at 19:41:05.88 0C27 0003 \| Dos32ReleaseMutexSem Entry at 19:41:05.88, Return Address = 0x0F4C7D0D (HTTPD 0001:00027D0D) \| Parameter 1: HMTX = 0x0001006C 0C27 0003 \| Dos32ReleaseMutexSem Exit at 19:41:05.88 PASS \| Return code: 0 (NO_ERROR) My sources contain no mutex request. I don't have a current map, but I suspect the call is from apr_thread_mutex_lock() 0C27 0003 \| Dos32RequestMutexSem Entry at 19:41:05.88, Return Address = 0x0F4C7CA9 (HTTPD 0001:00027CA9) \| Parameter 1: HMTX = 0x0001006C \| Parameter 2: ULONG = 0xFFFFFFFF 0C27 0003 \| Dos32RequestMutexSem Exit at 19:41:05.88 PASS \| Return code: 0 (NO_ERROR) \| API entry at 19:41:05.88, exit at 19:41:05.88 0C27 0003 \| Dos32ReleaseMutexSem Entry at 19:41:05.88, Return Address = 0x0F4C7D0D (HTTPD 0001:00027D0D) \| Parameter 1: HMTX = 0x0001006C 0C27 0003 \| Dos32ReleaseMutexSem Exit at 19:41:05.88 PASS \| Return code: 0 (NO_ERROR) Here we loop back to the top a try the read again... 0C27 0003 \| Dos32ResetEventSem Entry at 19:41:05.93, Return Address = 0x0F4AE763 (HTTPD 0001:0000E763) \| Parameter 1: HEV = 0x8001019A \| Parameter 2: PULONG = 0x0282BA28 0C27 0003 \| Dos32ResetEventSem Exit at 19:41:05.93 FAIL \| Return code: 300 (ERROR_ALREADY_RESET) \| Parameter 2: PULONG = 0x0282BA28 [0x00000000] 0C27 0003 \| Dos32Read Entry at 19:41:05.93, Return Address = 0x0F4AE788 (HTTPD 0001:0000E788) \| Parameter 1: HFILE = 0x0000000E \| Parameter 2: PVOID = 0x006DAD68 \| Parameter 3: ULONG = 0x00001F40 \| Parameter 4: PULONG = 0x0282BA2C 0C27 0003 \| Dos32Read Exit at 19:41:05.93 FAIL \| Return code: 232 (ERROR_NO_DATA) \| Parameter 2: PVOID = 0x006DAD68 \| Parameter 4: PULONG = 0x0282BA2C [0x00000000] This loop continues on until the read returns NO_ERROR and count 0 Paul, do you have any idea where the mutex calls are coming from? Lewis: Looks like we're moving in the right direction. Fewer stuck processes, and the PHP instances were killable if I first used the normal kill from top and then the force kill option. httpd instances did not want to shutdown from -k, but they did kill, all but one last child, which I had to take out with Ctrl-C. Steve: That is better. You still seem to have the stuck write issue. The problem seems to be on the httpd end because I can see the timed out messages going to the log. It's hard to believe the php code is this slow, but maybe it is Let's try this. Add SEM to the traced items and Use Customize->Timestamping and enable timestamping. This will allow us the see when php writes relative to when httpd times out. Paul [responding to Steve's earlier question about the origin of the mutex calls]: grepping apache2 and apr source, DosRequestMutexSem could be from: Apache2: server\mpm\mpmt_os2\mpmt_os2_child.c APR: locks\os2\proc_mutex.c locks\os2\thread_mutex.c locks\os2\thread_rwlock.c

LewisR 2014-12-20 09:58 developer ~0002948	Steve: This is true, but I can't see how this occurs for the code with my patches. Unless I am missing something, the calls appear to occur while apr_file_read is looping. Paul: This is the case for buffered reads in apr_file_read() There are multiple calls to apr_thread_mutex_lock/unlock, which in turn call DosRequestMutexSem/DosReleaseMutexSem This code path is used (for example) when reading a text file from a shared drive. Steve: I understand, but I don't understand how this code could be running at the time it shows up in the logs. At the time the DosRequestMutexSem is reported, I believe that the unbuffered read code is running and that the code is between the Dos32Read and the Dos32WaitEventSem. As th apr_file_read patch is currently written the only thing that can happen after Dos32Read returns ERROR_NO_DATA is that DosWaitEventSem. is called. I guess it's possible some sort of signal event caused an exception handler to be invoked. Perhaps enabling tracing for the exception APIs will reveal more. Lewis: error_log says: [Sat Nov 15 21:25:09 2014] [error] [client 192.168.100.8] File does not exist: J:/APPS/apache2/htdocs/2rosenthals.net/coppermine/scripts.js [Sat Nov 15 21:25:12 2014] [error] [client 192.168.100.8] Script timed out before returning headers: php.exe Steve: This is how we fail. Httpd is reporting the fact that the event semaphore wait timed out. The Dos32Write that wrote this message to the log is reported in the .trc file. Recall what apr_fileread is doing: for (;;) { int post_count; if (thefile->pipe) DosResetEventSem(thefile->pipeSem, &post_count); rc = DosRead(thefile->filedes, buf, len, &nbytes); if (!thefile->pipe) break; // Blocking i/o - we are done if (rc == ERROR_MORE_DATA) { // Buffer is full - caller will read more eventually rc = NO_ERROR; break; } if (thefile->timeout == 0) break; if (thefile->timeout == 0) break; if (rc != ERROR_NO_DATA) break; rc = DosWaitEventSem(thefile->pipeSem, thefile->timeout >= 0 ? thefile->timeout / 1000 : SEM_INDEFINITE_WAIT); if (rc == ERROR_TIMEOUT) { len = 0; return APR_TIMEUP; } } // for This code loops until DosRead returns data, DosWaitEventSem times out or DosRead reports and error. [...] 232 is ERROR_NO_DATA and is a normal response for a Dos32Read on a non-blocking pipe when the writer (i.e. php.exe) is running slow relative to the reader. 640 is ERROR_TIMEOUT and in our case this is a bad thing. It means the writer is really slow relative to the reader. It causes the reader to stop reading which results in the write (i.e. php.exe) hanging waiting for Dos32Write to complete. The reason I want to see the traces with stamping is so that I can see when the httpd timeout occurs relative to the php Dos32Write that get stuck. The fix to the stuck php.exes may be as simple as adjusting the timeout settings.

LewisR 2014-12-20 10:06 developer ~0002949	Steve: I recommend that we adjust the timeouts, if that is really the cause of what we are seeing. The timeouts should be see to match "should not occur" situations. Paul: fwiw, best I can tell, this is done using the timeout parameter in httpd.conf - default value is 300(!) seconds. more at: http://httpd.apache.org/docs/2.2/mod/core.html#timeout Lewis: Got 1 stuck PHP process after a half dozen tries. Siege was set for max 10 seconds between reuqests, and 2 clients. I think all together we had 4 PHP processes fire, and one was stuck (the original, I think, though honestly, I didn't check anything other than the timestamp. I was able to kill the process normally from TOP. Steve: I looked at the zip and it confirms my analysis. To avoid the failures we need to increase the pipe (actually the filter response) timeout. This is controlled by the TimeOut directive. The docs claim it defaults to 300 seconds, so you must have set it to 10 seconds because that's what the traces show. Lewis: I then pushed the timeout to 600 seconds (which shouldn't be necessary in any case, as the siege only lasts a minute or two), and went to 20 clients. This got me to a 72.8% availability in siege and a longest transaction time of 29.77 seconds. All PHP sessions closed, but then the VM fell over and rebooted. I will run some more scenarios with this, but I am still concerned that if the timeout value is too low, the worst that should happen is we return a blank page and log an error. Steve: I agree. There are two things going on here. First is what httpd does when it times out. As we know, it logs the timeout. I'm pretty sure it just aborts the connection after this. As we have seen in the traces, the amount of data that php.exe writes before hanging can vary. What we don't know is how much of this httpd wrote to the client before aborting. The second thing is what's going on on the php.exe side when httpd times out. It appears that httpd is not really shutting down the pipe and I'm not sure why. If the pipe was closed, I would expect the DosWrite on the php site to fail with ERROR_BROKEN_PIPE. I suspect this is something that needs to be tweaked in the APR.

LewisR 2014-12-20 10:12 developer ~0002950	Lewis: Sieging with 200 clients with a max of 1 second apart (the original settings), got me to 59 total processes, and httpd stopped serving requests. Prior to this (default_socket_timeout = 30), I was able to fire almost 300 total processes, and all but 50 PHP sessions shut down. Steve: This makes some sense. Setting the socket timeout too low is probably just another way to kill the httpd side of conversation before the php side can finish writing to the pipe. The result is a hung php. Lewis: I can say without a doubt that performance and stability are measurably better with php running as module than as cgi. Steve: Assuming one does not run out of memory in the httpd process, this would be my expectation. OS/2 processes are expensive and slow to start compared to Linux processes. Lewis: Running as module, Apache starts up with 205 threads. Once traffic hits, the thread count will increase to somewhere over 500, and then gradually decline, however, it does not get all the way back down to 205 (it leaves off somewhere around 315). Subsequent sieges gradually erode stability, until either the daemon stops serving pages or the VM reboots. Steve: This implies that httpd is leaking something killing it does not recover. [Steve supplied (final) cleaned-up readwrite.c.diff (readwrite4.c.diff).]

LewisR 2014-12-20 10:19 developer ~0002951	Next phase... Steve: I think that the results of the siege testing failure have exposed a subte error in how the apr closes pipe handles. The docs have this to say about what happens when the server closes the pipe handle: "If the server end closes when the client end is already closed, the pipe is deallocated immediately; otherwise, the pipe is not deallocated until the last client handle is closed." In the context of our CGI testing, when there are no timeouts, the 1st case applies. The php side closes its side of the pipe and when the httpd closes its side of the pipe, the pipe resources are deallocated. I assume this includes the pipe semaphore. When we get a timeout, the 2nd case applies. From the php client's POV the pipe is still open, so the write never completes and the php client hangs around forever. To avoid this, the server needs to disconnect from the named pipe before closing the handle. The will cause the write to fail with ERROR_BROKEN_PIPE. It the client is gone, as is the normal case, the disconnect should no effect. What should work is inserting the following in apr_file_close if (file->pipe == 1) { DosCloseEventSem(file->pipeSem); DosDisconnectNPipe(file->filedes); } before rc = DosClose(file->filedes); There's no need to check for errors. We only want to disconnect the pipe handle, not the client file handle, so we need to modify pipe.c:112 (out)->pipe = 1; to read (out)->pipe = 2; // 2014-11-17 SHL mark as client side of pipe What I don't yet know yet is if we really need to close the pipe semaphore. This may be done automatically when we close the pipe handle. It depends on what the docs mean by pipe resources. IAC, we can verify this later with a bit of testing. [...] FWIW, I went poking about the internet for more code that uses the APIs in question. There's not much, but I did find http://www.virtualbox.org/svn/vbox/trunk/src/VBox/Runtime/r3/os2/pipe-os2.cpp which is probably by Knut. This code explicitly does not disconnect, but that's probably because it closes both sides of the pipe itself. I also found: http://fpc.sourcearchive.com/documentation/2.2.4/os2_2simpleipc_8inc-source.html which implements the close as I am proposing. [...] Another though just occurred to me. Thread ordinals get reused, so this means that pipe names get reused to. Apache assumes that when a thread terminates, the associated named pipe is gone too. However, when we have a hanging php instance, this might not be true. Paul [providing a new build with the proposed mod]: http://smedley.id.au/tmp/httpd-dll-20141118.zip [diffs of Steve's proposed mods attached as open.c.diff and pipe.c.diff]

LewisR 2014-12-20 10:26 developer ~0002952	Lewis: I've trimmed the httpd timeout from 600 seconds to 90 seconds (all php timeouts - socket, mysql, and script execution are set to 60 seconds). Steve: I can't make the timeouts trigger here even with Timeout set to 1 second. What I've done is add a 30 second sleep to the php code to ensure it does not complete in anything like a timely manner. Very odd. [...] I starting looking at pdump.000 from arcanoae-server-dump-after-sixth-siege-2014-11-18-all.zip As expected all the httpd threads are blocked. I need to do more work to figure out if all the threads are waiting on the same semaphore or whatever. [testing left off 2014-11-18 on this system]

psmedley 2015-01-02 07:32 administrator ~0002985	Hey Lewis, can you please post here the siege parameters you're using for testing?

LewisR 2015-01-02 07:48 developer ~0002986	Surely, Paul. Benchmarking is done with: siege -b -t60S http://www.2rosenthals.net/ The above url hits index.html, which contains pure HTML. Stress testing is done with: siege -d 5 -c 200 -f siegeurls1 siegeurls1 is a listing of collected urls from http://www.2rosenthals.net (attached)

psmedley 2015-01-02 15:49 administrator ~0002993	Is it worth also posting the relevant php.ini / httpd.conf here?

LewisR 2015-01-02 16:07 developer ~0002994	Uploaded php.ini, httpd.conf, httpd-mpm.conf, and httpd-php.conf. httpd-ssl.conf and httpd-vhosts.conf contain a bunch of stuff which I would prefer to not have to edit for public consumption, and other icnluded confs (see bottom of httpd.conf) are irrelevant for these purposes. Also note that php.ini loads the pdf module, but testing shows that this is not relevant to success or failure when testing against php pages. ;-)

psmedley 2019-09-29 10:41 administrator ~0003339	is this ticket still adding value? or should it be closed and new ones raised targeting the remaining issues?

LewisR 2020-02-10 05:09 developer ~0003381	Let's close this as suspended. Time for fresh data from newer builds.

Date Modified	Username	Field	Change
2014-12-20 08:59	LewisR	New Issue
2014-12-20 09:01	LewisR	Note Added: 0002940
2014-12-20 09:09	LewisR	Note Added: 0002941
2014-12-20 09:15	LewisR	Note Added: 0002942
2014-12-20 09:15	LewisR	File Added: testing-http-php-with-os2trace.txt
2014-12-20 09:19	LewisR	File Added: testing-http-php-with-os2trace-20141108.txt
2014-12-20 09:20	LewisR	Note Edited: 0002942
2014-12-20 09:22	LewisR	Note Added: 0002943
2014-12-20 09:36	LewisR	Note Added: 0002944
2014-12-20 09:36	LewisR	File Added: readwrite.c.diff
2014-12-20 09:38	LewisR	File Added: readwrite2.c.diff
2014-12-20 09:38	LewisR	Note Edited: 0002944
2014-12-20 09:42	LewisR	Note Added: 0002945
2014-12-20 09:46	LewisR	Note Added: 0002946
2014-12-20 09:46	LewisR	File Added: readwrite3.c.diff
2014-12-20 09:53	LewisR	Note Added: 0002947
2014-12-20 09:58	LewisR	Note Added: 0002948
2014-12-20 10:06	LewisR	Note Added: 0002949
2014-12-20 10:12	LewisR	Note Added: 0002950
2014-12-20 10:12	LewisR	File Added: readwrite4.c.diff
2014-12-20 10:19	LewisR	Note Added: 0002951
2014-12-20 10:20	LewisR	File Added: open.c.diff
2014-12-20 10:20	LewisR	File Added: pipe.c.diff
2014-12-20 10:26	LewisR	Note Added: 0002952
2014-12-20 10:47	psmedley	File Deleted: testing-http-php-with-os2trace.txt
2014-12-23 09:51	Steven Levine	Relationship added	related to 0000638
2015-01-02 07:32	psmedley	Note Added: 0002985
2015-01-02 07:48	LewisR	Note Added: 0002986
2015-01-02 07:48	LewisR	File Added: siegeurls1
2015-01-02 15:49	psmedley	Note Added: 0002993
2015-01-02 15:58	LewisR	File Added: php.ini
2015-01-02 16:03	LewisR	File Added: httpd.conf
2015-01-02 16:03	LewisR	File Added: httpd-mpm.conf
2015-01-02 16:04	LewisR	File Added: httpd-php.conf
2015-01-02 16:07	LewisR	Note Added: 0002994
2019-09-29 10:41	psmedley	Note Added: 0003339
2020-02-10 05:09	LewisR	Assigned To	=> LewisR
2020-02-10 05:09	LewisR	Status	new => closed
2020-02-10 05:09	LewisR	Resolution	open => suspended
2020-02-10 05:09	LewisR	Note Added: 0003381

View Issue Details

Relationships

Activities

Issue History