View Issue Details

IDProjectCategoryView StatusLast Update
0000684Subversion for OS/2 & eCSBugpublic2019-03-12 12:37
ReporterLewisRAssigned ToSteven Levine 
PrioritynormalSeverityminorReproducibilityalways
Status assignedResolutionopen 
Summary0000684: svnsync 1.7.21 (2018-12-30) crashes
DescriptionPrior to this latest SVN update, TRP files in my SVN bin directory were rare. Now, I get them every night when a cron job runs to sync a server.
Steps To Reproducesvnsync.exe sync --source-username <username> --source-password <pass> file:///subversion/data/local-repo-dir
Additional InformationI wish I could get more details out of this other than to say that the sync fails and produces a TRP file. There is no POPUPLOG entry.

(TRP file sanitized; temp file referenced in the TRP also attached.)
TagsNo tags attached.

Activities

LewisR

2019-02-25 15:29

developer  

svn-XGvwXQ (44 bytes)
HAWKING:b7dd1690-31a7-11e9-bf3e-e1ce4a75b3ef
svn-XGvwXQ (44 bytes)
EDD6_01.TRP (31,319 bytes)

Steven Levine

2019-03-06 03:51

manager   ~0003240

We are going to need an xqs file or a debug build to be able the decipher the .trp file more easily.

The trapping routine is a utility function that grabs the next entry from a linked list and the next pointer is unexpectedly NULL.

Since svnsync is a libc app, there will almost never be a popuplog. If you want to see the libc trap report, you need to capture it from the command line output.

Does svnsync fail the same way when run from the command line?

Steven Levine

2019-03-06 03:55

manager   ~0003241

One more thing. I recommend you update to exceptq 7.11.5.

LewisR

2019-03-06 10:22

developer   ~0003242

Reminder sent to: psmedley

Running from a prompt produces the same TRP.

Updated exceptq to 7.11.5 beta1. Had to unlock exceptq.dll, and have not rebooted. I'll give it another go when I have a good opportunity to reboot the server (10 days, 6 hours uptime; latest Apache fixes have been doing well).

Meanwhile, Paul, if you would be so kind as to provide an xqs or a map or a debug build of this, I'll give it another go once I have that.

psmedley

2019-03-06 20:28

administrator   ~0003243

Hi guys, I'm interstate today, not home until late Thursday. I'll provide a map/xqs most likely Friday my time.

psmedley

2019-03-08 18:57

administrator   ~0003244

http://smedley.id.au/tmp/subversion-1.7.21-src.zip

Steven Levine

2019-03-09 06:11

manager   ~0003245

apr_file_close is calling apr_thread_mutex_destroy but the file has no mutex assigned so apr_thread_mutex_destroy traps.

The call occurs only for buffered files so I have to suspect that the mutex was not created when the file was opened.

I've attached an annoated .trp file which may help here.

Note that EBX points to to apr_file_t file, which is helpful.

FWIW, I noticed that the OS/2 code still uses APR_BUFFERED, which is deprecated:

#define APR_BUFFERED APR_FOPEN_BUFFERED /**< @deprecated @see APR_FOPEN_BUFFERED */

although I don't think this contributed to the failure.

It appears that part of the apr_file_t *progfile got clobbered. This will probably required some quality time with the debugger to track down. The file is marked open and has a file handle, but the mutex pointer, the buffer pointer and the buffer size are zero which should not occur for a buffered file.

When did this issue start to occur?

Does the previous svnsync build continue to work correctly?

EDD6_01.TRP-shl (32,580 bytes)

LewisR

2019-03-09 07:17

developer   ~0003246

> When did this issue start to occur?

The TRP files have only been produced since upgrading to the 20181230 build. However, the plot thickens. See next.

> Does the previous svnsync build continue to work correctly?

When I set this up in 2017, I initialized the sync repository, which pulled the latest rev at that time.

Paul and I exchanged email on sync issues, and he sent me the link to the apr-posix build of 1.7.16 from 20140809. *That* build works (though I did not put it into production for other reasons). Here's what I've just discovered:

1.7.21 20160707 - no TRP, but no sync. It quietly returns to the prompt.
1.7.21 20150810 - same
1.7.6 20121002 - same
1.6.19 20121002 - same

There were other issues with the apr-posix build overall (e.g., svn log would come back with "svn: E070008: Error resolving case of <project>"), which was why I never really used it.

So, sync has never worked for the above builds. This is not a recent breakage, only recent reporting. Without reporting, I thought that no updates were being committed to the source repository. When I saw the TRP files accumulating, I decided to actually check.

Steven Levine

2019-03-09 12:43

manager   ~0003247

Did the 1.7.16 from 20140809 really work? What I mean is where you able to access the mirror repository normally?

Errors such as:

  svn: E070008: Error resolving case of <project>"

will usually cause an operation to terminate.

LewisR

2019-03-09 13:24

developer   ~0003248

I'm sorry, I wasn't clear.

Yes, svnsync from 20140809 does actually work. In fact, I just now updated my local workspace from the mirror, and as svnsync reported during the sync, everything is up to date with the latest commit, now.

The E070008 error came back running svn.exe specifically, which was why I did not deploy the entire 20140809 package. Also, interactive prompts do not work (for any executable in that package, I believe), so for svnsync, without specifying the --source-username on the command line, when prompted, svnsync does not accept (or acknowledge) input, and waits forever. So, right now, using it as I need to do right now (automated), the 20140809 svnsync.exe is a viable workaround.

psmedley

2019-03-09 17:53

administrator   ~0003249

Last edited: 2019-03-09 18:27

View 2 revisions

Hey guys... going back to older APR source where win32 and os2 code were quite similar:

win32:
        if (file->mutex) {
            apr_thread_mutex_destroy(file->mutex);
        }

os2:
    if (file->buffered)
        apr_thread_mutex_destroy(file->mutex);

Not sure if there's any good reason for OS/2 to be different?

edit: http://smedley.id.au/tmp/subversion-1.7.21-os2-20190309.zip has this change...

LewisR

2019-03-11 13:40

developer   ~0003250

Law of unintended consequences, Paul. With the 20190309 build, I get:

svnsync: E000010: Error waiting for process '/subversion/data/<project>/hooks/pre-revprop-change': No children

pre-revprop-change says:

#!sh.exe
exit 0

On the upside(?), no TRP file was created. Of course, now that the repo is fully synced, I have a hard time determining whether a new sync is actually working, so I'll have to set up another sync with something else (or re-init this one) to be sure.

psmedley

2019-03-11 14:53

administrator   ~0003251

Hey Lewis, any tips on setting up svnsync so I can test locally?

LewisR

2019-03-11 15:28

developer   ~0003252

The following is abstracted and slightly modified from our email exchange of March 1 & 2, 2017, Paul:

1. Create a new repo for the mirror:

   svnadmin create mirror-repo <enter>

2. Configure access to the mirror as you would for any new repository.

3. Edit pre-revprop-change to read simply:

   #!sh.exe
   exit 0

(pre-revprop-change must be present and must come back with a 0 return code; it doesn't need to do anything but that. See http://svnbook.red-bean.com/en/1.7/svn.ref.svnsync.c.init.html and http://svnbook.red-bean.com/en/1.7/svn.ref.reposhooks.pre-revprop-change.html)

4. (Not sure if this is required. I recall getting errors until I did this, and recently, moving this path out of the way caused me other issues. YMMV.) My path rewrite for /dev/null to c:\dev\null didn't seem to work, (and svnsync init kept barfing on being unable to write to /dev/null), so I created \dev in the root of the SVN volume (V:) and created "null" as a 0-length file there.

5. Initialize the mirror:

   svnsync init --source-username <username> --source-password <password> file:///<local-data-path>/mirror-repo svn://source-repo-url

This should get you a working mirror repo.

psmedley

2019-03-11 16:24

administrator   ~0003253

OK I get the same error(s) - both with the Dec 2018 build (the TRP) and the March build: (svnsync: E000010: Error waiting for process '/dev/svn-test/mirror-repo/hooks/pre-revprop-change': No children)

Steven Levine

2019-03-11 17:17

manager   ~0003254

Paul, the change you suggested in https://mantis.smedley.id.au/view.php?id=684#c3249 makes sense. I have to suspect the change was done to prevent traps similar to what we see in the case where the file open fails. The only reason I can think of that is was not done for the OS/2 code is that no one but us it looking at it.

Lewis, regarding the /dev/null issue, this is probably APR related. If APR was ported using libc, the libc runtime would have handled the /dev/null support.

LewisR

2019-03-12 12:37

developer   ~0003255

> Lewis, regarding the /dev/null issue, this is probably APR related. If APR was ported using libc, the libc runtime would have
> handled the /dev/null support.

Agreed 100%, and judging by some of the other wonky behavior we've encountered, this all fits into the pattern.

Issue History

Date Modified Username Field Change
2019-02-25 15:29 LewisR New Issue
2019-02-25 15:29 LewisR File Added: svn-XGvwXQ
2019-02-25 15:29 LewisR File Added: EDD6_01.TRP
2019-03-06 03:51 Steven Levine Note Added: 0003240
2019-03-06 03:55 Steven Levine Assigned To => Steven Levine
2019-03-06 03:55 Steven Levine Status new => feedback
2019-03-06 03:55 Steven Levine Note Added: 0003241
2019-03-06 10:22 LewisR Note Added: 0003242
2019-03-06 20:28 psmedley Note Added: 0003243
2019-03-08 18:57 psmedley Note Added: 0003244
2019-03-09 06:11 Steven Levine File Added: EDD6_01.TRP-shl
2019-03-09 06:11 Steven Levine Note Added: 0003245
2019-03-09 07:17 LewisR Note Added: 0003246
2019-03-09 07:17 LewisR Status feedback => assigned
2019-03-09 12:43 Steven Levine Status assigned => feedback
2019-03-09 12:43 Steven Levine Note Added: 0003247
2019-03-09 13:24 LewisR Note Added: 0003248
2019-03-09 13:24 LewisR Status feedback => assigned
2019-03-09 17:53 psmedley Note Added: 0003249
2019-03-09 18:27 psmedley Note Edited: 0003249 View Revisions
2019-03-11 13:40 LewisR Note Added: 0003250
2019-03-11 14:53 psmedley Note Added: 0003251
2019-03-11 15:28 LewisR Note Added: 0003252
2019-03-11 16:24 psmedley Note Added: 0003253
2019-03-11 17:17 Steven Levine Note Added: 0003254
2019-03-12 12:37 LewisR Note Added: 0003255