View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000495 | Rsync | Bug | public | 2011-11-06 05:58 | 2017-11-29 19:41 |
| Reporter | LewisR | Assigned To | Steven Levine | ||
| Priority | normal | Severity | minor | Reproducibility | always |
| Status | feedback | Resolution | open | ||
| Product Version | 3.0.6 | ||||
| Summary | 0000495: Rsync has trouble with --xattrs when whitespace is present in path and/or no EAs present (certain files) | ||||
| Description | This is with Steve's 3.0.8 build of 2011-04-19. (Originally, we had --no-xattrs specified due to another issue with EAs which I can't recall offhand.) With --xattrs specified, and the rsync pass hits a directory with a space, I'm seeing a DosQueryPathInfo error 8 at lib/sysxattrs.c(241). This error does not occur on *all* directories with whitespaces, nor on all files without EAs, so I'm a bit stymied as to what the exact cause may be (example file attached, through the miracle of mdirs). | ||||
| Additional Information | Command is run from cron: c:\opt\rsync\bin\rsync.exe -r --xattrs --times --stats --human-readable --log-file=c:/var/log/rsync_cgp.log --exclude-from=c:/MPTN/ETC/rsync-cgp-exclude.lst j:/Communigate j:/CGP_Backup >nul 2>&1 Log shows: 2011/11/06 00:32:52 [4654] [generator] unigetxattr: DosQueryPathInfo(Communigate/Accounts/lgrosenthal.macnt/OS2 Wireless.folder/all-messages.mdir/102097-S_______-20031123115013-89, FIL_QUERYEASFROMLIST) failed with error 8 at lib/sysxattrs.c(241) Dir (from 4OS2 3.0.7) shows: Volume in drive J is DATA Serial number is 657E:D5BA Directory of J:\Communigate\Accounts\lgrosenthal.macnt\OS2 Wireless.folder\all-messages.mdir\102097-S_______-20031123115013-89 8-01-09 23:44 3,538 0 102097-S_______-20031123115013-89 3,538 bytes in 1 file and 0 dirs 4,096 bytes allocated (file attached) (Steve, you may note that this is one of the messages which we massaged with your Perl script when I migrated to Communigate Pro; in fact, all of these older files are generating this error. These files were originally contiguous, i.e., contained in a single mbox, and later, split into separate files in an mdir via another Perl script - along with other mbox files.) | ||||
| Tags | No tags attached. | ||||
| Attached Files | 102097-S_______-20031123115013-89 (3,538 bytes)
X-UIDL: 2342
X-Mozilla-Keys:
Return-Path: os2-wireless_users-owner@2rosenthals.com
Received: from mail.2rosenthals.com (localhost [127.0.0.1] ) by mail.2rosenthals.com
(Hethmon Brothers Smtpd) ; Sun, 23 Nov 2003 11:50:13 -0500
Received: from mail1.no-ip.com (mail1.no-ip.com [63.215.241.221] ) by mail.2rosenthals.com
(Hethmon Brothers Smtpd) ; Sun, 23 Nov 2003 11:50:12 -0500
X-Envelope-To: <os2-wireless_users@2rosenthals.com>
Received: (qmail 10904 invoked by uid 89); 23 Nov 2003 16:49:49 -0000
Received: from unknown (HELO MAIL03.toast.net) (206.244.185.10)
by mail1.no-ip.com with SMTP; 23 Nov 2003 16:49:49 -0000
Received: from 2rosenthals.com (unverified [206.149.148.84]) by MAIL03.toast.net
(Vircom SMTPRS 3.0.273) with ESMTP id <B0057899567@MAIL03.toast.net> for <os2-wireless_users@2rosenthals.com>;
Sun, 23 Nov 2003 11:48:39 -0500
Message-ID: <3FC0E53F.9020703@2rosenthals.com>
Organization: Rosenthal & Rosenthal
User-Agent: Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.4.1) Gecko/20030909
X-Accept-Language: en-us, en
MIME-Version: 1.0
References: <auto-000208779654@admin.nni.com>
In-Reply-To: <auto-000208779654@admin.nni.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Date: Sun, 23 Nov 2003 11:50:07 -0500
Sender: os2-wireless_users-owner <os2-wireless_users-owner@2rosenthals.com>
X-Listname: os2-wireless_users@2rosenthals.com
Reply-To: os2-wireless_users@2rosenthals.com
From: Lewis G Rosenthal <os2-wireless_users@2rosenthals.com>
To: os2-wireless_users@2rosenthals.com
Subject: [OS2Wireless] OS/2 software's site survey function
X-List-Unsubscribe: Send email to mailusers-request@2rosenthals.com
X-List-Owner: mailusers-owner@2rosenthals.com
Stan, it's been my experience recently, that when WEP is enabled on the
AP and the client has WEP turned off, that I haven't even been able to
associate with it. I suppose it could be a driver level thing; I've only
tried this with the IBM High Rate and the Cisco 350 cards. This was also
under Windows XP, BTW.
On 11/22/2003 08:51 am, Stanley Sidlov thus wrote :
>On Fri, 21 Nov 2003 20:25:19 -0500, Lewis G Rosenthal wrote:
>
>
>
>> 4. If the AP has WEP enabled, even if it is beaconing, you won;t be
>> able to associate with it (or see its SSID in WiFiState).
>>
>>
>
> why do I remember attaching and having frame errors when the wep is incorrect? In windows, that is?
>
>
>Stan Sidlov
>--
>BOD, Warpstock, Inc.
>
>Did YOU go to WARPSTOCK 2003 ? No?! You missed a lot! Make a bid for 2004!
>
> http://www.warpstock.org
>
>
--
Lewis
------------------------------------------------------------
Lewis G Rosenthal, CNA
Rosenthal & Rosenthal
Accountants / Network Consultants
New York / Northern Virginia www.2rosenthals.com
Team OS/2 / NetWare Users International www.novell.com
------------------------------------------------------------
This OS/2 system (Apollo) uptime is 0 days 00:17 hours and 20 seconds
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
To unsubscribe from this list, send a message to
steward@2rosenthals.com with the command
"unsubscribe os2-wireless_users" in the body
(omit the quotes).
For help with other commands, send a message
to steward@2rosenthals.com with the command
"help" in the body (omit the quotes).
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
| ||||
|
|
Since the pathname in the the error message is relative, it implies that the problem is with EAs on the destination file. What does the dir command have to say out this file? |
|
|
Volume in drive J is DATA Serial number is 657E:D5BA Directory of J:\CGP_Backup\Communigate\Accounts\lgrosenthal.macnt\OS2 Wireless .folder\all-messages.mdir\102097-S_______-20031123115013-89 8-01-09 23:44 3,538 0 102097-S_______-20031123115013-89 3,538 bytes in 1 file and 0 dirs 4,096 bytes allocated 16,920,453,120 bytes (15GB) free No EAs, as expected. Only the archive bit is set on the backup file, as well. Do you think this could be some underlying filesystem corruption? The volume had already undergone that long chkdsk pass which we discussed on one of our mutual lists (that was 10/21, IIRC). The thing is that this file was already copied, and had no EAs on eithe rthe original or the first copy. It hasn't changed since then, so rsync should have just skipped over it (unless it had some difficulty accessing the file for comparison). The only things in the exclude list are the queues & logs directories, too, BTW. Oh, and no EAs on the directory itself (source or destination), either. |
|
|
Let's see if http://mantis.smedley.info/file_download.php?file_id=221&type=bug can tell us anything. It adds the failing EA name to the error report. |
|
|
Hehehe... No details, and no error (at least, not this error) running rsync-3.0.8-20111230-shl. Let's close this as resolved, unless you think that whatever changed between 20110419 and 20111231 regarding xattrs could not have addressed this. To test, after the first rsync pass didn;t copy any files (Doh! because they were already there!), I deleted about 50 of the backup copies (including the sample, referenced in this bug) in the problem directory, and waited for the next hourly pass. The entire group of files copied without error. Thanks! |
|
|
I recommend we leave this ticket open for a bit. I did not change anything that could have resolved the error you reported. |
|
|
Yep... Hit it again: 2012/01/09 13:14:29 [8556] [sender] unigetxattr: DosQueryPathInfo(CGP_Backup/Communigate/Domains/driven-zen.org/bulkimages.macnt/INBOX.mdir/42350-________-20100509231812-372, FIL_QUERYEASFROMLIST) for FLAGS failed with error 8 at lib/sysxattrs.c(241) This time, no whitespaces in the path, and this file should not have been in use by any other process, as it's: 1. An old message file (2010); 2. Already in the backup directory (CGP_Backup); 3. Unchanged since originally backed up, and the hourly backup pass would not have had this open at the same time. A dir on the file shows: Volume in drive J is DATA Serial number is 657E:D5BA Directory of J:\CGP_Backup\Communigate\Domains\driven-zen.org\bulkimages.macnt \INBOX.mdir\42350-________-20100509231812-372 5-09-10 19:18 12,387 124 42350-________-20100509231812-372 12,387 bytes in 1 file and 0 dirs The 124 bytes of EAs appear to all be LIBC EAs. rsync-3.0.8-20111230-shl made two attempts (it would appear from the log) to access this file and two others in that directory, all of which failed with the same error. These files were not copied to the destination at all. rsync command used: c:\opt\rsync\bin\rsync.exe --archive --xattrs --os2-perms --del --itemize-changes --times --stats --human-readable --log-file=c:/var/log/rsync_backup_j.log j:/ e:/ I don't know how the LIBC EAs would have even been attached to the original file, as really none of the CGP mail files should have any EAs on them, however, it appears that every file touched by rsync now has LIBC EAs attached. In fact, looking more closely, I find that I actually have duplicate files, one version without EAs and another with EAs (the ones with EAs do *not* exist in the live CGP mail directory). Here's an example (from a different mailbox backup): 10-05-11 2:58 75,268 0 1-S_______-20111005065823-1262 10-05-11 2:58 75,268 124 1-________-20111005065823-1262 10-05-11 3:21 32,351 0 2-S_______-20111005072112-776 10-05-11 3:21 32,351 124 2-________-20111005072112-776 10-05-11 3:44 3,839 0 3-S_______-20111005074412-104 10-05-11 3:44 3,839 124 3-________-20111005074412-104 10-05-11 4:33 13,396 0 4-S_______-20111005083312-322 10-05-11 4:33 13,396 124 4-________-20111005083312-322 10-05-11 9:00 16,640 0 5-S_______-20111005130031-639 10-05-11 9:00 16,640 124 5-________-20111005130031-639 10-05-11 9:06 3,821 0 6-S_______-20111005130636-109 10-05-11 9:06 3,821 124 6-________-20111005130636-109 10-05-11 9:39 47,094 0 7-S_______-20111005133952-785 10-05-11 9:39 47,094 124 7-________-20111005133952-785 10-05-11 10:24 15,184 0 8-S_______-20111005142455-555 10-05-11 10:24 15,184 124 8-________-20111005142455-555 10-05-11 10:28 2,587 0 9-S_______-20111005142811-62 10-05-11 10:45 649 0 10-S_______-20111005144514-15 and the live mailbox: 10-05-11 2:58 75,268 0 1-S_______-20111005065823-1262 10-05-11 3:21 32,351 0 2-S_______-20111005072112-776 10-05-11 3:44 3,839 0 3-S_______-20111005074412-104 10-05-11 4:33 13,396 0 4-S_______-20111005083312-322 10-05-11 9:00 16,640 0 5-S_______-20111005130031-639 10-05-11 9:06 3,821 0 6-S_______-20111005130636-109 10-05-11 9:39 47,094 0 7-S_______-20111005133952-785 10-05-11 10:24 15,184 0 8-S_______-20111005142455-555 10-05-11 10:28 2,587 0 9-S_______-20111005142811-62 10-05-11 10:45 649 0 10-S_______-20111005144514-15 10-05-11 10:45 5,539 0 11-S_______-20111005144552-141 10-05-11 11:26 647 0 12-S_______-20111005152616-15 10-05-11 11:58 649 0 13-S_______-20111005155826-15 10-05-11 12:10 131,806 0 14-S_______-20111005161034-1789 10-05-11 12:22 16,963 0 15-S_______-20111005162212-399 10-05-11 13:09 21,189 0 16-S_______-20111005170920-490 10-05-11 13:25 10,021 0 17-S_______-20111005172505-331 10-05-11 13:41 3,335 0 18-S_______-20111005174144-98 Not sure why the "S" was replaced with a "_" when the EAs were tacked on. I'm happy to open a separate bug for the duplicate files or perhaps we should create a meta-bug for EA-related issues? |
|
|
I understand why I have duplicate files. The filenames changed as a result of the state of the message changing (not read -> read). Now to figure out how to get rsync to replace the old versions with the ones of changed state, or better, skip them entirely. Still, we should not be *adding* EAs to the copied files when --xattrs is specified, should we? |
|
|
I am not sure what you mean by "we should not be *adding* EAs to the copied files when --xattrs is specified." Rsync sync's EAs. It does not add EAs that do not exist at the source. If something "added" EAs to the source file, it is unlikely that it was rsync. Note that there may be an errant libc based app out there that does add EAs to every file it accesses. I have had one report of a user system that had libc EAs attached to every file on the boot volume. |
|
|
CGP is not a gcc app. None of CGP's files have *any* EAs on them, yet when running rsync.exe --xattrs, *all* backed up copies get libc EAs (exactly 124 bytes) attached. This, though, as I said in comment #2159, is not the real issue insofar as multiple copies of the same content are concerned. It *is*, however, an issue either for libc or rsync, apparently. What I've found is that while I haven't been able to get 3.0.8 to tack on the EAs (perhaps those came from a previous build?) it *will* consistently do it when creating new directories. I'll open a separate issue for that. |
|
|
I finally figured out what you did to generate the EAs. Way back when you were running rsync without --xattrs. This means that libc will add the libc private EAs and rsync will leave them alone. IMO, this is appropriate. It's not different that if rsync was run without --perms. The destination file permissions will be set to to runtime defaults. Once you started using --xattrs, rsync brought the EAs into sync for existing files. Files that no longer existed at the source remaining unchanged. If you want them to go away you can use my StripLibcEAs script or the equivalent. Note that libc064 supports the LIBC_UNIX_EAS environment variable which allows the libc EAs logic to be suppressed for specific volumes. Unfortunately, libc064 is not yet sufficiently stable for rsync use, so we can not yet use this. The directory EAs are a known issue. I won't bore you with the gory details, but currently it takes two rsync runs to get the directory EAs into sync. Once we have a working libc with LIBC_UNIX_EAS support, I might be able to use this to avoid the spurious EAs. |
|
|
Okay. Understood. So, we're still back where we started in this bug, trying to figure out why we get the error 8 on some files and not others. Per my PM, as long as we're not getting EA data *replaced*, the fact that we're adding 124 extra bytes of EAs is of little consequence. The presence (or lack thereof) of these EAs is not what is causing the multiple copies of the CGP files, as we've determined, which was my initial concern. Thanks for the clarification. |
|
|
Where can I learn more about LIBC_UNIX_EAS ? Has libc065.dll support for the env var LIBC_UNIX_EAS I see at: http://svn.netlabs.org/libc/ticket/43 "the value of the variable is on the forms: set LIBC_UNIX_EAS=!c,!e-f,d,g-h " Does this work now? Oh, and where may I find the StripLibcEAs script? |
|
|
There's not a lot of external documentation. http://svn.netlabs.org/libc/search?q=LIBC_UNIX_EAS will tell you a bit more than you already know. As the query shows, the last change to the support was http://svn.netlabs.org/libc/changeset/3697. LIBC065 was the first release to formally support the feature. I'll send you a copy of StripLibcEAs privately with some comments. It's not 100% generic. |
|
|
Please retest with http://www.warpcave.com//betas/rsync-3.0.9.1-20171111-shl.zip |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2011-11-06 05:58 | LewisR | New Issue | |
| 2011-11-06 05:58 | LewisR | File Added: 102097-S_______-20031123115013-89 | |
| 2011-12-31 00:16 | Steven Levine | Note Added: 0002089 | |
| 2011-12-31 00:22 | Steven Levine | Assigned To | => Steven Levine |
| 2011-12-31 00:22 | Steven Levine | Status | new => assigned |
| 2011-12-31 03:37 | LewisR | Note Added: 0002091 | |
| 2011-12-31 05:35 | Steven Levine | Note Added: 0002098 | |
| 2011-12-31 05:35 | Steven Levine | Status | assigned => feedback |
| 2011-12-31 21:27 | LewisR | Note Added: 0002118 | |
| 2011-12-31 21:27 | LewisR | Status | feedback => assigned |
| 2011-12-31 21:38 | Steven Levine | Note Added: 0002119 | |
| 2012-01-09 19:24 | LewisR | Note Added: 0002158 | |
| 2012-01-09 19:33 | LewisR | Note Added: 0002159 | |
| 2012-01-10 08:36 | Steven Levine | Note Added: 0002162 | |
| 2012-01-10 16:54 | LewisR | Note Added: 0002163 | |
| 2012-01-10 21:33 | Steven Levine | Note Added: 0002164 | |
| 2012-01-11 02:55 | LewisR | Note Added: 0002171 | |
| 2013-03-12 17:51 | asavage | Note Added: 0002416 | |
| 2013-03-12 18:12 | asavage | Note Edited: 0002416 | |
| 2013-03-12 20:38 | Steven Levine | Note Added: 0002417 | |
| 2017-11-29 19:41 | Steven Levine | Status | assigned => feedback |
| 2017-11-29 19:41 | Steven Levine | Note Added: 0003153 |
