Go to bug ID
Hello, guest. We have noticed that you are not registered at this bug tracker. Your experience will be greatly enhanced if you log in. To do so, you first must register by clicking on the Register tab at the top. If you are already registered, you can login at the Login tab.
Syndicate Syndicate Listing Display Search Login/Register
Bug Id ?
Reporter ?
Guest
Product/Version ?
Crimson Editor / Version 3.70
Status ?
Confirmed
Severity ?
Minor
Duplicate Of ?
- none -
Summary ?
Writing to Samba shares is extremely slow
Report Time ?
April 20, 2007 10:15:36 AM
Assignment ?
- none -
Resolution ?
Open
Priority ?
Normal
Dependencies ?
- none -


Attachments
cedtDoc.cpp Patched version of cedtDoc.cpp on revision 224
cedtElement.cpp Patched version of cedtElement.cpp on revision 224
Votes
For: 2 (100%)
Against: 0 (0%)
Total: 2

April 20, 2007 10:15:36 AM Guest
http://www.crimsoneditor.com/english/board/CrazyWWWBoard.cgi?db=forum3&mode=read&num=6104&page=1&ftype=6&fval=samba&backdepth=1
http://www.crimsoneditor.com/english/board/CrazyWWWBoard.cgi?db=forum3&mode=read&num=6138&page=1&ftype=6&fval=samba&backdepth=1
http://www.crimsoneditor.com/english/board/CrazyWWWBoard.cgi?db=forum3&mode=read&num=5344&page=1&ftype=6&fval=samba&backdepth=1
http://www.crimsoneditor.com/english/board/CrazyWWWBoard.cgi?db=forum3&mode=read&num=5027&page=1&ftype=6&fval=samba&backdepth=1

Thanks!

August 29, 2007 12:21:02 AM pva
Yes, this is actually an issue for me. I'm happily using CE from work, but I cannot use it from home: opening a file over the net takes forever... This is why I have to jump between CE and PSPAD -:( So if you could fix this for the next release that would be really cool!!!! Thanks.

August 29, 2007 08:29:08 AM Pvt_Ryan
does this affect the development binaries?

August 29, 2007 02:26:38 PM pva
Yes. I got the latest and it still behave the same way (slow).

August 29, 2007 02:45:56 PM Pvt_Ryan
Relevent posts from the CE forum

Post 6104
---------
I have a debian linux server running samba. Saving files from notepad, and other editors is instantaneous. Saving using Crimson Editor is much slower. After investigating the issue, I have found that Crimson editor creates a temporary file that is 0 bytes. Because of the samba protocol, windows will then write in 1 byte increments until the file is written. Two solutions could are:
1. don't create the temp file, just write to the original file.
I assume you had a reason to do it the way you are, so this is probably not satisfactory.
2. create the temp file with the filesize of the file being saved.
Samba will then assume that if the file was created, that the server will not run out of disk space, and therefore will write in normal chunks.

This may be an artifact of using a linux samba server, but it is annoying. Saving a 52K file over a local network takes at least 7 seconds(7.4K/sec).
-------------

Post 6106
---------
I run samba on my Slackware box and have the shares mapped to X: and W: on my Windowsbox. I've used recommended settings (from some website) in my smb.conf. I DO NOT experience ANY lag saving files to the samba shares.

Have you tryed mapping the shares?

Regards Nikolai
----------

Post 6107
---------
Yes, I'm mapping the shares. What version of samba are you running? and what do you smb.conf files look like? I know that it's the debian version/default settings that is exposing this issue. But I was unable to determine what setting may be causing the problem.

John
---------

Post 6138
---------
I have exactly the same problem!!
The file saving by Crimson Editor on Samba shares is EXTREMELY SLOW.
I am running SuSE Linux 8.2 and 9.3 and this has been an issue for all 3.x versions of Samba.
Please fix it in the next release if possible! Crimson Editor is such a great piece of software after all.
---------

Post 5343
---------
This is about the best editor ive seen so far. Im in love with it.

I welcomed the split screen (which was a big plus) on V3.70. One of the most noticable differences that ive seen is that this version seems to send huge ammounts of data over the wan on a mounted Unix file system (Samba). I logged way over 10 megs on a 3K file so it takes about 50 seconds to save it :-) I didnt notice this with the previous version 3.5x. Is there any obvious reason for this?

Either way awsome program.
---------

Post 5344
---------
Ok i think i exagerated a bit here. It was 180K transfer on a 24K file ;-) but its taking 60 seconds to save. Loading the same file with Crimson takes 1 second. winvi saves the same file as a 35K transfer in about 6 seconds. It Takes 10 seconds to saves a 17k file in a 35K transfer with dos copy cmd.

My documents area is on a server which could cause the delay in writing if some tmp cache was used by crimson. So could this be the cause and if so is there any way to repoint to C: instead of documents area ?

Regards
Pete
--------

Post 5347
---------
Hiya,

I am also experiencing this same problem in version 3.70. I think Crimson Editor is a fantastic program, and the only thing that puts me off using it is the length of time it takes to open and save UNIX files over Samba.

Since most of my source files are on UNIX systems, this is a real problem for me.

I don't experience this problem with any other text editor, and it seems I'm not the only one to experience this problem.

Keep up the good work with Crimson - if you can get this 1 bug fixed, it'll be ideal! ;-)

Thanks,

Rob.
----------

Post 5349
---------
I did some more testing and found that its sending about 400 - 500 bytes per second only. So a 15 hundred line script was taking 70 seconds to save but load times are 3 seconds or less.

After I reinstalled Crimson Editor V3.51 and it saved the same file in 6 seconds which is the same as what winvi is saving it as.

Any ideas welcome.

Regards Pete
------------

Post 5020
---------
I use crimson to edit files on a share network folder, this connection is powered by a samba server, and sometimes crimson saves them with size equal to zero.
Have somebody had this problem ?
---------

Post 5027
---------
I have the same problem and think this problem is related with tmp files, but I haven't a solution.
---------

Post 5052
---------
Yes, It's a dangerous bug, my files were lost too, can somebody help us?
A Crimson's user from Toronto.
---------




August 29, 2007 04:25:22 PM pva
OK, I see, people had and still have similar problems as I do. Are you planning to fix this in foreseeable future?

Also, it would be very helpful if you post on your web site the most current development binaries (you can call them pre-release). This will allow the users to use/test whatever you have right now making it more interactive.

So basically I’m suggesting having two things available for download:
1) The main release (the stable and highly tested version for the end-user that is updated once in a while);
2) Frequently updated (once a month or so) version based on the current bug fixes, new added features, etc.

Thanks!

August 29, 2007 05:34:42 PM Pvt_Ryan
Actually we do provide pre-release binaries. look in the development thread.
and the latest release 3.70 is avaialable from the downloads section.

As to fixing this problem I dont know ill have a quick look at it and ifs its easy it'll be done asap if not (ie it takes me more than a day) ill leave it for 3.73.

Sadly the slowness in itself is an inconvience as opposed to a problem so its not a very high priority for me atm.

August 29, 2007 05:51:03 PM pva
> Actually we do provide pre-release binaries.

Yes, I know. But they are hidden somewhere in threads so it always takes some clicking around to finally find it... However if I go to http://www.emeraldeditor.com/downloads/ I don't see any pre-release version but the only one old 3.70 which I can get from CE site as well. My suggestion was just this simple: you put two links at http://www.emeraldeditor.com/downloads/ , one is for official release and another one is for the most current pre-release version. Like this everyone (incl. newbies) immediately sees this.

> As to fixing this problem I dont know ill have a quick look

Thanks a lot!

> Sadly the slowness in itself is an inconvience as opposed to a problem so its not a very high priority for me atm.

Well, when it takes from a minute to more to save a file then it becomes a good reason (unfortunately) to use the alternatives. But anyway I'm glad you're considering this to fix and I'm looking forward for the next (pre-)release!

Thanks a lot!

August 29, 2007 06:45:32 PM Pvt_Ryan
The reason that they are only available in the threads and not on the downloads page is because I do not maintain this site, its Arantor that does so he would have to add them.

But as phil and I are mods on the forum we can update the binaries there at will.

September 16, 2007 02:13:46 PM andrei.n
I was the original submitter of this bug back in April (as a guest in the first post). Exactly the same way as pva describes above I have to jump between CE at work and PSPAD at home because of this stupid behaviour. I'd like to kindly ask the maintainers to give this bug a priority if possible. After all, it should not be that hard to change the saving routine, should it? I mean, I am not a C language pro, but implementing this:

"create the temp file with the filesize of the file being saved. Samba will then assume that if the file was created, that the server will not run out of disk space, and therefore will write in normal chunks."

instead of creating a zero-byte file should not be such a huge amount of work...

By the way, it seems that currently CE will create a zero-byte file with the same filename as the original file, i.e. essentially erasing it before saving. It is a very bad practice and I lost many files in the past due to this (they were nullified and then the network link to samba share would be broken during save). So in the new implementation, is it possible to save the file as described above but with a real temporary filename and only once the the operation is complete and successful the temporary file should be renamed to the original.

Thank you very much for looking into this!!

September 17, 2007 10:33:49 AM Pvt_Ryan
Sorry havent had a chance to look into this yet been busy irl. I'll try to get a vm running on my laptop tonight and see if i can replicate the problem failing that I'll setup another PC in the house.

I'll try it under debian / ubuntu as it seems to not be distro specific (seeing as there are suse users complaining too (although i reakon that their probelm is that they are using suse, if they would only switch to a real distro.... :P) ).

December 13, 2007 07:13:21 PM
OK. I just wanted to share what I found out. My saves of small files are taking several seconds each time. I encountered this problem with Crimson Editor 3.7 and thought it was Samba related too, BUT:

Then I discovered that the issue was only related to Crimson Editor. I tried Notepad++ and no issues there.
I then checked saving files against Windows 2003 server. SAME ISSUE and only with Crimson Editor again!

I found some insight running Ethereal. Saving small file from Crimson Editor returns probably few THOUSANDS of packets transmitting 2-40 bytes of data, but being several hundreds bytes total size each. And those are exchanges between computer and server. Each send receives response. Seems like the response is requested or just the way file is transmitted requires such response.
These thousands of packets go with infos: :Write AndX request.." and "Write AndX response.." all using SMB protocol

In contrast when saving file with Notepad++ whole file save takes around 15 packets starting same way using SMB protocol:
Trans2 Request, QUERY_FILE_INFO
Trans2 Response, QUERY_FILE_INFO
Write AndX Request
Write AndX Response
And then the rest of transfer goes via NBSS protocol, 10-15 packets:
NBSS Continuation Message

Another thing.. not sure if related.. we recenly changed one switch in our network and we upgraded windows 2003 small bussines to standard edition. Samba was newly setup server too. So I'm not sure where the issue came from yet.

December 13, 2007 07:20:05 PM
I forgot to say I'm mostly in windows environment with one ubuntu 7.10 lamp server w/ samba enabled. Windows 2003 server standard ed. XP Pro workstations.

January 18, 2008 02:34:26 AM
I used Crimson editor many years ago and loved it. It's great the source is open now. I'd like to give back. I've done MFC for a long time now and took a quick look at this issue. There are two separate issues in the comments: (a) saving to Samba is slow, (b) saving can potentially truncate the file if the network fails in the middle of the save, producing data loss.

I'm attaching a patch that should address issues (diff against version 224). The changes are very minor maybe only 20 lines.

To address issue (a), when saving the file it issues a system call (through CFile's SetLength method) to preallocate the size of the file to an estimate of the final size of the file (the estimate is a safe lower bound). Since the set length syscall will result in a single request/response against the SAMBA server it should cause the actual writes to proceed in large chunks. Also, even when not saving to SAMBA file systems it should speedup the saving of very large files and reduce disk defragmentation if lots of other disk writes are going on at the same time.

To address issue (b), when saving the file instead of truncating the file it saves the file to a temporary file in the same directory. If successful, it will then rename the file it is overwriting to some other temporary filename (i.e. filename.tmp-previous.version), rename the new temporary file that was just saved to the correct filename, and finally delete the old version of the file (filename.tmp-previous.version). This is a really safe way to save files, since at no point is a file truncated and the old version is not deleted until after the new version is in place.

Luckily Crimson Editor already overrode the appropriate MFC functions so these changes were really easy to make. I've done similar modifications to MFC apps before so I knew where to look.

I haven't tested this patch, but it should work, and it should be really easy to test (only 2 manual tests needed -- just save a new file, and then save a modified file that already exists).

January 18, 2008 02:37:42 AM
Changed so that instead of truncating file contents when saving to a file it saves to a temporary file in the same directory and then does some renaming to avoid data loss. Will need to localize one new failure message into a string resource. I just hard-coded the string for now. Changes in CCedtDoc::OnSaveDocument method.

January 18, 2008 02:39:16 AM
Help SAMBA performance and reduce fragmentation by preallocating file size before writing data. Changes to both CAnalyzedText::FileSave and CMemText::FileSave methods. Not tested but changes are very simple.

January 18, 2008 02:42:21 AM
To verify the fix I would suggest that the patch is integrated into the pre-release and then tested by those users who were seeing the issue before. 90% chance that the patch will fix the issue and will save a lot of work setting up a VM just to test this one issue. Just my 2-cents...
Best,
Kevin

January 22, 2008 10:25:54 AM Pvt_Ryan
Hi, Thanks for the patch I have applied it to my copy and seems to work for me (mind you an unpatched version for me has no issues.. :/)

One small point however is if possible please submit patches as unified diffs, it makes them easier to applied and quickly browse.

February 13, 2008 09:24:20 AM Pvt_Ryan
Can someone please confirm this as fixed so i can close the bug?

February 13, 2008 03:03:20 PM pva
Pvt_Ryan wrote:
> Can someone please confirm this as fixed so i can close the bug?

I would be VERY happy to test it. I've been waiting for it a couple of years already -:) Sorry for stupid question: where I can get the fixed version of CE? I went to www.emeraldeditor.com but I could not find anything (I was looking for a single exe file that installs by one click, as normally CE does)? Could you please post a link? THANK YOU!

February 13, 2008 04:24:54 PM Pvt_Ryan
http://forum.emeraldeditor.com/index.php?topic=361.0

February 16, 2008 07:56:50 AM pva
Hi All,
I just tried the latest CE and I'm afraid to say, the slowness issue is not fixed. It's still take ages to save a file. Actually now it even takes a long to open a file (although I don't remember if it was the case before). The same manipulations take no time in PSPAD and PFE32 ("programmer's editor"). Did anyone else try? Cheers, pva.

February 16, 2008 05:53:27 PM
This particular patch didn't change how files are opened. Some additional information on the behavior you see on the server side while the file is being saved will be helpful. For example, when saving a file, if you could list the directory listing several times on the server while the save is occurring so you can 'capture' what the temporary filename and filesize is that it begins saving it to. Also if you use a protocol analyzer like Ethereal to decode the SMB packets that would be helpful information as well. In this patched version, you should see it first create a temporary file with a similar name that is about the final filesize. You should then see it writing data to this temporary file, and finally it should rename the original file to a different temporary filename, rename the new temporary filename to the original filename, and then delete the old original file (that was renamed to something temporary). Do you see this behavior?

If your kernel supports inotify (2.6.13 I believe) you could install inotify-tools (http://inotify-tools.sourceforge.net/) and use that to monitor all activity in the directory on the Linux server where you are saving the files to. That will get us some detailed information about exactly what file operations are going on and whether they match what the client is asking the server to do.