So I had MVVM Light Nu-Get packages in my code and was partially using them but when working on mod tools for a game I'm developing this weekend I realized I was making more trouble by using MVVM Light and removed it. I also broke something in my project configuration while I was making changes and moved all of my code to a new project. Thought it was no longer a "SimpleBatchCopy" tool and that it deserved a new name ... so "SimpleThreadedCopy" is the new name.
It is so much more than it started out as and is now doing most of the things I wanted but was hesitant to implement due to complexity. It reads a source file once for every write it makes simultaneously to many remote computers at an adjustable throttled speed.
My file copies using the previous version of this tool continued to crash and it got frustrating - so I took a few hours tonight to make some major modifications.
1. Ability to resume failed Copies
2. Better tracking of where it failed - like on which server (although I usually get graceful handling while I'm testing on single copy - I'm getting a full program crash in real life.
3. Continuing on a single server write failure.
4. Better memory management.
Today I made some attempts at resolving the following needs.
1a. Resume Failed copies
Ability to resume failed copies worked. In my code I was creating a new thread on each read for each server to a method that accepted the FileStream, Buffer, and buffersize. I created a new class that contained information about the destination and also contained the FileStream object. That way I pass the DestinationData object to the new thread and if there is a failure I can set a property in that thread to indicate it failed and skip it on all further writes. I am also using this object to populate the new ListView that shows information about each individual server - just realized I didn't ever implement the calculation for Percent Complet - no biggie - the progress bar is an indication of the progress for how much of the source file has been read - which should match every server after they all catch up.
1b. Resume Failed copies - identifying files to overwrite v. files to resume.
If the destination file + ".partial" exists - it resumes and appends all data to .partial - This could technically end up in broken files if the partial isn't the same as the first part of the source. I don't care tonight- I want to resume my copy at work.
1c. Resume Multiple Failed copies - that may be at different stages
This is problematic - have to make sure that the bytes/sec between the files that failed matches the gap or some of the files will get broken if the buffers don't line up. For a server's file to start receiving bytes of data - the size of the destination file must match the Already Read size of the input source stream. - so if a file doesn't line up - it should get skipped in every pass and you will have a file that doesn't finish. Thinking of it - I'm not sure if it will properly indicate this at the end. I should add some logic in for that.
1d. Rename the .partial file to the actual destination file at end of copy.
File.Move() covers this ... does not yet overwrite if destination already exists - but will leave the partial behind and that works for me at this junction. Will fix.
2. Better tracking of where it failed.
I made an attempt at this with the new DestinationData object that contains information about each thread. It should let me know when a particular server fails - and shouldn't crash the program. We'll see - I didn't get enough testing into it.
3. Continuing on a single Failure
4. Better memory management
I didn't clean up hardly any objects in my code and relied heavily on C# garbage collection. But I made a few passes and nullifying objects where they were no longer in use. There are still lots of opportunities for me to go through the code and clean it up.
I did break the UI a little bit when I added the new ListView in - I need to modify the calls inside of the Resize Methods but didn't want to deal with it tonight so I made a few adjustments to prevent important stuff from getting hidden - and mostly succeeded - the "Add Job" button is mostly hidden.
I do want to save logs automatically and do better logging - It's pretty pathetic right now.