As a follow up on my Simple Threaded Copier - I ended up creating a new Visual Studio Solution with three projects.

1. ​The threaded copy classes to create a library like isolation
2. Graphical User Interface (GUI) implementation of the Threaded Copy Library
3. Command Line Interface (CLI) Implementation of the Threaded Copy Library

​The Command Line Implementation became the primary focus for my development and enhancements. 

​It is much easier to setup scheduled copies with the CLI rather than the GUI.  The ultimate goal was to make the GUI work with command line arguments and allow the CLI to optionally launch the GUI with a switch.  I am not there yet.

​The CLI Project adds only one file of code to initiate the console application, parse the command prompt, initiate copy, and monitor the progress of the copy.

​Features of the CLI:

• /L \\server \\server1 \\server2 ... \\serverN - Specify a manual list of Servers to copy to
  • OR 
• /F filename.txt - Specify a file with a list of server names to copy to
• /O - Overwrite existing files
• /RP - Remove Partials - if a .partial file already exists for the destination file delete start over.  If not specified it will get the size of the partial and continue as if the file is of the same origin.
• /T - Count of Bytes to be copied per second
• /S - single file copy path originating from the share or local drive - Share$\folder\folder\file.ext
• /D - the destination path originating from share to file name - Share$\folder\folder\destFile.ext
• /SF - Source Folder for Multiple copies in a directory
• /DFL - Destination File List - the list of files to be copied to the destination path from the source folder
• /DF - Destination Folder - Share$\folder\DestFolder

​Sample Usage
(single file)
​STCCLI.exe /L \\server \\server1 \\server2 /s C:\temp\myfile.file /d c$\temp\mycopiedfile.bat /t 6400 /o

​(multiple files)
STCCLI.exe /F servers.txt /SF C:\temp\ /df c$\temp /DFL Files.txt /T 64000 /O /RP

I realized now my pictures do not enlarge - no big deal they are ugly but will fix next time.



So I had MVVM Light Nu-Get packages in my code and was partially using them but when working on mod tools for a game I'm developing this weekend I realized I was making more trouble by using MVVM Light and removed it.  I also broke something in my project configuration while I was making changes and moved all of my code to a new project.  Thought it was no longer a "SimpleBatchCopy" tool and that it deserved a new name ... so "SimpleThreadedCopy" is the new name.

It is so much more than it started out as and is now doing most of the things I wanted but was hesitant to implement due to complexity.  It reads a source file once for every write it makes simultaneously to many remote computers at an adjustable throttled speed.

My file copies using the previous version of this tool continued to crash and it got frustrating - so I took a few hours tonight to make some major modifications. 

Starting Needs:

1.  Ability to resume failed Copies

2.  Better tracking of where it failed - like on which server (although I usually get graceful handling while I'm testing on single copy - I'm getting a full program crash in real life.

3.  Continuing on a single server write failure.

4.  Better memory management.




Today I made some attempts at resolving the following needs.
1a.  Resume Failed copies
Ability to resume failed copies worked.  In my code I was creating a new thread on each read for each server to a method that accepted the FileStream, Buffer, and buffersize.  I created a new class that contained information about the destination and also contained the FileStream object.  That way I pass the DestinationData object to the new thread and if there is a failure I can set a property in that thread to indicate it failed and skip it on all further writes.  I am also using this object to populate the new ListView that shows information about each individual server - just realized I didn't ever implement the calculation for Percent Complet - no biggie - the progress bar is an indication of the progress for how much of the source file has been read - which should match every server after they all catch up.

1b.  Resume Failed copies - identifying files to overwrite v. files to resume.

If the destination file + ".partial" exists - it resumes and appends all data to .partial - This could technically end up in broken files if the partial isn't the same as the first part of the source.  I don't care tonight- I want to resume my copy at work.

1c.  Resume Multiple Failed copies - that may be at different stages

This is problematic - have to make sure that the bytes/sec between the files that failed matches the gap or some of the files will get broken if the buffers don't line up.  For  a server's file to start receiving bytes of data - the size of the destination file must match the Already Read size of the input source stream. - so if a file doesn't line up - it should get skipped in every pass and you will have a file that doesn't finish.  Thinking of it - I'm not sure if it will properly indicate this at the end.  I should add some logic in for that. 

1d.  Rename the .partial file to the actual destination file at end of copy.

File.Move() covers this ... does not yet overwrite if destination already exists - but will leave the partial behind and that works for me at this junction.  Will fix.

2.  Better tracking of where it failed.

I made an attempt at this with the new DestinationData object that contains information about each thread.  It should let me know when a particular server fails - and shouldn't crash the program.  We'll see - I didn't get enough testing into it.

3.  Continuing on a single Failure

See #2

4.  Better memory management

I didn't clean up hardly any objects in my code and relied heavily on C# garbage collection.  But I made a few passes and nullifying objects where they were no longer in use.  There are still lots of opportunities for me to go through the code and clean it up.




I did break the UI a little bit when I added the new ListView in - I need to modify the calls inside of the Resize Methods but didn't want to deal with it tonight so I made a few adjustments to prevent important stuff from getting hidden - and mostly succeeded - the "Add Job" button is mostly hidden.




I do want to save logs automatically and do better logging - It's pretty pathetic right now.

Simple Batch Copy Tool

Version 0.1 - Rapid Prototype

Same results as a Batch Script to copy one file to many locations.  Not much UI. 

Uses System.IO.File.Copy() method for copying files to destination.

• Basic tool operation:
• Specify the input file or directory.
• Then click "Edit Servers or %servername% and then a dialog opens where you can enter a list of servers.  click OK.
• Type the common UNC path that would appear after \\%servername%\ for where the file or folder will be copied to
• Click Add and a "Copy Task" is created for each Server.
• Click Go to start processing each file.

There is threading so that the form still updates and functions while the copies are going.

Version 0.2 - UI Improvements

Improved UI significantly and added exception handling.

UI Additions:

• Added the ability to remove copy Tasks (Selected, All, Completed)
• Improved Log Area to show contents more than just one line.
• Log can be saved to text file.
• UI Scales better.

Version 0.3 - Copy Speed Throttle Added

Bandwidth Throttling Added

A need arose at work to copy 33 GB of data to around 70 site servers.  Each site has limited bandwidth so we can't interrupt employees by letting copy go full speed.

Throttling speed of 10 KB/s is hard coded.  This was a prototype version to prove I could do it.  I would not be able to complete the copy in a timely manner


The file copy progress was a mystery... the files on the remote server show 0kb until copy is complete so I had no idea how long copies were going.

Version 0.4 - Added Progress bar and customizable throttle speed

Progress Bar Added

Bandwidth Throttling UI Added

Modified the way the copy tasks are stored and processed.  Instead of one task per destination - multiple destinations are now stored in one task.


Added code to read the input stream buffer then write it to each multiple destination in the Copy Task.

This could have potential savings when doing large scale file copies because the disk is only being read once per source file.  UI only updated partially to support this.

Also added the old System.IO.File.Copy() functionality for un-throttled copies.



Version 0.5 - Added multi-threading copy.

No UI enhancments - some poorly implemented elements (like destination shows "Collection"  the Progress column isn't updated.  Log is poorly implemented from the changes implemented in adding threading (can be fixed simply). 

In Code added threading for the copy task.

I am reading in Bytes into my buffer then looping through all the destinations and creating a thread essentialy for each write into a destination stream.

Call a wait and then join all the threads. 


Results of multi-threaded copy

I tried to compare the copy times between v 0.4 and v 0.5 but due to other activity on the network and not having a proper testing lab my results are a little skewed.

Copying a 307,142,656 byte file to 3 Remotes sites.

v 0.4 (single threaded - read once write multiple)  = 38 Minutes 10 Seconds

v 0.5 (multi threaded - read once write multiple simultaneously) = 9 Minutes 51 Seconds (the throttle speed is different because I changed the wait from 1000 milliseconds to 900 milliseconds thinking that possibly the threads would not join within one second - I don't think it matters with the number of copies completed but the throttle speed will never be true unless I add a lot of logic to make it so and it will have to handle the different copy speed for each thread )



Copying a 307,142,656 byte file to only 1 Remote site

v 0.4 = 20 Minutes 48 Seconds

Reading and then sequentially writing into each destination (single threaded):



Where am I going from here...fixing UI, Making sure exceptions are handled, making sure directory copy is still functioning - It may have been  broken but I have been testing with single files.  Fixing the logging.

Posting code snippets - and the program.