I was presented with a problem where the organization wanted to display some simple text information on the desktop like what Microsoft's Sysinternals BGInfo provides.  BGInfo is old and has not been updated to support DPI scaling on High Resolution displays and the organization had not approved the free ware available so as a side project I began developing my own Informational Desktop Overlay... This is a post about my experiences and development process. 

​On first instinct I thought maybe – I could make a Windowed application that was borderless and had a transparent background with a text box.  I soon realized that I could not find a way to keep that window above the desktop when a user would press Win+D to show the desktop.  I attempted a few different methods but it seems like I could not restore the window until the user manually restored the application – there is likely a way but I did not find it in my early discovery. 

​I knew I could draw above everything if I used GDI – albeit if a window underneath the drawing redrew I would lose the GDI drawing.  I wrote a “tetris” like game called "Stacked" for a Game Development/Data Structures Class in C++ (can see a version of the game re-written for Android here).  The project guidelines intended for us to make a game that ran in Windows Command Prompt Console – so I did – in a way.  The menus were all based in Command prompt but when the game started it used GDI calls to draw the board, score, and objects on top of the desktop.  Input was received through the command prompt so the command prompt had to maintain focus to play the game.  The game would flicker and get slow the more it had to draw on the screen but hey it was pretty slick for a first year game development course.

Some of these were added via scope-creep but in the end this is what the application goals were:
​When the desktop is visible display in the upper right hand corner the Host name and IP address at a minimum.

​If the application is provided additional input at launch: run a command, or script capture the output and display that additional information.
Display all information without modifying the background image causing skewing, scaling, or changing the wallpaper mode.
​

​I chose to work with C# as it is the language I use on a near day to day basis for the last 4 years.  I did my research and was soon able to start drawing with GDI+ on the screen (the easiest part).  The tricky part was finding a way to make sure I don’t draw over other windows.  I was too close to the program and not thinking like a game developer but more like some business application data-manipulation developer (which is what I had been doing exclusively in programming for the last several months – not game dev). 
I approached it with the idea in mind that I want this to show up when a user presses Win+D or Alt+Tab, or based on the “active window.”  I was thinking too much about the windows.  I added more code using “User32.dll” to get the active window handle and then that window’s Title.  With some if-statements I was able to get some basic functionality where I could force it to draw only when the active window was null, task switcher, start menu/Cortana, or an empty string.
This was a good starting point but I was running into ugliness when the text flickered during each loop or would draw over some windows, and would randomly disappear when user pressed Win+D and then clicked on the desktop

​I started futzing about more looking for ways to tell when an active window changed so I could redraw less frequently and not only when it triggered in my main program loop.  This phase of development led to lots of code that I ended up ripping out but also allowed me to really strengthen up my methods that would last.
I started working with Windows Hooks.  I used more “user32.dll” methods to create hooks, read windows event messages, and strengthen my main Loop.  I added a .NET stop watch to the main loop, replacing a wait method, allowing redraw after a specific delta but allowing the loop to process windows messages.  I implemented a Windows hook with a delegate to an overloaded Redraw method of the Overlay object.  Getting the right windows events hooked was tricky – I ended up hooking to: SwitchStart, MinimizeStart, and Foreground (window changed).  This was almost good enough. 

• Draw when the user Alt+Tab’d, Win+D, Opened start menu, clicked on the Desktop, or any Window with empty string for Window Name.  The flicker was minimized although certain conditions made it flicker more than I wanted.  

At this point I thought I was 95% done that it was almost as good as I was going to get in C# so I switched back to another program that I had prototyped giving this one a rest.

I finished the other application about 8 hours of coding and testing – A command prompt application with a variety of command line switches.  It was a simple program, not very interesting and did not require much research but had a lot of little nuances and branching conditions. 
When I got back to my desk my code for Overlay was still up – Thinking about my wants for the program it finally occurred to me the solution I had been missing all along.

APPEAR to draw over the Desktop wallpaper ALWAYS when not obstructed. 
​
I was currently checking for whatever the active window was not caring about where it was on the screen or if it was obstructing the overlay draw area.  In game dev I can drop a ray and find what it intersects with why would this be any different – surely there is a way to know what window is under the four vertices of my draw area?
User32.dll has a method for that: IntPtr WindowFromPoint(Point point) - why did I not think of this.  Time wasted on getting the delegate in the correct hooks in windows events - Arrrgh. 
So initially I added to all my existing code (keeping the event hooks) a check in the main program loop to see if there was a window over my Overlay area… if not redraw.  This worked!  I then did some more tuning to how I managed the list of window titles that I was allowing the program to draw over preparing to add them to a config file that could be easily modified in the future.  I added code to work up the handles of objects over the overlay’s to get the root parent window because at times applications in Windowed (not full screen) mode were overlapping the overlay area were returning a handle with a window title but a null string unless placed very specifically.  I could now drag windows over the overlay area and it would get cleared out by the dragging window’s redraw and then when the window left the overlay area would redraw my overlay text.  It worked great.

​Implemented the .NET settings file to have a list of all the window names I will allow the Overlay to draw over.  I added handling of a “debug” parameter to my launch arguments so that I could force it to write the name of any window that was covering the overlay so I could add it to the Config file.  With the Debugging running I noticed a few oddities that I traced back to the Redraw’s called from the Windows Event hooks.  I commented out the creation of the hooks, the Overloaded Redraw, some extra variables and had a fairly good overlay…. But bloated by .NET and C#.

The performance of .NET and C# were not great using 8 MB of RAM constantly.  For a program that is expected to run 100% of the time and only display information sometimes, I was unhappy with the resources required.
Seeing as most of the workload portions of the application were using native windows GDI+ and User32 .dll's it was an easy task converting the functionality from C# .NET managed code to unmanaged lighter weight C++.  I wanted to also be careful that whatever I did I didn’t have any dependencies on .NET or having a particular C++ redistributable installed.  Some compile time settings took care of the C++ redistributable dependencies.  Alternative implementations of some methods and classes avoided the .NET dependency.  I wrote a quick and simple stop watch class to get timers back in my main loop.  By limiting the number of times it is able to evaluate if there is a window over the Overlay area and if it should redraw I can throttle the amount of CPU used and limit any redraw flicker. 

​The first new addition to the program during conversion was adding logic to make the program recalculate the draw area every few seconds so that if the display dimensions changed the overlay would adjust to the new proper location.

After I had the program completed and working on my Windows 10 desktop I started to make sure it would work on Windows 7 and Windows 10 computers without Dev Tools.  I had some Windows 10 imaged VM’s with minimal specs on an ESXi host and tested my application.
Nothing.
Tested on Windows 7 laptop – Nothing
​The program was running without crashing or errors but nothing was displaying on screen.
Installed Visual Studio on my Windows 7 laptop and started doing compiling there – nothing.  While debugging I found that the overlay position was being calculated incorrectly due to the display topology… easy fix but that didn’t explain the VM’s.  I installed remote debugger on a VM and debugged from my main Win 10 workstation – all the variables were correct, the return from the GDI+ DrawString() method was successful.  After some searching I found that the way the display driver was configured on the VM, the lack of resources available for Video, and not having VMware tools installed was to blame.

To continue my application testing I installed a copy of my customized image of Windows 10 on a newish test laptop I had available and tested my application… Nothing – no drawing.  What?! I thought this was figured out!  Installed remote debugger and started looking into the cause.
Seeing the same behavior – No Errors, Variables are showing the correct draw locations, and return from DrawString() states success.  While getting frustrated and going back and forth to get the resolution test different resolutions I eventually realized that this was the first windows 10 machine that had DPI scaling turned on.  I was aware of DPI scaling but was unaware of the way Microsoft had implemented a form of DPI Scaling emulation for applications that don’t tell Windows they are High DPI-aware. 
​
I had converted the program to use a Command Prompt window to display the state of variables while the program ran (instead of using breakpoints which is difficult when you are trying to measure the intersecting windows and such).  I could see where the Overlay was supposed to be, when it was supposed to be drawing, that the draws were successful, and when there was a window covering the overlay space.  Everything looked right in the application but it wasn’t looking right on the screen – nothing was displaying.  I thought that maybe due to the scaling the overlay was off the screen somewhere and added some test code to make the overlay move 10 pixels down and left every few seconds.  After running the program and waiting I saw that the overlay appeared.  It took me two more restarts to notice that it was only drawing inside of this smaller rectangle originating from the bottom left hand corner of the monitor (thinking back I believe it was bottom right because my debug information that displays in top left was not displaying either).  After doing some measuring I was able to conclude that it was drawing inside a RECT that matched the resolution of a DPI scaling of the native resolution.   Somehow DPI scaling was limiting the drawing to this emulated resolution. 
​

​
I worked with trying to find a way to fake the emulated desktop draw area… I think the issue was I was pulling the handle for the “Desktop” window to make the GDI+ DrawString() call and Windows was treating the area I wanted to draw out of bounds of the DPI Scaled “Desktop” window and not drawing.
 
I started reading more into DPI scaling in Windows 8 and 10 – more than the skimming I did before – and found there was a method that could be called in a thread that would tell windows to treat the process as “HighDPI” aware.  This worked on Windows 10 but does not on Windows 7.  I added code in to determine the OS at runtime and skip any calls to the function but it was no good.  I did not want to have two versions of my application one for Win 7 and older and one for Win 8 and newer.  While looking through compile options hoping there was a way I could bundle the required win10 resource into my application or have some sort of runtime ignore calling the function I just happened to notice answer:
Manifest Tool > Input and Output > DPI Awareness
 

​After cleaning up my earlier attempts at getting around DPI scaling, by just adding this setting the application now worked properly on Windows 7, Windows 10 without scaling, and Windows 10 with DPI scaling.

Main()
Creates Overlay object

• Reads in arguments to see if Debug is one of them
• Else treats the launch arguments as a Console command and arguments for that command
• Runs the Command + arguments  - gets the console output from that command feeds it into the Overlay object as additional text.
• Calls the Run method of the Overlay object.

Overlay.Run()

• Contains a main loop with switch statements that evaluate what action should be performed. 
• Each loop calls method to determines if there is a window above the Overlay area but the draw method is only able to be called at most once every 250 ms. 
• There is a wait at the end of the loop for 30 ms to limit CPU.  CPU usage is less than 2% while drawing and less than 1% while not drawing.  Because of the implementation of the stopwatch class when the overlay area is exposed there should be at most 30ms delay before ReDraw() can be called. 
• The method call for Overlay Location Recalculation is on a separate stopwatch limiting call to every 4500 ms.

As a follow up on my Simple Threaded Copier - I ended up creating a new Visual Studio Solution with three projects.

1. ​The threaded copy classes to create a library like isolation
2. Graphical User Interface (GUI) implementation of the Threaded Copy Library
3. Command Line Interface (CLI) Implementation of the Threaded Copy Library

​The Command Line Implementation became the primary focus for my development and enhancements. 

​It is much easier to setup scheduled copies with the CLI rather than the GUI.  The ultimate goal was to make the GUI work with command line arguments and allow the CLI to optionally launch the GUI with a switch.  I am not there yet.

​The CLI Project adds only one file of code to initiate the console application, parse the command prompt, initiate copy, and monitor the progress of the copy.

​Features of the CLI:

• /L \\server \\server1 \\server2 ... \\serverN - Specify a manual list of Servers to copy to
  • OR 
• /F filename.txt - Specify a file with a list of server names to copy to
• /O - Overwrite existing files
• /RP - Remove Partials - if a .partial file already exists for the destination file delete start over.  If not specified it will get the size of the partial and continue as if the file is of the same origin.
• /T - Count of Bytes to be copied per second
• /S - single file copy path originating from the share or local drive - Share$\folder\folder\file.ext
• /D - the destination path originating from share to file name - Share$\folder\folder\destFile.ext
• /SF - Source Folder for Multiple copies in a directory
• /DFL - Destination File List - the list of files to be copied to the destination path from the source folder
• /DF - Destination Folder - Share$\folder\DestFolder

​Sample Usage
(single file)
​STCCLI.exe /L \\server \\server1 \\server2 /s C:\temp\myfile.file /d c$\temp\mycopiedfile.bat /t 6400 /o

​(multiple files)
STCCLI.exe /F servers.txt /SF C:\temp\ /df c$\temp /DFL Files.txt /T 64000 /O /RP

I realized now my pictures do not enlarge - no big deal they are ugly but will fix next time.



So I had MVVM Light Nu-Get packages in my code and was partially using them but when working on mod tools for a game I'm developing this weekend I realized I was making more trouble by using MVVM Light and removed it.  I also broke something in my project configuration while I was making changes and moved all of my code to a new project.  Thought it was no longer a "SimpleBatchCopy" tool and that it deserved a new name ... so "SimpleThreadedCopy" is the new name.

It is so much more than it started out as and is now doing most of the things I wanted but was hesitant to implement due to complexity.  It reads a source file once for every write it makes simultaneously to many remote computers at an adjustable throttled speed.

My file copies using the previous version of this tool continued to crash and it got frustrating - so I took a few hours tonight to make some major modifications. 

Starting Needs:

1.  Ability to resume failed Copies

2.  Better tracking of where it failed - like on which server (although I usually get graceful handling while I'm testing on single copy - I'm getting a full program crash in real life.

3.  Continuing on a single server write failure.

4.  Better memory management.




Today I made some attempts at resolving the following needs.
1a.  Resume Failed copies
Ability to resume failed copies worked.  In my code I was creating a new thread on each read for each server to a method that accepted the FileStream, Buffer, and buffersize.  I created a new class that contained information about the destination and also contained the FileStream object.  That way I pass the DestinationData object to the new thread and if there is a failure I can set a property in that thread to indicate it failed and skip it on all further writes.  I am also using this object to populate the new ListView that shows information about each individual server - just realized I didn't ever implement the calculation for Percent Complet - no biggie - the progress bar is an indication of the progress for how much of the source file has been read - which should match every server after they all catch up.

1b.  Resume Failed copies - identifying files to overwrite v. files to resume.

If the destination file + ".partial" exists - it resumes and appends all data to .partial - This could technically end up in broken files if the partial isn't the same as the first part of the source.  I don't care tonight- I want to resume my copy at work.

1c.  Resume Multiple Failed copies - that may be at different stages

This is problematic - have to make sure that the bytes/sec between the files that failed matches the gap or some of the files will get broken if the buffers don't line up.  For  a server's file to start receiving bytes of data - the size of the destination file must match the Already Read size of the input source stream. - so if a file doesn't line up - it should get skipped in every pass and you will have a file that doesn't finish.  Thinking of it - I'm not sure if it will properly indicate this at the end.  I should add some logic in for that. 

1d.  Rename the .partial file to the actual destination file at end of copy.

File.Move() covers this ... does not yet overwrite if destination already exists - but will leave the partial behind and that works for me at this junction.  Will fix.

2.  Better tracking of where it failed.

I made an attempt at this with the new DestinationData object that contains information about each thread.  It should let me know when a particular server fails - and shouldn't crash the program.  We'll see - I didn't get enough testing into it.

3.  Continuing on a single Failure

See #2

4.  Better memory management

I didn't clean up hardly any objects in my code and relied heavily on C# garbage collection.  But I made a few passes and nullifying objects where they were no longer in use.  There are still lots of opportunities for me to go through the code and clean it up.




I did break the UI a little bit when I added the new ListView in - I need to modify the calls inside of the Resize Methods but didn't want to deal with it tonight so I made a few adjustments to prevent important stuff from getting hidden - and mostly succeeded - the "Add Job" button is mostly hidden.




I do want to save logs automatically and do better logging - It's pretty pathetic right now.

Simple Batch Copy Tool

Version 0.1 - Rapid Prototype

Same results as a Batch Script to copy one file to many locations.  Not much UI. 

Uses System.IO.File.Copy() method for copying files to destination.

• Basic tool operation:
• Specify the input file or directory.
• Then click "Edit Servers or %servername% and then a dialog opens where you can enter a list of servers.  click OK.
• Type the common UNC path that would appear after \\%servername%\ for where the file or folder will be copied to
• Click Add and a "Copy Task" is created for each Server.
• Click Go to start processing each file.

There is threading so that the form still updates and functions while the copies are going.

Version 0.2 - UI Improvements

Improved UI significantly and added exception handling.

UI Additions:

• Added the ability to remove copy Tasks (Selected, All, Completed)
• Improved Log Area to show contents more than just one line.
• Log can be saved to text file.
• UI Scales better.

Version 0.3 - Copy Speed Throttle Added

Bandwidth Throttling Added

A need arose at work to copy 33 GB of data to around 70 site servers.  Each site has limited bandwidth so we can't interrupt employees by letting copy go full speed.

Throttling speed of 10 KB/s is hard coded.  This was a prototype version to prove I could do it.  I would not be able to complete the copy in a timely manner


The file copy progress was a mystery... the files on the remote server show 0kb until copy is complete so I had no idea how long copies were going.

Version 0.4 - Added Progress bar and customizable throttle speed

Progress Bar Added

Bandwidth Throttling UI Added

Modified the way the copy tasks are stored and processed.  Instead of one task per destination - multiple destinations are now stored in one task.


Added code to read the input stream buffer then write it to each multiple destination in the Copy Task.

This could have potential savings when doing large scale file copies because the disk is only being read once per source file.  UI only updated partially to support this.

Also added the old System.IO.File.Copy() functionality for un-throttled copies.



Version 0.5 - Added multi-threading copy.

No UI enhancments - some poorly implemented elements (like destination shows "Collection"  the Progress column isn't updated.  Log is poorly implemented from the changes implemented in adding threading (can be fixed simply). 

In Code added threading for the copy task.

I am reading in Bytes into my buffer then looping through all the destinations and creating a thread essentialy for each write into a destination stream.

Call a wait and then join all the threads. 


Results of multi-threaded copy

I tried to compare the copy times between v 0.4 and v 0.5 but due to other activity on the network and not having a proper testing lab my results are a little skewed.

Copying a 307,142,656 byte file to 3 Remotes sites.

v 0.4 (single threaded - read once write multiple)  = 38 Minutes 10 Seconds

v 0.5 (multi threaded - read once write multiple simultaneously) = 9 Minutes 51 Seconds (the throttle speed is different because I changed the wait from 1000 milliseconds to 900 milliseconds thinking that possibly the threads would not join within one second - I don't think it matters with the number of copies completed but the throttle speed will never be true unless I add a lot of logic to make it so and it will have to handle the different copy speed for each thread )



Copying a 307,142,656 byte file to only 1 Remote site

v 0.4 = 20 Minutes 48 Seconds

Reading and then sequentially writing into each destination (single threaded):



Where am I going from here...fixing UI, Making sure exceptions are handled, making sure directory copy is still functioning - It may have been  broken but I have been testing with single files.  Fixing the logging.

Posting code snippets - and the program.