Moral of the story: If you're going to write an app with some threading,
test with 3 or more threads at least, even if you don't plan on using more than
2. In .NET, you can get away with murder if you only have 2 threads. Problems
with your threading code are much more apparent when 4 threads are in
contention for the same resource. In this scenario that resource is the UI
message loop.
I support a mobile project that synchronizes a great deal of data over the
internet (60meg all told) between a Compact Framework windows app and a
PHP/PERL/BSD backend. MSFT tools were not an option for the backend, so using
RDA Pull commands to get data is out. The syncrhonization is done via web
services, essentially the server sends a DataSet friendly format back and it is
merged locally. They recently asked if the sync could be made faster. My first
reply was "Yeah, if you could tell me which rows are Inserts vs. Updates so I
don't have to do a row-by-row merge" but apparently that is not going to happen
either. While working on a completely different application and trying to fight
through "medicine head" brought on by DayQuil a good idea came to me.
Like many good ideas it was simple and obvious. The current sync process was:
-
Ask for data
-
Get DataSet back from server
-
Process DataTables row by row (very time consuming in SQL CE 2.0)
-
Ask for more data
Lather, rinse, repeate. I had already threaded out the Sync process so the UI
could keep drawing itself during this time, but now it was clear that I could
also implement sort of a local
Message Queue so each individual bundle
of data could be processed Asynchronous to the other thread which was actually
pulling the data from the server. So, I came up with a simple class to queue
and dequeue messages, came up with classes and interfaces to represent a Queue
Work Item and now performance is considerably better. Messages are being
processed while the next message is fetched from the server, so the processor
on the mobile device and the bandwidth it can suck from the server are both
maxed out at all times.
The actual work queue itself is not rocket science either. Here's the code,
complete with all my debug nonsense:
using
System;
using
System.Collections;
using
System.Threading;
using
OpenNETCF.Windows.Forms;
namespace
Mobile.Sync
{
///
<summary>
///
Summary description for SyncWorkItemQueue.
///
</summary>
public
class SyncWorkItemQueue
{
private
object _syncRoot;
private
System.Collections.Queue _queue;
private
bool _run;
public
const int QUEUE_THRESHOLD
= 50;
public
const int CATCH_UP_THRESHOLD
= 8;
public
SyncWorkItemQueue()
{
_syncRoot
= new object();
_queue
= new Queue();
_run
= true;
}
///
<summary>
///
Default to true
///
</summary>
public
bool Run
{
get
{
lock(_syncRoot)
{
return
_run;
}
}
set
{
lock(_syncRoot)
{
_run
= value;
}
ApplicationEx.DoEvents();
}
}
///
<summary>
///
Get the number of SyncWorkItem s in the queue
///
</summary>
public
int ItemCount
{
get
{
lock(_syncRoot)
{
return
_queue.Count;
}
}
}
///
<summary>
///
(Synchronized) Add an item to the end of the queue
///
</summary>
///
<param name="workItem"></param>
public
void AddToQueue(SyncWorkItem workItem)
{
lock(_syncRoot)
{
Console.WriteLine("Adding:
" +workItem);
_queue.Enqueue(workItem);
}
ApplicationEx.DoEvents();
}
public
void StartProcessing()
{
ThreadStart
threadStart = new ThreadStart(StartWork);
Thread
workerThread = new Thread(threadStart);
workerThread.Start();
}
protected
void StartWork()
{
while(Run)
{
System.Windows.Forms.Application.DoEvents();
if
(_queue.Count > 0 )
{
Console.WriteLine("Processing
Item, Count=" + _queue.Count);
SyncWorkItem
workItem = (SyncWorkItem)_queue.Dequeue();
try
{
workItem.Updater.UpdateTable();
}
catch(Exception
ex)
{
Console.WriteLine(ex);
Console.WriteLine("Error
processing work item");
Console.WriteLine(workItem.Updater);
ApplicationEx.DoEvents();
}
Console.WriteLine("Processed:
" + workItem.Updater);
}
//Allow
everything to update
ApplicationEx.DoEvents();
//Now
check for greater than threshold amount, to let everything catch up
if
(_queue.Count > QUEUE_THRESHOLD)
{
Console.WriteLine("Catching
up");
lock(_syncRoot)
{
for(int
i = 0; i < CATCH_UP_THRESHOLD; ++i)
{
SyncWorkItem
workItem = (SyncWorkItem)_queue.Dequeue();
workItem.Updater.UpdateTable();
Console.WriteLine("CATCH_UP_THRESHOLD
item processed, Count=" + _queue.Count);
ApplicationEx.DoEvents();
}
}
ApplicationEx.DoEvents();
}
}
Console.WriteLine("Ending
Startwork");
}
}
}
At the end, if there are work items to process, I just do the following to let
the sync thread catch up:
while
(_syncQueue.ItemCount > 0)
{
Thread.Sleep(10);
}
_syncQueue.Run
= false;
From where I'm standing, this is not too hard to do. I did run into a thread
race condition while implementing this though. Even so, I still don't know why
the average developer/architect is so afraid of using threading. The dreaded
thread race issue was actually relatively easy to track down using the
debugger: just hit "Pause" and it displayed all my threads and I could clearly
see what thread was blocking where. In my case, the issue was sloppy coding. At
one point the sync was not on its own thread and it directly updated the UI.
When I put it on its own thread I did not change these messages to fire through
Control.Invoke(delegate);
I got away with it until more threads and were added to the mix, thus my
comment about getting away with murder when you have only two threads:
Application.DoEvents
is often enough to resolve a lockup but not once you get beyond 2 threads.
Subscribe