+972 52-548-6969
In general, I don’t like the notion of “I’m an XYZ developer”. I write code in C#, Python, TypeScript, C, Zig, Rust, and anything else that I feel like. Most of the production code I write is in C#, but I don’t like the tag of “I’m a C# dev”. It is… a very limiting term.
Having said that, I started writing C# code over 25 years ago. I got started during the .NET alpha days, around 2001, I think. Given that, I have a lot of experience with C# & .NET, all the way to contributing code to .NET.
I want to talk a bit about the meaning of having that kind of knowledge in a particular platform, especially when we have expert models. After all, we can assume that the AI knows the platform, so you just need to tell it what to build, no?
Let’s talk a bit about performance and the depth of knowledge you need to get great results with it.
I have an event loop where I’m processing events. I care about processing them as fast as possible. I do that by grabbing a batch of items from a queue, processing them, then notifying the callers about it. The code looks roughly like this:
var batch = new List<Event>(100);
while (true)
{
while (batch.Count < 100 && queue.TryDequeue(out var evt))
batch.Add(evt);
if (batch.Count == 0)
{
newEvents.Wait();
newEvents.Reset();
continue;
}
ProcessBatch(batch); // do the work...
foreach (var evt in batch)
evt.CompletionSource.TrySetResult();
batch.Clear();
}Note the call to evt.CompletionSource.TrySetResult()? It turns out that this is responsible for more than 25% of the cost of the loop.
How do you fix that? Well, you can do this…
Task.Factory.StartNew(() =>
{
foreach (var evt in batch)
evt.CompletionSource.TrySetResult();
});In other words, you move the notification for the clients to the thread pool, freeing the event loop to process more messages.
The problem here is that the clients are what is actually generating the work for the event loop. So the loop now moves from spending time notifying the clients to spending time waiting for the new events.
The problem is that calling TrySetResult() will try to execute the continuation for this code inline. That means that there is a single thread notifying all the clients. This is because the .NET thread pool will default to registering work to the local thread, and will spread the work to the rest of the cores using work stealing.
There is a good way to handle that: we can set those task completion sources with the RunContinuationsAsynchronously flag. We can go back to calling TrySetResult() inline in the event loop, and the RunContinuationsAsynchronously flag will ensure that this is very fast.
The problem is that the event loop is not a thread pool thread. So this will force the continuations for those tasks to be registered in the global thread pool. Then we pay additional costs with work stealing to actually get these continuations executed (basically, we contend on each and every one of them).
Instead, we can write this nonsensical code (without the RunContinuationsAsynchronously flag) and execute it inline in the event loop code:
const int BatchSize = 16;
for (var i = 0; i < workers.Length; i += BatchSize)
{
int count = Math.Min(BatchSize, workers.Length - i);
var segment = new ArraySegment<Worker>(workers, i, count);
ThreadPool.UnsafeQueueUserWorkItem(static seg =>
{
for (int j = 0; j < seg.Count; j++)
{
seg[j].CompletionSource.TrySetResult();
}
}, segment, preferLocal: false);
}This will register batches of 16 notifications on the global thread pool. So instead of contending on every single notification, we’ll have far less contention.
Overall, this sped up the scenario we tested by a lot. And the only thing that we had to do was actually understand, at a pretty deep level, exactly the impact that this code has and how it interacts with its environment.
That is the sort of thing that you gain only from actually experiencing a platform, not merely writing code for it.