📝 Lesson 2.3: Data Parallelism: Parallel and PLINQ

Data Parallelism

In Lesson 2.1 you used Task.Run for task parallelism — running different operations at once. Data parallelism is a common special case: applying the same operation to many data items, split across cores.

You did this manually in Lesson 2.1 (splitting a range into four Task.Run chunks). The tools in this lesson automate that pattern.

Parallel.For and Parallel.ForEach

Parallel.For and Parallel.ForEach look like ordinary loops but distribute their iterations across pool threads. They block until all iterations finish.

using System.Threading.Tasks;

// Like: for (int i = 0; i < 1000; i++) Process(i);  — but across cores
Parallel.For(0, 1000, i =>
{
    Process(i);
});

// Like foreach — over any IEnumerable
var files = Directory.GetFiles("data");
Parallel.ForEach(files, file =>
{
    ProcessFile(file);
});

You can cap the parallelism with ParallelOptions — useful to leave cores free or limit resource pressure:

var options = new ParallelOptions { MaxDegreeOfParallelism = 4 };
Parallel.ForEach(items, options, item => Process(item));

Aggregating Safely

A very common need is to combine results from every iteration — a sum, a count, a list. Doing it naively creates a race. There are two clean approaches.

Approach 1: a thread-safe accumulator

Use Interlocked or a concurrent collection (Lesson 2.2):

long total = 0;
Parallel.For(0, 1_000_000, i =>
{
    Interlocked.Add(ref total, Work(i));   // atomic — safe
});

Approach 2: thread-local partials (faster)

Better for hot loops: each thread accumulates into its own local total, and only combines at the end — minimizing contention. Parallel.For has an overload built exactly for this:

long total = 0;

Parallel.For(
    0, 1_000_000,
    () => 0L,                        // (1) initialize each thread's local total
    (i, state, localTotal) =>        // (2) body: add into the LOCAL total
    {
        return localTotal + Work(i);
    },
    localTotal =>                     // (3) combine each thread's local into the global
    {
        Interlocked.Add(ref total, localTotal);
    });

Console.WriteLine(total);

PLINQ

Parallel LINQ (PLINQ) parallelizes LINQ queries with a single addition: .AsParallel(). The familiar operators (Where, Select, aggregates) then run across cores.

// Sequential LINQ (Intermediate)
var primes = numbers.Where(IsPrime).ToList();

// PLINQ — same query, across cores
var primesParallel = numbers.AsParallel().Where(IsPrime).ToList();

PLINQ shines for CPU-heavy per-item work over a large source. It handles partitioning, threading, and merging for you:

double sum = Enumerable.Range(1, 10_000_000)
    .AsParallel()
    .Where(n => n % 3 == 0)
    .Select(n => Math.Sqrt(n))
    .Sum();

When Parallelism Helps

Parallelism is not free — partitioning, scheduling across threads, and merging all add overhead. It pays off only when the parallelizable work is large enough to dwarf that overhead.

💡 Rule of thumb: Reach for data parallelism when you have a lot of independent, CPU-heavy items. For I/O-bound work, stick with async/await and Task.WhenAll (or Parallel.ForEachAsync) — spawning CPU threads to wait on I/O helps nothing.

Exercise & Quiz

using System.Diagnostics;

var sw = Stopwatch.StartNew();
double seq = Enumerable.Range(1, 2000).Select(HeavyCompute).Sum();
sw.Stop();
Console.WriteLine($"Sequential: {seq:F1} in {sw.ElapsedMilliseconds} ms");

// TODO: PLINQ version (AsParallel) + timing
// TODO: Parallel.For with thread-local partials + timing

static double HeavyCompute(int n)
{
    double acc = 0;
    for (int i = 0; i < 50_000; i++) acc += Math.Sqrt(n + i);
    return acc;
}

using System.Diagnostics;

var sw = Stopwatch.StartNew();
double seq = Enumerable.Range(1, 2000).Select(HeavyCompute).Sum();
sw.Stop();
Console.WriteLine($"Sequential: {seq:F1} in {sw.ElapsedMilliseconds} ms");

sw.Restart();
double plinq = Enumerable.Range(1, 2000).AsParallel().Select(HeavyCompute).Sum();
sw.Stop();
Console.WriteLine($"PLINQ:      {plinq:F1} in {sw.ElapsedMilliseconds} ms");

sw.Restart();
double total = 0;
object gate = new();
Parallel.For(
    1, 2001,
    () => 0.0,
    (i, state, local) => local + HeavyCompute(i),
    local => { lock (gate) { total += local; } });
sw.Stop();
Console.WriteLine($"Parallel.For: {total:F1} in {sw.ElapsedMilliseconds} ms");

static double HeavyCompute(int n)
{
    double acc = 0;
    for (int i = 0; i < 50_000; i++) acc += Math.Sqrt(n + i);
    return acc;
}

Summary

📚 Additional Resources

• Data parallelism (TPL) — Microsoft Docs
• Introduction to PLINQ
• Parallel class — API reference

🚀 What's Next?

That completes Module 2 — you can run work correctly and efficiently across cores! In Module 3, we return to asynchrony in depth. First: Lesson 3.1: Cancellation and Timeouts, giving async operations a way to stop.