Parallel LINQ (PLINQ)
Parallel LINQ, also known as PLINQ, is a parallel implementation of LINQ to objects. PLINQ
implements all the LINQ query extension methods and has additional operators for parallel
operations. The degree of concurrency for PLINQ queries is based on the capabilities of the
computer running the query.
In many, but not all, scenarios, PLINQ can provide a significant increase in speed by using
all available CPUs or CPU cores. A PLINQ query can provide performance gains when
you have CPU-intensive operations that can be paralleled, or divided, across each CPU or
CPU core. The more computationally expensive the work is, the greater the opportunity for
performance gain. For example, if the workload takes 100 milliseconds to execute, a sequential
query over 400 elements will take 40 seconds to complete the work, whereas a parallel
query on a computer with eight cores might take only 5 seconds. This yields a speedup of 35
seconds.
One problem with Windows applications is that when you try to update a control
on your form from a thread other than the thread that created the control, an
InvalidOperationException
is thrown with the message, “Cross-thread operation not valid:
Control ‘txtLog’ accessed from a thread other than the thread it was created on.” To work with
threading, update in a thread-safe way the following extension method for TextBox to the
TextBoxHelper class.
Sample of Visual Basic Code
<Extension()> _
Public Sub WriteLine(ByVal txt As TextBox, _
ByVal format As String, _
ByVal ParamArray parms As Object())
Dim line As String = String.Format((format & Environment.NewLine), parms)
If txt.InvokeRequired Then
txt.BeginInvoke(New Action(Of String)(AddressOf txt.AppendText), _
New Object() {line})
Else
txt.AppendText(line)
224 CHAPTER 3 Introducing LINQ
End If
End Sub
Sample of C# Code
public static void WriteLine(this TextBox txt,
string format, params object[] parms)
{
string line = string.Format(format + Environment.NewLine, parms);
if (txt.InvokeRequired)
{
txt.BeginInvoke((Action<string>)txt.AppendText, line);
}
else
{
txt.AppendText(line);
}
}
You use the Invoke or BeginInvoke method on the TextBox class to marshal the callback to
the thread that was used to create the UI control. The BeginInvoke method posts an internal
dedicated Windows message to the UI thread message queue and returns immediately, which
helps avoid thread deadlock situations.
This extension method checks the TextBox object to see whether marshaling is required. If
marshaling is required (i.e., when the calling thread is not the one used to create the TextBox
object), the BeginInvoke method is executed. If marshaling is not required, the AppendText
method is called directly on the TextBox object. The BeginInvoke method takes Delegate as
a parameter, so txt.AppendText is cast to an action of String, a general-purpose delegate
that exists in the framework, which represents a call to a method that takes a string parameter.
Now that there is a thread-safe way to display information into the TextBox class, the
AsParallel
example can be performed without risking threading-related exceptions.
AsParallel Extension Method
The AsParallel extension method divides work onto each processor or processor core. The
following code sample starts a stopwatch in the System.Diagnostics namespace to show you
the elapsed time when completed, and then the Enumerable class produces a sequence of
integers, from 1 to 10. The AsParallel method call is added to the source. This causes the
iterations to be spread across the available processor and processor cores. Then a LINQ
query retrieves all the even numbers, but in the LINQ query, the where clause is calling a
Compute method, which has a one-second delay using the Thread class, which is in the
System.
Threading
namespace. Finally, a foreach loop displays the results.
Sample of Visual Basic Code
Private Sub AsParallelToolStripMenuItem_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) _
Handles AsParallelToolStripMenuItem.Click
Dim sw As New Stopwatch
sw.Start()
Dim source = Enumerable.Range(1, 10).AsParallel()
Dim evenNums = From num In source
Where Compute(num) Mod 2 = 0
Select num
For Each ev In evenNums
txtLog.WriteLine("{0} on Thread {1}", _
New Object() {ev, Thread.CurrentThread.GetHashCode})
Next
sw.Stop()
txtLog.WriteLine("Done {0}", New Object() {sw.Elapsed})
End Sub
Public Function Compute(ByVal num As Integer) As Integer
txtLog.WriteLine("Computing {0} on Thread {1}", _
New Object() {num, Thread.CurrentThread.GetHashCode})
Thread.Sleep(1000)
Return num
End Function
Sample of C# Code
private void asParallelToolStripMenuItem_Click(
object sender, EventArgs e)
{
Stopwatch sw = new Stopwatch();
sw.Start();
var source = Enumerable.Range(1, 10).AsParallel();
var evenNums = from num in source
where Compute(num) % 2 == 0
select num;
foreach (var ev in evenNums)
{
txtLog.WriteLine("{0} on Thread {1}", ev,
Thread.CurrentThread.GetHashCode());
}
sw.Stop();
txtLog.WriteLine("Done {0}", sw.Elapsed);
}
public int Compute(int num)
{
txtLog.WriteLine("Computing {0} on Thread {1}", num,
Thread.CurrentThread.GetHashCode());
Thread.Sleep(1000);
return num;
}
AsEnumerable results, showing even numbers, total time, and computing method:
6 on Thread 10
2 on Thread 10
4 on Thread 10
8 on Thread 10
10 on Thread 10
Done 00:00:05.0393262
Computing 1 on Thread 12
226 CHAPTER 3 Introducing LINQ
Computing 2 on Thread 11
Computing 3 on Thread 12
Computing 4 on Thread 11
Computing 5 on Thread 11
Computing 6 on Thread 12
Computing 7 on Thread 12
Computing 8 on Thread 11
Computing 9 on Thread 12
Computing 10 on Thread 11
The output from the Compute calls always shows after the foreach (Visual Basic For Each)
loop output because BeginInvoke marshalls calls to the UI thread for execution when the
UI thread is available. The foreach loop is running on the UI thread, so the thread is busy
until the loop completes. The results are not ordered. Your result will vary as well, and, in
some cases, the results might be ordered. In the example, you can see that the foreach loop
displayed the even numbers, using the main thread of the application, which was thread 10
on this computer. The Compute method was executed on a different thread, but the thread
is either 11 or 12 because this is a two-core processor. Although the Compute method has a
one-second delay, it took five seconds to execute because only two threads were allocated,
one for each core.
In an effort to get a clearer picture of PLINQ, the writing to a TextBox has been replaced
in the following code. Instead of using TextBox, Debug.WriteLine is used, which removes the
requirement to marshall calls back to the UI thread.
Sample of Visual Basic Code
Private Sub AsParallel2ToolStripMenuItem_Click( _
ByVal sender As System.Object, ByVal e As System.EventArgs) _
Handles AsParallel2ToolStripMenuItem.Click
Dim sw As New Stopwatch
sw.Start()
Dim source = Enumerable.Range(1, 10).AsParallel()
Dim evenNums = From num In source
Where Compute2(num) Mod 2 = 0
Select num
For Each ev In evenNums
Debug.WriteLine(String.Format("{0} on Thread {1}", _
New Object() {ev, Thread.CurrentThread.GetHashCode}))
Next
sw.Stop()
Debug.WriteLine(String.Format("Done {0}", New Object() {sw.Elapsed}))
End Sub
Sample of C# Code
private void asParallel2ToolStripMenuItem_Click(
object sender, EventArgs e)
{
Stopwatch sw = new Stopwatch();
sw.Start();
var source = Enumerable.Range(1, 10).AsParallel();
var evenNums = from num in source
where Compute2(num) % 2 == 0
select num;
foreach (var ev in evenNums)
{
Debug.WriteLine(string.Format("{0} on Thread {1}", ev,
Thread.CurrentThread.GetHashCode()));
}
sw.Stop();
Debug.WriteLine(string.Format("Done {0}", sw.Elapsed));
}
public int Compute2(int num)
{
Debug.WriteLine(string.Format("Computing {0} on Thread {1}", num,
Thread.CurrentThread.GetHashCode()));
Thread.Sleep(1000);
return num;
}
The result:
Computing 2 on Thread 10
Computing 1 on Thread 6
Computing 3 on Thread 10
Computing 4 on Thread 6
Computing 5 on Thread 10
Computing 6 on Thread 6
Computing 7 on Thread 10
Computing 8 on Thread 6
Computing 9 on Thread 10
Computing 10 on Thread 6
2 on Thread 9
4 on Thread 9
6 on Thread 9
8 on Thread 9
10 on Thread 9
Done 00:00:05.0632071
The result, which is in the Visual Studio .NET Output window, shows that there is no waiting
for the UI thread. Once again, your result will vary based on your hardware configuration.
ForAll Extension Method
When the query is iterated by using a foreach (Visual Basic For Each) loop, each iteration is
synchronized in the same thread, to be treated one after the other in the order of the sequence.
If you just want to perform each iteration in parallel, without any specific order, use
the ForAll method. It has the same effect as performing each iteration in a different thread.
Analyze this technique to verify that you get the performance gain you expect. The following
example shows the use of the ForAll method instead of the For Each (C# foreach) loop.
Sample of Visual Basic Code
Private Sub ForAllToolStripMenuItem_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) _
228 CHAPTER 3 Introducing LINQ
Handles ForAllToolStripMenuItem.Click
Dim sw As New Stopwatch
sw.Start()
Dim source = Enumerable.Range(1, 10).AsParallel()
Dim evenNums = From num In source
Where Compute2(num) Mod 2 = 0
Select num
evenNums.ForAll(Sub(ev) Debug.WriteLine(string.Format(
"{0} on Thread {1}", ev, _
Thread.CurrentThread.GetHashCode())))
sw.Stop()
Debug.WriteLine((string.Format("Done {0}", New Object() {sw.Elapsed}))
End Sub
Sample of C# Code
private void forAllToolStripMenuItem_Click(object sender, EventArgs e)
{
Stopwatch sw = new Stopwatch();
sw.Start();
var source = Enumerable.Range(1, 10).AsParallel();
var evenNums = from num in source
where Compute(num) % 2 == 0
select num;
evenNums.ForAll(ev => Debug.WriteLine(string.Format(
"{0} on Thread {1}", ev,
Thread.CurrentThread.GetHashCode())));
sw.Stop();
Debug.WriteLine(string.Format("Done {0}", sw.Elapsed));
}
ForAll result, showing even numbers, total time, and computing method:
Computing 1 on Thread 9
Computing 2 on Thread 10
Computing 3 on Thread 9
2 on Thread 10
Computing 4 on Thread 10
Computing 5 on Thread 9
4 on Thread 10
Computing 6 on Thread 10
Computing 7 on Thread 9
6 on Thread 10
Computing 8 on Thread 10
Computing 9 on Thread 9
8 on Thread 10
Computing 10 on Thread 10
10 on Thread 10
Done 00:00:05.0556551
Like the previous example, the results are not guaranteed to be ordered, and there is no
attempt to put the results in a particular order. This technique can give you better performance
as long as this behavior is acceptable.
AsOrdered Extension Method
Sometimes, you must maintain the order in your query, but you still want parallel execution.
Although this will come at a cost, it’s doable by using the AsOrdered extension method. The
following example shows how you can add this method call right after the AsParallel method
to maintain order.
Sample of Visual Basic Code
Private Sub AsOrderedToolStripMenuItem_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) _
Handles AsOrderedToolStripMenuItem.Click
Dim sw As New Stopwatch
sw.Start()
Dim source = Enumerable.Range(1, 10).AsParallel().AsOrdered()
Dim evenNums = From num In source
Where Compute2(num) Mod 2 = 0
Select num
evenNums.ForAll(Sub(ev) Debug.WriteLine(string.Format(
"{0} on Thread {1}", ev, _
Thread.CurrentThread.GetHashCode())))
sw.Stop()
Debug.WriteLine(string.Format("Done {0}", New Object() {sw.Elapsed}))
End Sub
Sample of C# Code
private void asOrderedToolStripMenuItem_Click(object sender, EventArgs e)
{
Stopwatch sw = new Stopwatch();
sw.Start();
var source = Enumerable.Range(1, 10).AsParallel().AsOrdered();
var evenNums = from num in source
where Compute2(num) % 2 == 0
select num;
evenNums.ForAll(ev => Debug.WriteLine(string.Format(
"{0} on Thread {1}", ev,
Thread.CurrentThread.GetHashCode())));
sw.Stop();
Debug.WriteLine(string.Format("Done {0}", sw.Elapsed));
}
AsOrdered result, showing even numbers, total time, and computing method:
Computing 2 on Thread 11
Computing 1 on Thread 10
2 on Thread 11
Computing 4 on Thread 11
Computing 3 on Thread 10
4 on Thread 11
Computing 6 on Thread 11
Computing 5 on Thread 10
6 on Thread 11
Computing 8 on Thread 11
Computing 7 on Thread 10
8 on Thread 11
230 CHAPTER 3 Introducing LINQ
Computing 9 on Thread 11
Computing 10 on Thread 10
10 on Thread 10
Done 00:00:05.2374586
The results are ordered, at least for the even numbers, which is what the AsOrdered extension
method is guaranteeing.
joi, 9 februarie 2012
miercuri, 8 februarie 2012
Lambda Expressions
http://blogs.msdn.com/b/ericwhite/archive/2006/10/03/lambda-expressions.aspx
Lambda Expressions
3 Oct 2006 2:43 PM
Note: This article is a topic in a much larger tutorial on writing pure functional code in C#. Lambda expressions are only one of the pieces in a much larger puzzle. Functional programming in C# allows us to write shorter programs that perform better, and are much more robust. Query Composition using Functional Programming Techniques explains lambda expressions in their proper context.
In order to learn functional programming and a more declarative style of writing code, we need first to cover some basic material. One of the first concepts is that of lambda expressions. Lambda expressions can be summarized in one sentence:
Lambda expressions are simply functions/methods.
They have a different syntax, primarily so that they can be written in expression context (more on this shortly) instead of as a member of a class. However, that is all they are. For instance, the following lambda expression:
c => c + 1
is a function that takes one argument, c, and returns the value c + 1.
Actually, they are slightly more complicated than this, but not much more. For the purposes of this tutorial, you only use lambda expressions when calling a method that takes a delegate as a parameter. Instead of writing a method, creating a delegate from the method, and passing the delegate to the method as a parameter, you can simply write a lambda expression in-line as a parameter to the method.
To show lambda expressions in context, consider the problem where you have an array with 10 digits in it, and you want to filter for all digits greater than 5. In this case, you can use the Where extension method, passing a lambda expression as an argument to the Where method:
int[] source = new[] { 3, 8, 4, 6, 1, 7, 9, 2, 4, 8 };
foreach (int i in source.Where(x => x > 5))
Console.WriteLine(i);
foreach (int i in source.Where(x => x > 5))
Console.WriteLine(i);
To understand the semantics of this code, you needn't find some method elsewhere in the source code that does the selection; the code that you need to read is much tighter and smaller. It reflects your intent in a much cleaner fashion.
Later on in this tutorial, you'll see a number of uses of the standard query operators. Many of the standard query operators, including Where, take delegates as an argument, so this means that we can call them passing a lambda as an argument.
First, a quick review of delegates:
Defining, Creating, and Using a Delegate
In C#, a delegate is a data structure that refers to either a static method, or an object and an instance method of its class. When you initialize a delegate, you initialize it with either a static method, or a class instance and an instance method.
The following code shows the definition of a delegate and a method that can be used to initialize the delegate:
// Defines a delegate that takes an int and returns an int
public delegate int ChangeInt(int x);
// Define a method to which the delegate can point
static public int DoubleIt(int x)
{
return x * 2;
}
public delegate int ChangeInt(int x);
// Define a method to which the delegate can point
static public int DoubleIt(int x)
{
return x * 2;
}
Now, you can create and initialize an instance of the delegate, and then call it:
ChangeInt myDelegate = new ChangeInt(DelegateSample.DoubleIt);
Console.WriteLine("{0}", myDelegate(5));
Console.WriteLine("{0}", myDelegate(5));
This, as you would expect, writes 10 to the console.
Using an Anonymous Method
With C# 2.0, anonymous methods allow you to write a method and initialize a delegate in place:
ChangeInt myDelegate = new ChangeInt(
delegate(int x)
{
return x * 2;
}
);
Console.WriteLine("{0}", myDelegate(5));
delegate(int x)
{
return x * 2;
}
);
Console.WriteLine("{0}", myDelegate(5));
Using a Lambda Expression
With Lambda expressions, the syntax gets even terser:
ChangeInt myDelegate = x => x * 2;
Console.WriteLine("{0}", myDelegate(5));
Console.WriteLine("{0}", myDelegate(5));
This lambda expression is an anonymous method that takes one argument x, and returns x * 2. In this case, the type of x and the type that the lambda returns are inferred from the type of the delegate to which the lambda is assigned.
If you wanted to, you could have specified the type of the argument, as follows:
ChangeInt myDelegate = (int x) => x * 2;
Console.WriteLine("{0}", myDelegate(5));
Console.WriteLine("{0}", myDelegate(5));
Using a Lambda with Two Arguments
When using the Standard Query Operators, on occasion, you need to write a lambda expression that takes two arguments.
If you have a delegate that takes two arguments:
// Defines a delegate that takes two ints and returns an int
public delegate int MultiplyInts(int arg, int arg2);
public delegate int MultiplyInts(int arg, int arg2);
You can declare and initialize a delegate:
MultiplyInts myDelegate = (a, b) => a * b;
Console.WriteLine("{0}", myDelegate(5, 2));
Console.WriteLine("{0}", myDelegate(5, 2));
Statement Lambda Expressions
You can write a more complicated lambda expression using statements, enclosing the statements in braces. If you use this syntax, you must use the return statement, unless the lambda returns void:
int[] source = new[] { 3, 8, 4, 6, 1, 7, 9, 2, 4, 8 };
foreach (int i in source.Where(
x =>
{
if (x <= 3)
return true;
else if (x >= 7)
return true;
return false;
}
))
Console.WriteLine(i);
foreach (int i in source.Where(
x =>
{
if (x <= 3)
return true;
else if (x >= 7)
return true;
return false;
}
))
Console.WriteLine(i);
Sometimes developers wonder how to pronounce the => token.
If the lambda expression is a predicate, expressing some condition: c => c.State == "WA" then the => can be spoken as "such that". In this example, you could say "c such that c dot state equals Washington". If the lambda expression is a projection, returning a new type: c => new XElement("CustomerID", c.CustomerID); then the => can be spoken as "becomes". In the above example, you could say "c becomes new XElement with a name of CustomerID and its value is c dot CustomerID". Or "maps to", or "evaluate to", as suggested in the comments below. But most often, I just say "arrow". J
A quick note: predicates are simply boolean expressions that are passed to some method that will use the boolean expression to filter something. A lambda expression used for projection takes one type, and returns a different type. More on both of these concepts later.
Lambda Expressions that Return Void
A lambda expression that returns void is not very useful in the context of functional programming because the only possible reason for such a function is that it has side-effects, and is not pure (more on this later in the tutorial), but it is part of C# 3.0 syntax, so I'll cover it here. Sometimes developers will use a void statement lambda expression for writing an event handler. This has the benefit that the syntax is terser, and the program is smaller. In addition, the lambda expression can refer to local variables in the enclosing scope. This is part of C#'s implementation of closures. The only way to write a lambda expression that returns void is to write a statement lambda expression. The following example shows defining a void delegate, declaring an instance of it, and calling it.
// Defines a delegate that takes a string and returns void
public delegate void OutputToConsole(string arg);
static void Main(string[] args)
{
OutputToConsole o = a => {
Console.WriteLine(a);
};
o("Hello, World");
}
public delegate void OutputToConsole(string arg);
static void Main(string[] args)
{
OutputToConsole o = a => {
Console.WriteLine(a);
};
o("Hello, World");
}
If you write a lambda expression for a delegate that returns void and takes no arguments, it results in interesting syntax:
// Defines a delegate that takes no arguments and returns void
public delegate void OutputHelloToConsole();
static void Main(string[] args)
{
OutputHelloToConsole o = () =>
{
Console.WriteLine("Hello, World");
};
o();
}
public delegate void OutputHelloToConsole();
static void Main(string[] args)
{
OutputHelloToConsole o = () =>
{
Console.WriteLine("Hello, World");
};
o();
}
The Func Delegate Types
The framework defines a number of parameterized delegate types:
public delegate TR Func<TR>();
public delegate TR Func<T0, TR>(T0 a0);
public delegate TR Func<T0, T1, TR>(T0 a0, T1 a1);
public delegate TR Func<T0, T1, T2, TR>(T0 a0, T1 a1, T2 a2);
public delegate TR Func<T0, T1, T2, T3, TR>(T0 a0, T1 a1, T2 a2, T3 a3);
public delegate TR Func<T0, TR>(T0 a0);
public delegate TR Func<T0, T1, TR>(T0 a0, T1 a1);
public delegate TR Func<T0, T1, T2, TR>(T0 a0, T1 a1, T2 a2);
public delegate TR Func<T0, T1, T2, T3, TR>(T0 a0, T1 a1, T2 a2, T3 a3);
In the above delegate types, notice that if there is only one type parameter, it is the return type of the delegate. If there are two type parameters, the first type parameter is the type of the one and only argument, and the second type is the return type of the delegate, and so on. Many of the standard query operators (which are just methods that you call) take as an argument a delegate of one of these types. These delegate definitions are useful to you when writing your own methods that take a delegate as an argument.
using System;
using System.Collections.Generic;
class Program
{
static List<T> MyWhereMethod<T>(IEnumerable<T> source,
Func<T, bool> predicate)
{
List<T> l = new List<T>();
foreach (T item in source)
if (predicate(item))
l.Add(item);
return l;
}
static void Main(string[] args)
{
int[] source = new[] { 3, 8, 4, 6, 1, 7, 9, 2, 4, 8 };
List<int> filteredList = MyWhereMethod(source,
i => i >= 5);
foreach (int z in filteredList)
Console.WriteLine(z);
}
}
using System.Collections.Generic;
class Program
{
static List<T> MyWhereMethod<T>(IEnumerable<T> source,
Func<T, bool> predicate)
{
List<T> l = new List<T>();
foreach (T item in source)
if (predicate(item))
l.Add(item);
return l;
}
static void Main(string[] args)
{
int[] source = new[] { 3, 8, 4, 6, 1, 7, 9, 2, 4, 8 };
List<int> filteredList = MyWhereMethod(source,
i => i >= 5);
foreach (int z in filteredList)
Console.WriteLine(z);
}
}
The Action Delegate Types
The framework defines a number of parameterized delegate types for delegates that return void:
public delegate void Action();
public delegate void Action<T0>(T0 a0);
public delegate void Action<T0, T1>(T0 a0, T1 a1);
public delegate void Action<T0, T1, T2>(T0 a0, T1 a1, T2 a2);
public delegate void Action<T0, T1, T2, T3>(T0 a0, T1 a1, T2 a2, T3 a3);
public delegate void Action<T0>(T0 a0);
public delegate void Action<T0, T1>(T0 a0, T1 a1);
public delegate void Action<T0, T1, T2>(T0 a0, T1 a1, T2 a2);
public delegate void Action<T0, T1, T2, T3>(T0 a0, T1 a1, T2 a2, T3 a3);
Sometimes API designers will include an event that takes one of these delegate types as an argument, and you can write a lambda expression for the argument. As with the Func delegate types, these delegate definitions are useful to you when writing your own methods that take a delegate as an argument. This uses the interesting syntax of () => { /* body of void function here */ };
using System;
using System.Collections.Generic;
using System.Threading;
class Program
{
static void SomeAsynchronousMethod(Action complete)
{
// just pretending to be asynchronous in this example
Thread.Sleep(1000);
complete();
}
static void Main(string[] args)
{
SomeAsynchronousMethod(() => { Console.WriteLine("Done"); });
}
}
using System.Collections.Generic;
using System.Threading;
class Program
{
static void SomeAsynchronousMethod(Action complete)
{
// just pretending to be asynchronous in this example
Thread.Sleep(1000);
complete();
}
static void Main(string[] args)
{
SomeAsynchronousMethod(() => { Console.WriteLine("Done"); });
}
}
Expression Trees
Lambda expressions can also be used as expression trees. This is an interesting topic, but is not part of this discussion on writing pure functional transformations.
Abonați-vă la:
Postări (Atom)