What is an AutoResetEvent?

An AutoResetEvent can be described as a turnstile mechanism, it lets a single waiting person through before re-latching waiting for the next signal. This is opposed to a ManualResetEvent which functions like an ordinary gate. Calling Set opens the gate, allowing any number of threads that are waiting to be let through. Calling Reset closes the gate.

AsyncAutoResetEvent

First of all here is the shape of the type that we will be building:

1
2
3
4

type AsyncAutoResetEvent =
    new : ?reusethread:bool -> AsyncAutoResetEvent
    member Set : unit -> unit
    member WaitAsync : unit -> Async<bool>

Fairly simple: implied constructor, Set and WaitAsync members.

Implied Constructor

Thinking about this logically we may need the following items:

• A queue mechanism to store asynchronous waiters - let mutable awaits = Queue<_>().
• A way of knowing if a signal has been made in the absence of any waiters - let mutable signalled = false.
• We can also declare a short-circuit asynchronous workflow for the situation that Set() is called before WaitAsync()
• let completed = async.Return true. This will save us constructing an AsyncResultCell<_> and going though the rest of the asynchronous mechanism.

Also notice that an optional parameter called reusethread is defined, we use the ? prefix when defining it to make it optional. We then make use of the defaultArg function to give it a default value of false if a one is not passed in. This will be used in the Set operation to determine if the code will run on the same thread or a thread in the ThreadPool.

1
2
3
4
5
6
7
8
9

open System
open System.Threading
open System.Collections.Generic

    type AsyncAutoResetEvent(?reusethread) =
      let mutable awaits = Queue<_>()
      let mutable signalled = false
        let completed = async.Return true
        let reuseThread = defaultArg reusethread false

WaitAsync()

The first step is to use a locking construct to control access to the mutable queue awaits. Inside this lock we check to see if signalled is true and if so we reset it to false and return our pre-built completed asynchronous workflow. If signalled is false then we create a new AsyncResultCell<_> and add it to the queue then return the AsyncResult to the caller.

1
2
3
4
5
6
7
8
9

        member x.WaitAsync() =
            lock awaits (fun () ->
                if signalled then
                    signalled <- false
                    completed
                else
                    let are = AsyncResultCell<_>()
                    awaits.Enqueue are
                    are.AsyncResult)

Set()

We first declare a function called getWaiter(), we use this function to return an option type that is either Some AsyncResultCell<bool> or None. We use the lock function to control access to the mutable queue lock awaits. Once inside the lock we use pattern matching to capture awaits.Count and signalled:

• The first pattern match (x,_) checks if there are any waiters (awaits.Count > 0) and then dequeues an AsyncResultCell<bool> from the queue and returns it within an option type: Some <| awaits.Dequeue().
• The second pattern match (_,y) checks whether signalled is set to false before setting its value to true. This causes next WaitAsync() caller to get the short-circuited value completed. This means that an AsyncResultCell<bool> does not need to be created and go though the whole async mechanism. We then return None as there is no waiter to be notified.
• The final pattern match (_,_) is used when there are no waiting callers and signalled has already being set, there is simply nothing to do in this situation so we return None.

We use the getWaiter() function via pattern match. If we have a result i.e. Some AsyncResultCell then we call RegisterResult passing in AsyncOK(true) to indicate a completion. Notice that we also pass in the reuseThread boolean that was declared as part of the constructor. If reuseThread is true then the notification to the waiter happens synchronously use this with care! Personally I would stick with the default of false to ensure that the operation is completed via the thread pool, unless you have a performance critical reason and the waiting code that executes is very fast.

1
2
3
4
5
6
7
8
9
10

       member x.Set() =
          let getWaiter()=
              lock awaits (fun () ->
                  match (awaits.Count, signalled) with
                  | (x,_) when x > 0 -> Some <| awaits.Dequeue()
                  | (_,y) when not y -> signalled <- true;None
                  | (_,_) -> None)
          match getWaiter() with
          | Some a -> a.RegisterResult(AsyncOk(true), reuseThread)
          | None _ -> ()

The reason for using the getWaiter() function is to separate the locking function away from the notification, if RegisterResult was called within the lock and reuseThread was true then the awaiting function would be called synchronously within the lock which would not be a very good situation to be in.

So there we have it, I could take this series further and convert the other primitives that Stephen Toub describes but there should be enough information in these two posts to set you on your way. If anyone would like me to complete the series then let me know. I may well finish them off and post them on GitHub in the future, time permitting.

Musical inspiration during the creation of this post:

• Pantera - Cowboys From Hell
• Cacophony - Go Off
  

Thanks for tuning in, until next time…

We are going to be looking at an asynchronous version of the ManualResetEvent. This was recently covered by Stephen Toub on the pfx team blog. We will be taking a slightly different view on this as we will be using asynchronous workflows which will give us nice idiomatic usage within F#.

First lets look of the shape of the type that Stephen defined:

1
2
3
4
5
6

public class AsyncManualResetEvent
{
    public Task WaitAsync();
    public void Set();
    public void Reset();
}

Now this can be used from within F# by using the Async.AwaitTask function from the Async module but this is like wrapping one asynchronous paradigm with another, and although this does work, what if you want to avoid the overhead of wrappers and stay strictly within async workflows.

1
2
3
4

type asyncManualResetEvent() =
    member x.WaitAsync() : unit -> Async<bool>
    member x.Set() : unit -> unit
    member x.Reset() : unit -> unit

That’s what we want to see! I don’t want to get into the details of the description of how the C# version works as Stephen does a very good job of that already. What I will explain though is how we essentially do the same thing while staying with the realm of functional programming. As we are getting into the lower lever details no doubt we will have to start relying on some low level locking primitives like Monitors, Semaphores, and Interlocked operations, even the F# core libraries have a cornucopia of those.

Lets look at the first member WaitAsync(). The first step is to create a something to store the result of the operation, all we will just be storing and returning asynchronously is a boolean to indicate that the wait handle has been set. To do this we use one of the types from the F# power pack AsyncResultCell<'T>. I think that such a type should of been exposed from the F# core libraries but it was omitted for some reason. There is a type called ResultCell<'T> with much the same functionality in the FSharp.Core.Control namespace but it is marked internal so it’s not available for our use.

We declare a reference cell of type AsyncResultCell<'T> and then create the WaitAsync() member, all we have to do is dereference the value of the reference cell with ! and call its AsyncResult member, this gives us an Async<bool> which we can easily use in an asynchronous workflow.

1
2
3
4

type asyncManualResetEvent() =
    let aResCell = ref <| AsyncResultCell<_>()

    member x.WaitAsync() = (!aResCell).AsyncResult

The next bit is fairly simple too. All we need to do is dereference the value of the reference cell, and invoke the RegisterResult member by passing in a value of AsyncOk(true). The boolean value of true will be used by the type inference system to constrain the value of the Async<_> returned from WaitAsync.

1

    member x.Set() = (!aResCell).RegisterResult(AsyncOk(true))

The last part is the most complex (as usual). Here we create a recursive function called swap that will try to exchange the AsyncResultCell<'T> for a new one. We dereference the reference cell to currentValue, then we use a CAS (Compare And Swap) operation to compare the aResCell with currentValue and if they are equal newVal will replace aResCell. On the next line if the result of the CAS operation means that result and currentValue are equal then we are finished, otherwise we spin the current thread for 20 cycles using Thread.SpinWait 20 before retrying the operation via recursion swap newVal. This will be a lot less expensive than switching to user or kernel mode locking, and the period of contention between threads should be very small. Finally the swap operation is started by passing in a new AsyncResultCell<'T>.

There are various other methods we could of used, for instance we could of wrapped a ManualResetEvent with a call to Async.AwaitWaitHandle, although this would of meant using the kernel mode locking of the ManualResetEvent which is a bit more expensive.

In Stephen Toub’s post he mentions Task’s being orphaned due to the Reset() method being called before the Task<'T> has been completed, that shouldn’t happen in our implementation due the the closures being stored internally for completion by the async infrastructure. Heres a quick test harness to make sure everything works as expected anyway.

1
2
3
4
5
6
7
8

    member x.Reset() =
        let rec swap newVal =
            let currentValue = !aResCell
            let result = Interlocked.CompareExchange<_>(aResCell, newVal, currentValue)
            if obj.ReferenceEquals(result, currentValue) then ()
            else Thread.SpinWait 20
                 swap newVal
        swap <| AsyncResultCell<_>()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

let amre = asyncManualResetEvent()
let x = async{let! x = amre.WaitAsync()
              Console.WriteLine("First signalled")}

let y = async{let! x = amre.WaitAsync()
             Console.WriteLine("Second signalled")}

let z = async{let! x = amre.WaitAsync()
              Console.WriteLine("Third signalled")}
//start async workflows x and y
Async.Start x
Async.Start y

//reset the asyncManualResetEvent, this will test whether the async workflows x and y 
// are orphaned due to the AsyncResultCell being recycled.
amre.Reset()

//now start the async z
Async.Start z

//we set a single time, this should result in the three async workflows completing
amre.Set()

Console.ReadLine() |> ignore

Here we can see everything works out as we expected:

Thats all there is too it, next time I will be exploring an asyncAutoResetEvent in much the same vein.

Musical inspiration during the creation of this post:

• Smashing Pumpkins - Zeitgeist
• Soulfly - Primitive
• FooFighters - FooFighters
  

Until next time…

I recently presented a paper on the benefits of F#, part of this was a comparison of the famous Black-Scholes equation in both C# and F#. I was mainly going to be looking at code succinctness and the inherent suitability of the language for calculation based work, but there ended up being more to it than that.

First of all I quickly set up a test rig to run 50 million iterations of the algorithm to see if there were any difference in the processing speed. I want expecting any major differences at this point but here’s what I got:

C# results for 50 million iterations

F# results for 50 million iterations

I think you will agree that’s quite a difference, lets have a look at the code to see what’s going on.

C# Implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

public class Options
{
    public enum Style
    {
        Call,
        Put
    }

    public static double BlackScholes(Style callPut, double s, double x, double t, double r, double v)
    {
        double result = 0.0;
        var d1 = (Math.Log(s / x) + (r + v * v / 2.0) * t) / (v * Math.Sqrt(t));
        var d2 = d1 - v * Math.Sqrt(t);
        switch (callPut)
        {
            case Style.Call:
                result = s * Cnd(d1) -x * Math.Exp(-r * t) * Cnd(d2);
                break;
            case Style.Put:
                result = x * Math.Exp(-r * t) * Cnd(-d2) -s * Cnd(-d1);
                break;
        }
        return result;
    }

    private static double Cnd(double x)
    {
        const double a1 = 0.31938153;
        const double a2 = -0.356563782;
        const double a3 = 1.781477937;
        const double a4 = -1.821255978;
        const double a5 = 1.330274429;
        var l = Math.Abs(x);
        var k = 1.0 / (1.0 + 0.2316419 * l);
        var w = 1.0 - 1.0 / Math.Sqrt(2 * Math.PI) *
            Math.Exp(-l * l / 2.0) * (a1 * k + a2 * k * k + a3 *
                Math.Pow(k, 3) + a4 * Math.Pow(k, 4) + a5 * Math.Pow(k, 5));
        if (x < 0)
        {
            return 1.0 - w;
        }
        return w;
    }
}

F# Implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

module options
open System

type Style = Call | Put

let cnd x =
   let a1 = 0.31938153
   let a2 = -0.356563782
   let a3 = 1.781477937
   let a4 = -1.821255978
   let a5 = 1.330274429
   let l  = abs x
   let k  = 1.0 / (1.0 + 0.2316419 * l)
   let w  = (1.0 - 1.0 / sqrt(2.0 * Math.PI) *
                exp(-l * l / 2.0) * (a1 * k + a2 * k * k + a3 *
                    (pown k 3) + a4 * (pown k 4) + a5 * (pown k 5)))
   if x < 0.0 then 1.0 - w
   else w

let blackscholes style s x t r v =
    let d1 = (log(s / x) + (r + v * v / 2.0) * t) / (v * sqrt(t))
    let d2 = d1 - v * sqrt(t)
    match style with
    | Call -> s * cnd(d1) -x * exp(-r * t) * cnd(d2)
    | Put -> x * exp(-r * t) * cnd(-d2) -s * cnd(-d1)

Differences

The most significant differences when the code is compiled comes down to a few areas.

The BlackScholes function

The first thing to note is the code size and number of local variables:

1
2
3
4

// Code size       122 (0x7a)
.maxstack  6
.locals init ([0] float64 d1,
         [1] float64 d2)

1
2
3
4
5
6

// Code size       164 (0xa4)
.maxstack  4
.locals init ([0] float64 d1,
         [1] float64 d2,
         [2] float64 result,
         [3] valuetype CsBs.Options/Style CS$0$0000)

The initial arguments that are loaded in the F# implementation is done in fewer IL op codes then C#.

1
2
3
4

IL_0001:  ldarg.1
IL_0002:  ldarg.2
IL_0003:  div
IL_0004:  call       float64 [mscorlib]System.Math::Log(float64)

1
2
3
4
5
6
7
8
9
10

IL_0000:  ldc.r8     0.0
IL_0009:  stloc.0
IL_000a:  ldc.r8     0.0
IL_0013:  stloc.1
IL_0014:  ldc.r8     0.0
IL_001d:  stloc.2
IL_001e:  ldarg.1
IL_001f:  ldarg.2
IL_0020:  div
IL_0021:  call       float64 [mscorlib]System.Math::Log(float64)

You can see in the C# code is intialising the local variable to 0.0 by pushing them to the stack ldc.r8 then storing them stloc.0.

The pattern matching in the F# code results in a call to get the style options/Style::get_Tag() and then a branch if not equal opcode bne.un.s which causes a jump to IL_005d

1
2
3

IL_0036:  call       instance int32 options/Style::get_Tag()```
IL_003b:  ldc.i4.1
IL_003c:  bne.un.s   IL_005d

The C# version loads the local variable for the Style IL_0053: stloc.3 and then uses the switch opcode to jump table to jump to either position IL_0064 or IL_0083.

1
2
3
4
5
6

IL_0053:  stloc.3
IL_0054:  ldloc.3
IL_0055:  switch     ( 
                      IL_0064,
                      IL_0083)
IL_0062:  br.s       IL_00a2

These are negligible, I’m mealy pointing out the differences in compilation between the two languages.
The F# compiler is more stringent when compiling the code.

The Cnd function

The Cnd function or cumulative normal distribution is where the performance differences occur.

Again at initialization you can see the C# version is larger by 41.

1
2
3
4
5

// Code size       213 (0xd5)
.maxstack  8
.locals init ([0] float64 l,
         [1] float64 k,
         [2] float64 w)

1
2
3
4
5

// Code size       254 (0xfe)
.maxstack  6
.locals init ([0] float64 l,
         [1] float64 k,
         [2] float64 w)

The C# version initialises all the local variables to 0.0.

1
2
3
4
5
6

IL_0000:  ldc.r8     0.0
IL_0009:  stloc.0
IL_000a:  ldc.r8     0.0
IL_0013:  stloc.1
IL_0014:  ldc.r8     0.0
IL_001d:  stloc.2

Interestingly the C# compiler optimises out the call to Math.PI * 2 but the F# compiler doesn’t.

1
2
3

IL_003a:  ldc.r8     2.
IL_0043:  ldc.r8     3.1415926535897931
IL_004c:  mul

1

IL_0057:  ldc.r8     6.2831853071795862

From here everything is identical until we get to the power operator section (Math.Pow in the C# version and pown in F#).

1
2
3
4

IL_0089:  ldloc.1
IL_008a:  ldc.i4.3
IL_008b:  call       float64 [FSharp.Core]Microsoft.FSharp.Core.Operators/OperatorIntrinsics::PowDouble(float64, 
                                                                                                        int32)

In the F# code we are using the pown function which calculates the power to an integer. This is shown in the call to OperatorIntrinsics::PowDouble which uses the value in IL_0089: ldloc.1 and also loads the integer 3 with IL_008a: ldc.i4.3.

1
2
3
4

IL_009c:  ldloc.1
IL_009d:  ldc.r8     3.
IL_00a6:  call       float64 [mscorlib]System.Math::Pow(float64,
                                                        float64)

The C# code is using the standard Math.Pow operator which operates on two float64 numbers. The value of 3 is implicitly converted into a float64 during compilation IL_009d: ldc.r8 3..

The final difference is at the end of the function.

1
2
3
4
5
6
7
8
9
10
11

IL_00b8:  stloc.2
IL_00b9:  ldarg.0
IL_00ba:  ldc.r8     0.0
IL_00c3:  clt
IL_00c5:  brfalse.s  IL_00d3
IL_00c7:  ldc.r8     1.
IL_00d0:  ldloc.2
IL_00d1:  sub
IL_00d2:  ret
IL_00d3:  ldloc.2
IL_00d4:  ret

The F# version uses the clt opcode. This pushes 1 if value one on the stack is less than value two otherwise it pushes 0. There is then a brfalse.s which jumps to location IL_00d3 if the first value on the stack is less than or equal to the second value.

1
2
3
4
5
6
7
8
9
10

IL_00b8:  stloc.2
IL_00b9:  ldarg.0
IL_00e5:  ldc.r8     0.0
IL_00ee:  bge.un.s   IL_00fc
IL_00f0:  ldc.r8     1.
IL_00f9:  ldloc.2
IL_00fa:  sub
IL_00fb:  ret
IL_00fc:  ldloc.2
IL_00fd:  ret

The C# version uses the bge.un.s to jump to location IL_00fc if the first value on the stack is greater than the second. This is negligible in normal runtime but it is interesting to note the difference between the two.

Conclusion

Wow, there was a lot of IL to get through, I hope you stayed with me!

Although the difference in some areas are negligible, every little counts. The implicit conversion of an integer field to a float64 hides the fact that we were using an optimized integer power function in F#, that’s performance increase of 168%! Some other side effects of implicit conversion can also lead to subtle bugs due to truncation and overflow. The other benefits are the compiled code uses less instructions and the source code only uses 25 lines compared to 44 in C#.

Until next time!

First, lets start with the simple ones, these don’t really require much discussion.

DefaultTimeout

1
2
3
4
5

let mutable defaultTimeout = Timeout.Infinite

member x.DefaultTimeout
   with get() = defaultTimeout
   and set(value) = defaultTimeout <- value

This simply provides a mutable property using Timeout.Infinite as a default setting.

CurrentQueueLength

1

member x.CurrentQueueLength() = incomingMessages.Count

Another simple one, this methods uses into the underlying BufferBlock to extract its current queue length using its Count property.

TryReceive

1
2
3
4
5
6

member x.TryReceive(?timeout) =
    let ts = TimeSpan.FromMilliseconds(float <| defaultArg time out defaultTimeout)
    Async.AwaitTask <| incomingMessages.ReceiveAsync(ts)
                           .ContinueWith(fun (tt:Task<_>) ->
                                             if tt.IsCanceled || tt.IsFaulted then None
                                             else Some tt.Result)

Here we get a little help from TPL to apply a continuation on completion using ContinueWith. We use a lambda to return either None, in a time out condition, or Some tt.Result when we successfully receive an item.

TryPostAndReply

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

type AsyncResultCell<'a>() =
    ...
  member x.TryWaitResultSynchronously(timeout:int) =
      //early completion check
      if source.Task.IsCompleted then
          Some source.Task.Result
      //now force a wait for the task to complete
      else
          if source.Task.Wait(timeout) then
              Some source.Task.Result
          else None

member x.TryPostAndReply(replyChannelMsg, ?timeout) :'Reply option =
    let timeout = defaultArg timeout defaultTimeout
    let resultCell = AsyncResultCell<_>()
    let msg = replyChannelMsg(new AsyncReplyChannel<_>(fun reply -> resultCell.RegisterResult(reply)))
    if incomingMessages.Post(msg) then
        resultCell.TryWaitResultSynchronously(timeout)
    else None

Things get a little more interesting from here on in. Firstly we need to add a new synchronisation member to the AsyncResultCell<'a> type: TryWaitResultSynchronously. We again enlist the help of the TPL primitives to check for the early completion using source.Task.IsCompleted returning the result if it is there, otherwise we use the Task property’s Wait method to check the item returns within the time out interval. In the usual manner, Some source.Task.Result is returned or None for a failure.

PostAndReply

1
2
3
4

member x.PostAndReply(replyChannelMsg, ?timeout) : 'Reply =
    match x.TryPostAndReply(replyChannelMsg, ?timeout = timeout) with
    | None ->  raise (TimeoutException("PostAndReply timed out"))
    | Some result -> result

This one wraps a call to TryPostAndReply with some pattern matching. In the event of a time out None is returned from TryPostAndReply in this instance we raise a TimeoutException otherwise we unwrap the result from the option using | Some result -> result.

TryScan

1
2
3
4
5
6
7
8
9
10
11
12
13
14

member x.TryScan((scanner: 'Msg -> Async<_> option), timeout): Async<_ option> =
    let ts = TimeSpan.FromMilliseconds( float timeout)
    let rec loopForMsg = async {
        let! msg = Async.AwaitTask <| incomingMessages.ReceiveAsync(ts)
                                      .ContinueWith(fun (tt:Task<_>) ->
                                          if tt.IsCanceled || tt.IsFaulted then None
                                          else Some tt.Result)
        match msg with
        | Some m ->  let res = scanner m
                     match res with
                     | None -> return! loopForMsg
                     | Some res -> return! res
        | None -> return None}
    loopForMsg

This one also uses the same ContinueWith functionality in the recursive loopForMsg function, perhaps some of these functions could extracted out and refactored but I prefer to keep the code like this to better explain what’s going on. The the code is available on GitHub anyway so feel free to clean up any detritus and send me a pull request. Again we use pattern matching to keep calling the loopForMsg function until the result is returned or a time out occurs.

Scan

1
2
3
4
5

member x.Scan(scanner, timeout) =
    async { let! res = x.TryScan(scanner, timeout)
            match res with
            | None -> return raise(TimeoutException("Scan TimedOut"))
            | Some res -> return res }

Finally we have Scan, this is much like PostAndReply in that it just acts as a wrapper around TryScan making use of pattern matching throwing an exception on a time out.

That sums up the last few pieces, completing the TDF implementation of the MailboxProcessor. I think this series of posts has shown the elegance of F#’s asynchronous workflows. The use of recursive functions and the compositional nature of asynchronous workflows really helps when you are doing this type of programming. It’s also very nice on the eye, each section being clearly defined.

The more astute of you may have noticed something a little different. Scan and TryScan are destructive in this implementation, the unmatched messages are purged from the internal queue. Although I could have mirrored the same functionality of the MailboxProcessor by using an internal list to keep track of unmatched messages, this leads to performing checks during Receive and Scan and their derivatives to make sure that this list is used first when switching from Scan and Receive functionality.

I think the separation of concerns are a little fuzzy in the MailboxProcessor. The scan function seems like an after thought, even if you don’t use Scan you still pay a price for it as there are numerous checks between the internal queue and the unmatched messages list. You can also run into issues while using Scan and TryScan that can result in out of memory conditions due to the inherent unbounded nature. I will briefly describe and explore the conditions that can lead to that in the next post. In the implementation presented here we can get bounded checking by passing in an optional DataflowBlockOptions and setting a value for the BoundedCapacity property.

EDIT: The code for this series of articles is now available on GitHub: FSharpDataflow

Until next time…

Construction

This is pretty straight forward and I don’t want to detract from the important bits of this post, the only thing of note is the cancellationToken which is initialized to a default value using the defaultArg function if the optional parameter cancellationToken is not supplied. The TDF construct that we to use to perform most of the hard work is incomingMessages which is a BufferBlock<'Msg>.

1
2
3
4
5
6
7

type DataflowAgent<'Msg>(initial, ?cancellationToken) =
    let cancellationToken =
        defaultArg cancellationToken Async.DefaultCancellationToken
    let mutable started = false
    let errorEvent = new Event<System.Exception>()
    let incomingMessages = new BufferBlock<'Msg>()
    let mutable defaultTimeout = Timeout.Infinite

Error

This is the public facing part for the Error event. The [<CLIEvent>] attribute exposes the event in a friendly manner to other .Net languages by adding the add_Error and remove_Error event handler properties to allow subscription to take place. The Error event fires when an exception is thrown in the initial asynchronous workflow.

1
2

[<CLIEvent>]
member this.Error = errorEvent.Publish

Start

This is implemented the same as the MailboxProcessor. An exception is thrown if the agent has already started as this is not valid operation. We set the mutable field started to true and proceed to start the initial asynchronous workflow. This workflow is wrapped in a try with block so that if an exception is thrown we catch it and trigger the Error event. The computation is then started with Async.Start(...).

1
2
3
4
5
6
7
8

member this.Start() =
    if started
        then raise (new InvalidOperationException("Already Started."))
    else
        started <- true
        let comp = async { try do! initial this
                           with error -> errorEvent.Trigger error }
        Async.Start(computation = comp, cancellationToken = cancellationToken)

Receive

The Receive member is used by the agent as a way of waiting for a message to arrive without blocking. Because the TDF functionality is all TPL Task based we use the the Async helper functions. In this instance we utilise the Async.AwaitTask passing in the incomingMessages ReceiveAsync method to wait for a message to arrive. The integration between F# async and TDF is nice and succinct here.

1
2

member this.Receive(?timeout) =
    Async.AwaitTask <| incomingMessages.ReceiveAsync()

Post

The Post member allows a message to be sent to the agents, this member simply calls the incomingMessages Post method passing in the item. We raise an exception if there is a problem posting (i.e. the incomingMessages internal queue is full).

1
2
3
4

member this.Post(item) =
    let posted = incomingMessages.Post(item)
    if not posted then
        raise (InvalidOperationException("Incoming message buffer full."))

PostAndTryAsyncReply / PostAndAsyncReply

I’m grouping both of these together as they are related in functionality. In the previous post I purposely left out some ancillary code as it added unnecessary complexity to the introduction. There are a two types we need to be able to replicate the PostAndTryAsyncReply and PostAndAsyncReply members of the MailboxProcessor.

AsyncReplyChannel

The first type we need is the AsyncReplyChannel<'Reply>. This type takes a function that accepts a generic 'Reply and returns a unit. It is used as a way of communicating back to the caller of the PostAndTryAsyncReply and PostAndAsyncReply members via its single member Reply. This should become a little clearer when we see it used in context.

An AsyncRepyChannel does actually exist in F# under the Microsoft.FSharp.Control namespace and is used my the MailboxPRocessor, unfortunately its constructor is marked as internal so we are not able to reuse it here.

1
2

type AsyncReplyChannel<'Reply>(replyf : 'Reply -> unit) =
    member x.Reply(reply) = replyf(reply)

AsyncResultCell

The next type we need is the AsyncResultCell<'a>. We use this as a way to await for the results of an asynchronous operation. We create a TaskCompletionSource (source), which is a TPL type that we use as a way of signalling to a callback / lambda expression when a message has arrived.

RegisterResult is used as a way of notifying when a message has been arrived, this is used internally by our agent as a result of a reply being made to the AsyncReplyChannel.

AsyncWaitResult is a continuation wrapper, it is called when we want to wait indefinitely for the result to be returned. It wraps a successful completion with a call to task.Result which then returns the result.

GetWaitHandle is used as a mechanism to force the asynchronous result to return within a specified timeout interval. If a result is not returned within the timeout then this function will return false.

GrabResult returns the result from the TaskCompletionSource object source. This is set earlier by the RegisterResult member.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

type AsyncResultCell<'a>() =
    let source = new TaskCompletionSource<'a>()
    member x.RegisterResult result = source.SetResult(result)

    member x.AsyncWaitResult =
        Async.FromContinuations(fun (cont,_,_) ->
            let apply = fun (task:Task<_>) -> cont (task.Result)
            source.Task.ContinueWith(apply) |> ignore)

    member x.GetWaitHandle(timeout:int) =
        async { let waithandle = source.Task.Wait(timeout)
                return waithandle }

    member x.GrabResult() = source.Task.Result

PostAndTryAsyncReply

This one is a little more tricky and I have added a few line number references to try and make it easier. On line 3 we declare an resultCell to collect the result of the asynchronous operation. This is used on line 4 when we create a msg to post to incomingMessages on line 5. The replyChannelMsg is a function that takes an AsyncReplyChannel and returns a message, so we create an AsyncReplyChannel with a lambda expression that registers the reply with the resultCell. This is the key to how this works, you have to remember that will be done the other side of the operation which will be within the asynchronous processing loop of the agent when Reply is called on the AsyncReplyChannel.

Finally pattern matching is used on line 7 to call either AsyncWaitResult or GetWaitHandle on the resultCell. The AsyncWaitResult function is used to wait indefinitely and the GetWaitHandle function is used if we want to use a timeout. Both of these are asynchronous workflows that either return a result or return an option type containing the result.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

member this.PostAndTryAsyncReply(replyChannelMsg, ?timeout) =
    let timeout = defaultArg timeout defaultTimeout
    let resultCell = AsyncResultCell<_>()
    let msg = replyChannelMsg(AsyncReplyChannel<_>(fun reply -> resultCell.RegisterResult(reply)))
    let posted = incomingMessages.Post(msg)
    if posted then
        match timeout with
        |   Threading.Timeout.Infinite ->
                async { let! result = resultCell.AsyncWaitResult
                        return Some(result) }
        |   _ ->
                async { let! ok =  resultCell.GetWaitHandle(timeout)
                        let res = (if ok then Some(resultCell.GrabResult()) else None)
                        return res }
    else async{return None}

PostAndAsyncReply

This member uses the same functionality as PostAndTryAsyncReply, creating a message using the AsyncReplyChannel. The main difference is that an asynchronous workflow is created that wraps a call to PostAndTryAsyncReply if the timeout is specified.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

member this.PostAndAsyncReply( replyChannelMsg, ?timeout) =
    let timeout = defaultArg timeout defaultTimeout
    match timeout with
    |   Threading.Timeout.Infinite ->
        let resCell = AsyncResultCell<_>()
        let msg = replyChannelMsg (AsyncReplyChannel<_>(fun reply -> resCell.RegisterResult(reply) ))
        let posted = incomingMessages.Post(msg)
        if posted then
            resCell.AsyncWaitResult
        else
            raise (InvalidOperationException("Incoming message buffer full."))
    |   _ ->
        let asyncReply = this.PostAndTryAsyncReply(replyChannelMsg, timeout=timeout)
        async { let! res = asyncReply
                match res with
                | None ->  return! raise (TimeoutException("PostAndAsyncReply TimedOut"))
                | Some res -> return res }

Static Start

The static Start function is used as a way to construct and start the agent than using the constructor and then calling the Start function. This is really just a simple short cut for this common use case.

1
2
3
4

static member Start(initial, ?cancellationToken) =
    let dfa = DataflowAgent<'Msg>(initial, ?cancellationToken = cancellationToken)
    dfa.Start()
    dfa

Until next time…

TPL Dataflows heritage and background

TPL Dataflow or (TDF) has been around for quite a while, it first surfaced more than a year ago as the successor to the Concurrency and Coordination Runtime (CCR) and with coming release of .Net 4.5 it will be part of the System.Threading.Tasks.Dataflow namespace. Elements of the now halted project Axum are also present within the design of TDF.

Concurrency and Coordination Runtime (CCR)

CCR is a library that deals with asynchrony, concurrency, and coordination between blocks of asynchronous code so that the programmer doesn’t have to. All of the low level details of synchronization and error propagation are taken care of in a consistent fashion. CCR is still is included in Microsoft Robotics Studio where it is used extensively to exploit parallel hardware and deal with partial failure of systems.

Axum

Axum was another interesting Microsoft research project, it also utilized the actor model embracing the principles of isolation, and message-passing. There was also extensive use symbolic operators as a terse short hand way to indicate operations between actors. For example <-- defined a way to pass a message to an actor. Theres was also a similarity to CCR as Axum used the concepts of Ports and channels in a similar way. It was a very interesting project and it was a shame it was put on hold.

TPL Dataflow (TDF)

TDF builds on CCR and Axum, consolidating and refine to produce a more friendly fluent interface, much in the same vain as Language-Integrated Query (LINQ) and Reactive Extensions (RX).

TDF is built around a number of different blocks which can be combined or linked together. There are three different categories of blocks are as follows:

Buffering Blocks

Buffering blocks simply buffer data in various ways before passing the data on to another block.

• BufferBlock<'T> - The BufferBlock act as a first-in-first-out (FIFO) queue, buffering each input.
• BroadcastBlock<'T> - The BroadcastBlock linking to multiple targets copying the data to each of the connected blocks.
• WriteOnceBlock<'T> - The WriteOnceBlock acts like an immutable target, after an item first item is passed to it, it effectively becomes read only.

Executor Blocks

The executor blocks run user supplied code in the form of a lambda expressions or a Task<'T>.

• ActionBlock<'TInput> - The ActionBlock acts like the Action<'T> delegate performing an action on each datum posted to it.
• TransformBlock<'TInput,'TOutput> - The TransformBlock acts just like the ActionBlock except that the action performed can have an output, this output is buffered and behaves just like a BufferBlock.
• TransformManyBlock<'TInput,'TOutput> - The TransformManyBlock is just like a TransformBlock except that is can produce more than one output for a given datum.

Joining Blocks

The Joining Blocks Combining or join data together in different ways.

• BatchBlock<'T> - The BatchBlock Combines multiple single items together, the items are represented by arrays of elements. The items are grouped together is batches and then passed on to another block.
• JoinBlock<'T1,'T2,…> - The JoinBlock acts as a form of Enumerable.Zip<'T1,'T2,'TResult> except the zip operation is performed on the items in the source array.
• BatchedJoinBlock<'T1,'T2,…> This block as the name suggests simply aggregates the JoinBlock and the BatchBlock together.

Thats an ultra high level tour thats only just scratches the surface. I recommend you check out the Introduction to TPL Dataflow document to read up on the details. Theres a few more resources in the DevLabs area that you might find useful. Hopefully this series should also shed a bit more light on TDF as we go along…

F# Asynchronous Workflows and Agents

So where does that leave us in F#?
In F# we have Asynchronous Workflows and agents and they help immensely in the concurrency and message passing, but that doest mean that we cant take advantage of the new features and refinements much in the same way as we can use Asynchronous Workflows to take advantage of Tasks.

This post is going to be centered around F# agents but with a twist. First of all are going to be reimplementing a MailboxProcessor using TDF for the underlying processing. This will allow us to to use all of our existing agent code and examples and also stay within the F# agent paradigm. Following this approach we could also make use of the DataflowBlockOptions type, it has some interesting properties which we will look at in future posts:

• TaskScheduler
• CancellationToken
• MaxMessagesPerTask
• BoundedCapacity

Implementation

In this post we are going replicate the MailboxProcessor, we will be using Tomas Petricek’s caching agent example from FSSnip). I have made a couple of modification to Tomas’s code.
I replaced the Dictionary type with a ConcurrentDictionary so that the caching agent could be called multiple times successively without the dictionary throwing an exception due to it already containing a key from a previous cached result. I also changed the example code so that it requests cached HTML from the caching agent ten times with a 400ms interval in between each.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

module TplAgents
open System
open System.Collections.Generic
open System.Collections.Concurrent
open FsDataflow
open System.Net
open Microsoft.FSharp.Control.WebExtensions

type CachingMessage =
| Add of string * string
| Get of string * AsyncReplyChannel<option<string>>
| Clear

let caching = DataflowAgent.Start(fun agent -> async {
   let table = ConcurrentDictionary<string, string>()
   while true do
      let! msg = agent.Receive()
      match msg with
      | Add(url, html) ->
         // Add downloaded page to the cache
         table.AddOrUpdate(url, html, fun k v -> html) |> ignore
      | Get(url, repl) ->
         // Get a page from the cache - returns 
         // None if the value isn't in the cache
         if table.ContainsKey(url) then
            repl.Reply(Some table.[url])
         else
            repl.Reply(None)
      | Clear ->
           table.Clear() })

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

/// Prints information about the specified web site using cache
let printInfo url = async {
   // Try to get the cached HTML from the caching agent
   let! htmlOpt = caching.PostAndAsyncReply(fun ch -> Get(url, ch))
   match htmlOpt with
   | None ->
       // New url - download it and add it to the cache
       use wc = new WebClient()
       let! text = wc.AsyncDownloadString(Uri(url))
       caching.Post(Add(url, text))
       Console.WriteLine( sprintf "Download: %s (%d)" url text.Length)
   | Some html ->
       // The url was downloaded earlier 
       Console.WriteLine( sprintf "Cached: %s (%d)" url html.Length) }

let printfuncpro = printInfo "http://functional-programming.net"
// Print information about a web site -
// Run this repeatedly to use cached value
for i in 1 .. 10 do
   printfuncpro |> Async.Start
   Async.RunSynchronously <| Async.Sleep 400

// Clear the cache - 'printInfo' will need to
// download data from the web site again
Console.WriteLine(sprintf "Clearing the cache")
caching.Post(Clear)
printfuncpro |> Async.Start

Console.ReadKey() |> ignore

Looking at the implementation above you can see that we need to implement the following members:

• Start:unit -> unit
• Receive:?int -> Async<'Msg>
• Post:'Msg -> unit
• PostAndTryAsyncReply:(AsyncReplyChannel<'Reply> -> 'Msg) * ?int -> Async<'Reply option>
• PostAndAsyncReply:(AsyncReplyChannel<'Reply> -> 'Msg) * int option -> Async<'Reply>
• static member Start:(MailboxProcessor<'Msg> -> Async<unit>) * ?CancellationToken -> MailboxProcessor<'Msg>

These are the only members we need to complete the caching agent example, I didn’t want bamboozle everyone with an explosion of code from the onset so the remaining members will be implemented as and when we need them. When we have implemented all the members from MailboxProcessor Ill post the full source on my GitHub account.

The following members will be outstanding but it should be fairly trivial to implement them once we have completed the code here.

• PostAndReply:(AsyncReplyChannel<'Reply> -> 'Msg) * int option -> 'Reply
• Scan:('Msg -> Async<'T> option) * ?int -> Async<'T>
• TryPostAndReply:(AsyncReplyChannel<'Reply> -> 'Msg) * ?int -> 'Reply option
• TryReceive:?int -> Async<'Msg option>
• TryScan:('Msg -> Async<'T> option) * ?int -> Async<'T option>
• CurrentQueueLength:int
• DefaultTimeout:int with get, set

So here we go, this is the Dataflow implementation of the MailboxProcessor:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

module FsDataflow
open System
open System.Threading
open System.Threading.Tasks
open System.Threading.Tasks.Dataflow
open System.Collections.Concurrent

type DataflowAgent<'Msg>(initial, ?cancellationToken) =
    let cancellationToken =
        defaultArg cancellationToken Async.DefaultCancellationToken
    let mutable started = false
    let errorEvent = new Event<System.Exception>()
    let incomingMessages = new BufferBlock<'Msg>()
    let mutable defaultTimeout = Timeout.Infinite

    [<CLIEvent>]
    member this.Error = errorEvent.Publish

    member this.Start() =
        if started
            then raise (new InvalidOperationException("Already Started."))
        else
            started <- true
            let comp = async { try do! initial this
                               with error -> errorEvent.Trigger error }
            Async.Start(computation = comp, cancellationToken = cancellationToken)

    member this.Receive(?timeout) =
        Async.AwaitTask <| incomingMessages.ReceiveAsync()

    member this.Post(item) =
        let posted = incomingMessages.Post(item)
        if not posted then
            raise (InvalidOperationException("Incoming message buffer full."))

    member this.PostAndTryAsyncReply(replyChannelMsg, ?timeout) =
        let timeout = defaultArg timeout defaultTimeout
        let resultCell = AsyncResultCell<_>()
        let msg = replyChannelMsg(AsyncReplyChannel<_>(fun reply -> resultCell.RegisterResult(reply)))
        let posted = incomingMessages.Post(msg)
        if posted then
            match timeout with
            |   Threading.Timeout.Infinite ->
                    async { let! result = resultCell.AsyncWaitResult
                            return Some(result) }
            |   _ ->
                    async { let! ok =  resultCell.GetWaitHandle(timeout)
                            let res = (if ok then Some(resultCell.GrabResult()) else None)
                            return res }
        else async{return None}

    member this.PostAndAsyncReply( replyChannelMsg, ?timeout) =
            let timeout = defaultArg timeout defaultTimeout
            match timeout with
            |   Threading.Timeout.Infinite ->
                let resCell = AsyncResultCell<_>()
                let msg = replyChannelMsg (AsyncReplyChannel<_>(fun reply -> resCell.RegisterResult(reply) ))
                let posted = incomingMessages.Post(msg)
                if posted then
                    resCell.AsyncWaitResult
                else
                    raise (InvalidOperationException("Incoming message buffer full."))
            |   _ ->
                let asyncReply = this.PostAndTryAsyncReply(replyChannelMsg, timeout=timeout)
                async { let! res = asyncReply
                        match res with
                        | None ->  return! raise (TimeoutException("PostAndAsyncReply TimedOut"))
                        | Some res -> return res }

    static member Start(initial, ?cancellationToken) =
        let dfa = DataflowAgent<'Msg>(initial, ?cancellationToken = cancellationToken)
        dfa.Start()
        dfa

The crux of the implementation from TDF’s point of view is the use of the BufferBlock.
This is one of the most fundamental blocks within TDF. Its the equivalent of the Port<'T> type from CCR and the Mailbox type from F# which is used internally within the MailboxProcessor. As mentioned abouve the BufferBlock type is a first-in-first-out (FIFO) buffer and is responsible for buffering any data that is Posted to it.

OK, I’m going to leave it at that for now while you digest the code presented here.

In part II I will be drilling into the detail on whats going on internally and also describing more of the TDF model, so tune in soon for Part II.

Until next time…

Well, there were a few factors involved…

I found the following problems and frustrations with Wordpress:

• Changing a Wordpress theme is somewhat of a black art.
• My blog didn’t look its best on mobile devices, there was no fluid layout.
• Wordpress seems to introduce a fair bit of latency for somethings thats essentially a simple task i.e. get some text from a database.
• Reliance on a database.
  All my post were in a MySQL database although easily maintained I had to have it hosted somewhere.
• Work press editing is both buggy and frustrating.
  I used to get near the end of a complex post only to find that save draft crashed the editor and I lost all my changes!
• Embedding code into a blog involved a Wordpress plugin, a Visual Studio plugin and manual manipulation of Html.

So what do I get from Octopress?

In a single word:

Simplicity

• No database or hosting is is needed, the blog is statically generated and all hosted on Github.
• I don’t need to use a clunky web based editor. I can use a simple text editor as all post are written in .markdown format.
• I get a nice simple elegant blog:
  • A Semantic Html 5 template.
  • A Mobile first responsive layout.
  • Built in 3rd party support for Twitter, Google Plus One, Disqus Comments, Pinboard, Delicious, and Google Analytics.
  • An easy deployment strategy using Github pages or Rsync.
  • Built in support for POW and Rack servers.
  • Easy theming with Compass and Sass.
  • A Beautiful Solarized syntax highlighting.

So here here I am with my first post, I’m really pleased so far. If you want to find out more details on how this all works checkout Octopress.

Until next time.

Using process explorer from Mark Russinovich I watched the allocated memory grow as the iterations progressed:

Theres definitely something leaking in there! So what can we do to find this? Simple, we use a memory profiler.  There are several really good memory profilers out there.  I have listed some of the best ones below:

• SciTech memory profiler
• RedGates ANTS Memory Profiler
• JetBrains dotTrace
• YourKit Profiler for .NET To demonstrate finding the leak I will be using RedGates ANTS MemoryProfiler.

First of all we launch the profiler and set it up to profile the application, this is just a simple case of browsing to the release folder and picking the application so I won’t bore with those trivial details here. Now that the application is running we hit any key which caused the test application to post 100 operations into the pipeline.  We want to create a baseline snapshot of the memory allocation so we can see where our leak is.  To do this click Take Memory Snapshot at the top right of the screen.  Next we hit any key again in the test application, again causing it to post another 100 operations into the pipeline.  Now we click Take Memory Snapshot again. Now we have a snapshot of the difference between the two operations.  The summery screen is shown below:

From this screen you can see that there is 51.56KB of new memory allocated since the last snapshot, and you can see some nice piecharts showing the various allocations in G1, G2 etc.  On the right hand side of the pie chart you can see that the largest classes are: object[], AsyncParamsAux, Pipelets+loop@37-7<Unit, string,string>, and AsyncParams.

Now if we click on Class List button we can investigate these further, heres the Class List:

Here things start to get interesting.  If you click on the instance Diff (+/-) column you can sort the list of classed by the differences to the last snapshot.

Now looking at the results we have:

• 300 more instances of AsyncBuilderImpl, AsyncParamsArgs, and AsyncParams
• 200 more instances of Pipelets+loop@37-7<Unit, string, string>
• 100 more instances of Pipelets+loop@37-7<Unit, string, FSharpRef>

Is it a coincidence that we just pushed 100 operations through the pipeline? I think not!

Now that we have a target for further inspection we can highlight the row for the function Pipelets+loop@37-7<Unit, string, FSharpRef>> and then click on the icon that has three little blue boxes on it.  This will take us to the instance List as shown below:

I have sorted the instance list by the distance from the GC Root, you can see there is a strange pattern emerging, the GC root distant increase by three each time.  Now lets look at the Instance Retention graph for the first one with a GC Root distance of 9, this is the icon on the right hand side of the function name, it looks like a few rectangles joined up with a line:

The Pipelets+loop function is linked from the mailbox processor shown at the top of the graph and flows into the Async infrastructure, and finally to the loop function at the bottom.

Lets look at the next one, this has a GC Root distance of 12:

If you look carefully there is another pattern here, the field references args, aux@, econt@ are repeated in the red boxes.  The functions look to be quite similar too.  Lets look at the next one GC Root Distance  of 15:

Looking at this we have a definite repeat of the functions and arguments,  if we look down to GC Root at a depth of 60 we get this:

So whats happening here is that there is a continuation that has been built around the asynchronous calls that gets bigger and bigger on each iteration.

Now that we have identified the leak, lets look at the code and see whats going on.  That would be the loop function in Pipelets:

Have a look at lines 9 and 12.  Can you guess whats wrong?

Well, to quote the F# Teams blog:

On the .NET platform, there are limitations on where tail calls may occur. One restriction is that tail calls cannot be performed in try-catch or try- finally blocks (neither in the body of the try nor in the catch or finally handlers).

It goes on further to discuss another subtle issue with use bindings:

use bindings implicitly generate a try-finally around the code that follows them to ensure that the Dispose method is called on the bound value.  This means that no calls following a use binding will be tail calls.

So all we have to do change the way the try catch block is formulated in that section.  The most idiomatic way of dealing with this is to use the Async.Catch function which would result in code something like the following:

Alternatively you could move the entire try with section out to a more local section thats not in the recursive async loop construct:

Anyway I hope that sheds a bit of light on how to spot where memory leaks are stemming from, and also some of the little known and often forgotten caveats with tail recursion.

Until next time…

EDIT: Just to make things a little bit clearer.  The memory leak here is caused by the async block being transformed into chains of continuation passing-style functions, and due to tail call elimination not being possible inside of the try catch blocks, the continuation grows and grows during each recursion.

Server / Cloud

Obviously first up is the F# 3.0 session by Don Syme, really looking forward to this session!

F# 3.0: data, services, Web, cloud, at your fingertips

Modern programming thrives on rich spaces of data, information and services. With F# 3.0 and Visual Studio 11, you now have a tool that massively simplifies information-rich analytical programming. F# 3.0 provides integrated support for F# Information Rich Programming, a new and powerful way of integrating data and services into your programming experience. In this talk, we will describe the new features of F# 3.0, including the first released version of F# Type Providers and F# Queries, with apps to leverage technologies such as SharePoint, Azure Data Market, OData, Entity Framework and SQL Server.

One of the areas I have a big interest in is performance so the next session will be interesting.

Windows Server performance improvements and optimisations

This session discusses the major performance improvements of Windows Server 8 and demonstrates how customers and partners can achieve their performance and scalability goals. We will describe some of the ways you can take advantage of the new capabilities to improve scalability, networking, throughput, and get a more powerful virtualization experience with Windows Server 8.

Network performance is another area I have a big interesting so this session will be good to see.

New techniques to develop low-latency network apps

For performance-critical apps, every microsecond saved means money. Windows Server 8 makes it possible to increase throughput, handle higher message rates, reduce latency and jitter, and lower CPU usage, all with standard server hardware. The new Registered I/O (RIO) model in Windows 8 delivers a low-latency solution while maintaining on-the-wire compatibility with existing TCP and UDP protocols. Additionally, a new fast path loopback allows high- speed apps to achieve a higher level of performance. Attend this session to learn how to program and tune these new capabilities and to witness them in action.

Parallel programming is a topic that deserves more attention so it will be good to see what the next in this area.

Building parallelized apps with .NET and Visual Studio

The Task Parallel Library (TPL), PLINQ, and Visual Studio 2010 provide managed code developers with a solid foundation for parallelizing loops, queries, and other common constructs in both server and client apps. But that’s only the beginning. In this code-intensive session, learn about what’s next for parallelization with .NET and Visual Studio, diving deep into performance enhancements, new Visual Studio tooling, and new libraries and language support for parallelization and asynchrony.

Next up here are the sessions in the Tools section that I will be taking a look at.

Tools

I will probably use this session to see if F# will map easily over to the new Windows Runtime

Using the Windows Runtime from C# and Visual Basic

C#, Visual Basic and the .NET tools have first-class support for the Windows Runtime. Learn about this integration and how to use C# and Visual Basic to write Metro style apps that call the Windows Runtime and how to build libraries that integrate with your Metro style apps using HTML.

DirectX isn’t really something I spend a lot of time doing but I like to keep my hand in.

A lap around DirectX game development tools

Visual Studio 11 brings the most significant set of improvements for developing graphics-intensive apps in over a decade. Whether you are just getting started with 2D/3D games or a self-proclaimed “guru,” there’s something for you in this talk. We will walkthrough a slew of new tools integrated into Visual Studio that will make your life better.

I dont tend to spend as much time in the debugger with F# but when I do this information will be good to know.

Advanced IntelliTrace in production with Visual Studio 11

Did you ever have that bug that you just could not reproduce? Not getting enough information from production environments to really solve your problems quickly? Why does it always take way longer to figure out what the problem is then to make the fix? In this session, see how IntelliTrace, within Visual Studio 11, gives you the ability retrieve rich, detailed information about problems, even in production, enabling you to spend less time on bug diagnostics and more time writing software.

Again this section will be good to see just to find out if this can be ported over to F#

Taming GPU compute with C++ AMP

Developers today inject parallelism into their compute-intensive apps in order to take advantage of multi-core CPU hardware. Beyond CPUs, however, compute accelerators such as general-purpose GPUs can provide orders of magnitude speed-ups for data parallel algorithms. How can you as a C++ developer fully utilize this heterogeneous hardware from your Visual Studio environment? How can you benefit from this tremendous performance boost in your Visual C++ solutions without sacrificing developer productivity? The answers will be presented in this session introducing C++ AMP.

Delving into the depths of stuff is always interesting so this will be great.

Deep dive into the kernel of the .NET Framework

The Common Language Runtime is the cutting-edge virtual machine at the heart of the .NET Framework. In this session, we’ll dive deep under the covers of the CLR and discuss some of our key innovations for .NET Framework 4.5 and Windows 8. Topics will include updates in the code-generation and diagnostics space, improvements in our garbage collector for low latency server scenarios, and automatic NGen.

It will be good to keep up with the latest developments in the C# world. The new async/await features are heavily influenced by F#’s Asynchronous workflows.

Future directions for C# and Visual Basic

In this talk, Technical Fellow Anders Hejlsberg will share project plans for the future directions of C# and Visual Basic, including a discussion of what trends are influencing and shaping the direction of programming languages. Anders will talk about asynchronous programming and Windows 8 programming, coming in the next version of Visual Studio. He will also discuss the long-lead project “Roslyn”, including object models for code generation, analysis, and refactoring, and upcoming support for scripting and interactive use of C# and Visual Basic

New features of Visual Studio are always good to know.

What’s new in Visual Studio 11

Microsoft Visual Studio 11 enable developers to take full advantage of the capability of Windows using the skills and technologies developers already know and love to deliver exceptional and compelling apps. Whether working individually or in a small, medium or large development team the Visual Studio 11 sets a new standard for development tools, helping teams deliver superior results for their customers that help set them apart from their competitors. In this session we’ll walk through the new features in the Visual Studio 11 Developer Preview to give you an understanding of the breadth of tooling available in this release.

Looking forward to hearing about the new .NET 4.5 features.

What’s new in .NET Framework 4.5

The next major release of the .NET Framework, .NET 4.5, allows you to easily use Windows 8 technologies, like Windows Runtime, directly from .NET 4.5. Accessing your data is easier than ever with support for the newest features in SQL Server and support for WebSockets. Programs are more responsive, with the AWAIT keyword, faster ASP.NET startup and an improved server Garbage Collector. .NET 4.5 incorporates key customer feedback, with the newest MEF features, support for long running workflows with State Machines, and improved HTML 5 support in ASP.NET. In this overview talk, you’ll learn about all of these technologies, and get pointers to deeper dives where you can learn more.

Finally heres the session in the platform section that I find interesting.

Platform

Socket based programming is an interesting area, at least it is for me, so this should be an interesting session.

Building Windows runtime sockets apps

Today’s networks have grown more complicated as a result of multihoming, web proxies, security issues, internationalization and other issues. Because dealing with these complexities is hard, it is either ignored or significant resources are spent attempting to do so. In this session, we will demonstrate how the Windows Runtime sockets API simplifies what an app must do to use TCP or UDP. We also will present exciting new functionality, such as proximity discovery and using WebSockets for HTTP proxy traversal, as well as how to handle complex security and cost issues.

Looking throughout the more client based areas in the platform section the next two might be worth a look.

Achieving high performance 2D graphics with Direct2D

Make it fast! Great performance is a huge motivator of satisfaction and user preference with apps. Direct 2D powers high-performance 2D graphics rendering in Windows 8. In this session, you will learn advanced techniques for optimizing your Direct2D code for maximum speed and efficiency in your apps.

Create cool image effects with Direct2D

Every day, people are using apps that process images to create a variety of effects. The new image processing infrastructure in Windows 8 enables apps to perform high-performance image enhancements, transformation and composition using the GPU for photos, vector graphics and UI elements. In this session, you will learn about the Direct2D effect API, and see how easy it is to apply effects to photos and 2D and 3D content in your Windows 8 app to deliver more polished visual experiences to your customers.

I had a look through the Apps and Hardware sections but there were not any specific areas in there that I found really interesting.

Wow this post is a bit longer than I wanted it to be! There looks to be some very interesting sessions here, lots of Metro based information but Ill be keeping a bit of a distance from that stuff to concentrate on the server-side material for now.

Until next time…

There has been an increasing amount of exposure for F# and functional programming lately.  If you come from an object-orientated background a change in mindset is required when working with functional programming, there is a lot of misinformation on functional languages and their relationship with object-orientated design. In this post we run quickly through SOLID to see if these object-orientated principles apply to F#, and if so, how.

This post assumes you are familiar with SOLID principles, if not here is a link. 

Lets take a quick overview of what SOLID stands for:

Single responsibility principle

This is the notion that an object should have only a single responsibility.

An object-orientated program consists of layers of abstract classes with less abstract classes layered on top of ones that are more abstract.  Functional programming is similar, although abstractions are used throughout the design and are composed into a final solution.  Programming in F# naturally forms small succinct functions which should have a single purpose, so the single responsibility rule holds strong here.

Open closed principle

The notion that “software entities should be open for extension, but closed for modification”

This comes down to building abstractions via inheritance, and behavior changes through polymorphism.  Early on a decision must be made on which parts will change, and which will be fixed.   The open closed principal is geared towards languages where inheritance is a core concept, inheritance and polymorphism are not strongly used in F#, so this principle is very weak here.  Composition and Type augmentation are the core methods for extension in F#.

Liskov substitution principle

Liskov substitution states “objects in a program should be replaceable with instances of their subtypes without altering the correctness of that program”.

This principle is all about inheritance - derived types preserving the specification of base types.  Functional languages like F# do not always use inheritance, it is used quite rarely and only in certain situations.   The idioms in the language like referential transparency lean strongly towards enforcing this principle.  Functional languages also have a heritage in mathematics and algebraic reasoning so referential transparency is key in this respect.

Interface segregation principle

The notion that “many client specific interfaces are better than one general purpose interface.”

If you are violating single responsibility then your interface will probably be bloated too with unnecessary properties and methods.  The same rule applies to F#, keep your interfaces modular and keep indifferent concepts separated.

Dependency inversion principle

The notion that one should “Depend upon Abstractions. Do not depend upon concretions.”

1. High-level modules should not depend on low-level modules. Both should depend on abstractions.
2. Abstractions should not depend upon details. Details should depend upon abstractions. Dependency inversion is often solved with concepts such as dependency injection; common techniques for this involves things like constructor injection and interface injection.  Inversion of control is often solved with a factory pattern or a service locator.  From a functional point of view, these containers and injection concepts can be solved with a simple higher order function, or hole-in-the-middle type pattern which are built right into the language.

Summary

So we can assume that in the functional paradigm:

• Types should have only a single responsibility.
• The open closed principle doesn’t apply in the majority of cases unless we are using an object-oriented approach.  With functional programming we favour functional composition and type augmentation.
• The same can be said of the Liskov substitute principle, pure functions uphold this tenant via immutability and referential transparency.
• Interface segregation also applies and idiomatic F#  produces small concise functions with modular interfaces with a separation of concerns.
• Dependency inversion is not a relevant issue for F# as the language itself supports higher-order-functions to encapsulate this concept. So only the ’S’ and the ’I’ parts are relevant in functional programming.  The other tenants are not fully representative as they stand. We could probably explore the tenants of good functional design and form a quirky acronym in the process, but Ill leave that for another time.

Until next time…

The article has proved to be more more popular than I envisaged.  I think a lot of .Net developers are interested in F# but are unsure on the path to take when trying to accomplish this.  For me it was almost a leap of faith, I saw the potential benefits and just jumped right in.

I had to overcome numerous obstacles along the way before I become comfortable within the language.  I had question like:

• How do design patterns and principles fit in.
• How do I structure my applications.
• How can I work seamlessly with other libraries in the .Net ecosphere.

I will try and answer some of these question over the coming weeks as well as introducing some new topics.  If anyone has any comments on the article or suggestions on future content please leave them below, and I will try to work them into future posts.

You can find the article here.

Until next time…

Agents can be used for a multitude of different purposes ranging from: isolated message passing, object caching, finite state machines, web crawling, and even reactive user interfaces.  One of the ideas that I have been looking into lately is agent based scheduling.

SchedulerAgent

Listing 1 shows a simple Agent based scheduler:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

    module AgentUtilities
    open System
    open System.Threading

    //Agent alias for MailboxProcessor
    type Agent<'T> = MailboxProcessor<'T>

    /// Two types of Schedule messages that can be sent
    type ScheduleMessage<'a> =
      | Schedule of ('a -> unit) * 'a * TimeSpan * TimeSpan * CancellationTokenSource AsyncReplyChannel
      | ScheduleOnce of ('a -> unit) * 'a * TimeSpan * CancellationTokenSource AsyncReplyChannel

    /// An Agent based scheduler
    type SchedulerAgent<'a>()=
      let scheduleOnce delay msg receiver (cts: CancellationTokenSource)=
        async { do! Async.Sleep(delay)
            if (cts.IsCancellationRequested)
            then cts.Dispose()
            else msg |> receiver }
      let scheduleMany initialDelay  msg receiver delayBetween cts=
        let rec loop time (cts: CancellationTokenSource) =
           async { do! Async.Sleep(time)
               if (cts.IsCancellationRequested)
               then cts.Dispose()
               else msg |> receiver
               return! loop delayBetween cts}
        loop initialDelay cts
      let scheduler = Agent.Start(fun inbox ->
        let rec loop() = async {
          let! msg = inbox.Receive()
          let cs = new CancellationTokenSource()
          match msg with
          | Schedule(receiver, msg:'a, initialDelay, delayBetween, replyChan) ->
            Async.StartImmediate(scheduleMany
                         (int initialDelay.TotalMilliseconds)
                         msg
                         receiver
                         (int delayBetween.TotalMilliseconds)
                         cs )
            replyChan.Reply(cs)
            return! loop()
          | ScheduleOnce(receiver, msg:'a, delay, replyChan) ->
            Async.StartImmediate(scheduleOnce
                         (int delay.TotalMilliseconds)
                         msg
                         receiver
                         cs)
            replyChan.Reply(cs)
            return! loop()
        }
        loop())
      ///Schedules a message to be sent to the receiver after the initialDelay.
      ///  If delaybetween is specified then the message is sent reoccuringly at the delaybetween interval.
      member this.Schedule(receiver, msg, initialDelay, ?delayBetween) =
        let buildMessage replyChan =
          match delayBetween with
          | Some(x) -> Schedule(receiver,msg,initialDelay, x, replyChan)
          | _ -> ScheduleOnce(receiver,msg,initialDelay, replyChan)
        scheduler.PostAndReply (fun replyChan -> replyChan |> buildMessage)

The structure of the SchedulerAgent broken down into sections below:

ScheduleMessage

Lines 9-11 show the definition of ScheduleMessage. This is a discriminated union of two different types of Schedule message.

ScheduleOnce

ScheduleOnce has four parameters:

1. A function which is called at the schedule time (‘a -> unit).
2. The message that is sent at the schedules time (‘a).
3. A TimeSpan which is the length of time to wait before triggering the schedule.
4. An AsyncReplyChannel(CancellationTokenSource AsyncReplyChannel).  This is used to return a CancellationTokenSource which can be used to cancel the Schedule.

Schedule

Schedule has five parameters which are as follows:

1. A function which is called at the schedule time (‘a -> unit).
2. The message that is sent at the schedules time (‘a).
3. A TimeSpan which is the initial length of time to wait before first triggering the schedule function.
4. A TimeSpan which is used as an interval between each subsequent triggering of the schedule function.
5. An AsyncReplyChannel(CancellationTokenSource AsyncReplyChannel).  This is used to return a CancellationTokenSource which can be used to cancel the Schedule.

SchedulerAgent

scheduleOnce

Lines 16-20 define an async workflow, which asynchronously sleeps for the specified time before checking that the schedule hasn’t been cancelled before finally calling the schedule function.

scheduleMany

Lines 22-29 define a recursive async workflow, which asynchronously sleeps for the specified interval (3rd Parameter) before checking the schedule hasn’t been cancelled before finally calling the schedule function. The loop function is then called passing in the second TimeSpan interval (4th Parameter).

scheduler

This is the main processing loop for the agent.  A recursive loop function is declared on line 32.  On line 33 the agent waits for a message to arrive.  Once a message arrives a CancellationTokenSource is created on line 36 which can be used to cancel an already scheduled message. Pattern matching is used on line 35 to find the type of message that has been received.  The first pattern matching block on lines 36-43 matches the Schedule message.  The parameters from the Schedule message are passed into the scheduleMany function.  This is then invoked asynchronously via the Async.StartImmediate function.  The CancellationTokenSource is now returned to the caller on line 43. This allows the caller to cancel an already running schedule.   Finally the recursive loop function is called on line 44.  The second pattern matching block on lines 45-52 is much the same passing the parameters from the ScheduleOnce message into the scheduleOnce function, again this is invoked via the Async.StartImmediate function.  Like the Schedule message the CancellationTokenSource returned on line 51 and the recursive loop function is called on line 52.

The agent is then started on line 51 by calling the loop function for the first time.

Members

The SchedulerAgent has only a single member Schedule. This member function takes three parameters and an optional parameter delayBetween.  A function called buildMessage on line 59 uses the optional parameter with pattern matching to determine whether a ScheduleOnce or a Schedule message is created.  The agent is posted the correct message type on line 63 using the synchronous call scheduler.PostAndReply.  We use a synchronous call to return the cancellationTokenSource immediately, and this can be used to cancel a running schedule.

Sample Application

Listing 2 shows a test harness that creates and uses a simple string based message scheduler.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

    open AgentUtilities
    open System
    open System.Threading
    let scheduler = SchedulerAgent<_>()
    let printer message =
      printfn "%s: %s" (DateTime.Now.TimeOfDay.ToString()) message
    let singlecancel = scheduler.Schedule(printer,
                        "Hello from the scheduler",
                        TimeSpan(0,0,0,5))
    let multicancel = scheduler.Schedule( printer,
                        "Hello from the multi scheduler",
                        TimeSpan(0,0,0,5),
                        TimeSpan(0,0,0,0,500))
    printfn "Press any key to cancel."
    Console.ReadKey() |> ignore
    //Cancel the multi scheduler
    multicancel.Cancel()
    printfn "Cancelled, press any key to exit."
    Console.ReadKey() |> ignore

I hope this gives you a feel for what you can do with agent based scheduling. The library here could be expanded further in several ways.  You could replace the fixed message with a message generator function or even an agent based message generator.  If the schedule function was abstracted somewhat it could be made to accept an agent as the receiver.

One of the key areas I am looking at is building a distributed agent library that would allow an agent to communicate over network layers transparently.  A scheduler agent would be even more powerful in this environment.  I could envisage them used for a many different things in this environment:  heart beat messages, performance sampling, diagnostics and testing.

Until next time…

There was a series of posts last April by Stephen Toub from the pfxteam at Microsoft.  I was reading through some of the posts again the other day and thought some of the ideas presented there would make interesting projects in F# to demonstrate the flexibility and succinctness of the language.  I thought the ObjectPool example would make an interesting project in F# using agents aka MailboxProcessors. An ObjectPool is basically a pool of objects that have been pre-created so that you can grab one and use it, and then place it back in the pool when you’re finished.  They are useful in situations where the cost of creating object from scratch is very high or you want to cut down on allocations in the garbage collector.

First of all heres the C# code as it was presented in the Parallel Extensions download:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

using System.Collections.Generic;
using System.Diagnostics;
namespace System.Collections.Concurrent
{
  /// <summary>Provides a thread-safe object pool.</summary>
  /// <typeparam name="T">Specifies the type of the elements stored in the pool.</typeparam>
  [DebuggerDisplay("Count={Count}")]
  [DebuggerTypeProxy(typeof(IProducerConsumerCollection_DebugView<>))]
  public sealed class ObjectPool<T> : ProducerConsumerCollectionBase<T>
  {
    private readonly Func<T> _generator;
    /// <summary>Initializes an instance of the ObjectPool class.</summary>
    /// <param name="generator">The function used to create items when no items exist in the pool.</param>
    public ObjectPool(Func<T> generator) : this(generator, new ConcurrentQueue<T>()) { }
    /// <summary>Initializes an instance of the ObjectPool class.</summary>
    /// <param name="generator">The function used to create items when no items exist in the pool.</param>
    /// <param name="collection">The collection used to store the elements of the pool.</param>
    public ObjectPool(Func<T> generator, IProducerConsumerCollection<T> collection)
      : base(collection)
    {
      if (generator == null) throw new ArgumentNullException("generator");
      _generator = generator;
    }
    /// <summary>Adds the provided item into the pool.</summary>
    /// <param name="item">The item to be added.</param>
    public void PutObject(T item) { base.TryAdd(item); }
    /// <summary>Gets an item from the pool.</summary>
    /// <returns>The removed or created item.</returns>
    /// <remarks>If the pool is empty, a new item will be created and returned.</remarks>
    public T GetObject()
    {
      T value;
      return base.TryTake(out value) ? value : _generator();
    }
    /// <summary>Clears the object pool, returning all of the data that was in the pool.</summary>
    /// <returns>An array containing all of the elements in the pool.</returns>
    public T[] ToArrayAndClear()
    {
      var items = new List<T>();
      T value;
      while (base.TryTake(out value)) items.Add(value);
      return items.ToArray();
    }
    protected override bool TryAdd(T item)
    {
      PutObject(item);
      return true;
    }
    protected override bool TryTake(out T item)
    {
      item = GetObject();
      return true;
    }
  }
}

There’s also a base class which looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

/// <summary>
/// Provides a base implementation for producer-consumer collections that wrap other
/// producer-consumer collections.
/// </summary>
/// <typeparam name="T">Specifies the type of elements in the collection.</typeparam>
[Serializable]
public abstract class ProducerConsumerCollectionBase<T> : IProducerConsumerCollection<T>
{
private readonly IProducerConsumerCollection<T> _contained;
/// <summary>Initializes the ProducerConsumerCollectionBase instance.</summary>
/// <param name="contained">The collection to be wrapped by this instance.</param>
protected ProducerConsumerCollectionBase(IProducerConsumerCollection<T> contained)
{
  if (contained == null) throw new ArgumentNullException("contained");
  _contained = contained;
}
/// <summary>Gets the contained collection.</summary>
protected IProducerConsumerCollection<T> ContainedCollection { get { return _contained; } }
/// <summary>Attempts to add the specified value to the end of the deque.</summary>
/// <param name="item">The item to add.</param>
/// <returns>true if the item could be added; otherwise, false.</returns>
protected virtual bool TryAdd(T item) { return _contained.TryAdd(item); }
/// <summary>Attempts to remove and return an item from the collection.</summary>
/// <param name="item">
/// When this method returns, if the operation was successful, item contains the item removed. If
/// no item was available to be removed, the value is unspecified.
/// </param>
/// <returns>
/// true if an element was removed and returned from the collection; otherwise, false.
/// </returns>
protected virtual bool TryTake(out T item) { return _contained.TryTake(out item); }
/// <summary>Attempts to add the specified value to the end of the deque.</summary>
/// <param name="item">The item to add.</param>
/// <returns>true if the item could be added; otherwise, false.</returns>
bool IProducerConsumerCollection<T>.TryAdd(T item) { return TryAdd(item); }
/// <summary>Attempts to remove and return an item from the collection.</summary>
/// <param name="item">
/// When this method returns, if the operation was successful, item contains the item removed. If
/// no item was available to be removed, the value is unspecified.
/// </param>
/// <returns>
/// true if an element was removed and returned from the collection; otherwise, false.
/// </returns>
bool IProducerConsumerCollection<T>.TryTake(out T item) { return TryTake(out item); }
/// <summary>Gets the number of elements contained in the collection.</summary>
public int Count { get { return _contained.Count; } }
/// <summary>Creates an array containing the contents of the collection.</summary>
/// <returns>The array.</returns>
public T[] ToArray() { return _contained.ToArray(); }
/// <summary>Copies the contents of the collection to an array.</summary>
/// <param name="array">The array to which the data should be copied.</param>
/// <param name="index">The starting index at which data should be copied.</param>
public void CopyTo(T[] array, int index) { _contained.CopyTo(array, index); }
/// <summary>Copies the contents of the collection to an array.</summary>
/// <param name="array">The array to which the data should be copied.</param>
/// <param name="index">The starting index at which data should be copied.</param>
void ICollection.CopyTo(Array array, int index) { _contained.CopyTo(array, index); }
/// <summary>Gets an enumerator for the collection.</summary>
/// <returns>An enumerator.</returns>
public IEnumerator<T> GetEnumerator() { return _contained.GetEnumerator(); }
/// <summary>Gets an enumerator for the collection.</summary>
/// <returns>An enumerator.</returns>
IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); }
/// <summary>Gets whether the collection is synchronized.</summary>
bool ICollection.IsSynchronized { get { return _contained.IsSynchronized; } }
/// <summary>Gets the synchronization root object for the collection.</summary>
object ICollection.SyncRoot { get { return _contained.SyncRoot; } }

Wow!  Thats a fair bit of code in C#, fair enough there is a lot of noise in the xml doc comments, but theres also a lot of boiler plate code in there too.

Ok now we have gotten that out of the way heres the good bit.  

Below is an agent based design which implements the same functionality but uses a lot less code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

module Poc

//Agent alias for MailboxProcessor
type Agent<'T> = MailboxProcessor<'T>

///One of three messages for our Object Pool agent
type PoolMessage<'a> =
  | Get of AsyncReplyChannel<'a>
  | Put of 'a * AsyncReplyChannel<unit>
  | Clear of AsyncReplyChannel<List<'a>>

/// Object pool representing a reusable pool of objects
type ObjectPool<'a>(generate: unit -> 'a, initialPoolCount) =
  let initial = List.init initialPoolCount (fun (x) -> generate())
  let agent = Agent.Start(fun inbox ->
    let rec loop(x) = async {
      let! msg = inbox.Receive()
      match msg with
      | Get(reply) ->
        let res = match x with
              | a :: b ->
                reply.Reply(a);b
              | [] as empty->
                reply.Reply(generate());empty
        return! loop(res)
      | Put(value, reply)->
        reply.Reply()
        return! loop(value :: x)
      | Clear(reply) ->
        reply.Reply(x)
        return! loop(List.empty<'a> )
    }
    loop(initial))
  /// Clears the object pool, returning all of the data that was in the pool.
  member this.ToListAndClear() =
    agent.PostAndAsyncReply(Clear)
  /// Puts an item into the pool
  member this.Put(item) =
    agent.PostAndAsyncReply((fun ch -> Put(item, ch)))
  /// Gets an item from the pool or if there are none present use the generator
  member this.Get(item) =
    agent.PostAndAsyncReply(Get)

We have a discriminated union (PoolMessage) which describes the messages that we are going to use with this agent, they are pretty straight forward to follow.  

• Get simply returns either a stored item or generates a brand new one using the generator function which is passed into the ObjectPools constructor (generate: unit -> ‘a).  
• Put simply adds the item onto the internal list.
• Clear simply returns the current pool and then clears it.

The core processing all happens in the async{} block, we simply wait for a message to arrive, then we pattern match on one of the messages either Get,Put, or Clear.

Get takes an item from the internal list if there are items present, otherwise it invokes the generator function and returns a newly generated object.

For a Put operation we use the cons (::) operator to add the item onto the internal list via the recursive loop.

For the Clear operation we return the entire list then return an empty list to the recursive loop.

I think you will agree this is a nice succinct example of the flexibility and elegance of agents and yet another reason to use F# for more server side activities.  It’s not simply a language for the mathematical and finance orientated developers.

For anyone interested all of the code should be in my GitHub repository to download.

Thanks to Tomas Petricek for suggesting using the recursive loop to pass the list rather than using a ref cell and the (:=) operator.

Until next time…

The Sockets and Bockets articles have become Flack and the pipeline articles have become Pipelets.  Flack is named so because I always like to use quirky names for libraries and its original name was a bit controversial, hence Flack i.e. I get a lot of flack about my naming :-)

I have been working on consolidating and expanding the demo code to produce some useful libraries rather than just proof of concepts and demo’s.  I was contacted pretty early on by Ryan Riley who has been working on Frack, for those of you who dont know Frack is an F# implementation based on Rack:

Rack provides a minimal, modular and adaptable interface for developing

web applications in Ruby.  By wrapping HTTP requests and responses in
the simplest way possible, it unifies and distills the API for web
servers, web frameworks, and software in between (the so-called
middleware) into a single method call.

Ryan wanted to see if the SocketAsyncEventArgs pattern could be used in Frack to improve efficiency.  So far I have been pushed for time to make a large amount of progress on integrating the two, but things are progressing faster now that the library is on GitHub.

Please check out:

• Flack
• Pipelets
• Frack

Also check out Ryan’s technical blog: Wizards of Smart

The other thing in the pipeline, pun intended, is a distributed pipeline engine where a pipeline would be exposed via a socket and could be connected to other pipeline systems.  The beauty of a system like this is you could compose systems of pipelines to run either on either a single machine or distributed over a network, this would effectively marry together functional composition and distributed systems.  Its early days yet but I hope to get together some more details of how this will work in the coming weeks, Im quite excited to get working on it!

As always I appreciate any comments or suggestions.

See you soon.

Since the last post I have been reflecting on the pipeline design and it had a distinct object orientated feel to it that I wasnt happy with, so I have amended the structure of the code and come up with the following which simplifies in some areas and expands in others…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

module Pipeline
  open System.Collections.Concurrent

  [<Interface>]
  type IPipelineInput<'a> =
    abstract Insert: 'a -> unit

  [<Interface>]
  type IPipelineConnection<'a> =
    abstract Attach: IPipelineInput<'a> -> unit
    abstract Detach: IPipelineInput<'a> -> unit

  [<Interface>]
  type IPipeline<'a,'b> =
    inherit IPipelineConnection<'b>
    inherit IPipelineInput<'a>

  type PipelineStage<'a,'b>(processor, router: seq<IPipelineInput<'b>> * 'b -> seq<IPipelineInput<'b>>, ?overflow, ?capacity, ?blockingTime) =
    let processor = processor
    let router = router
    let createBlockingCollection x =
        match x with
        | Some c -> new BlockingCollection<'a>(c:int)
        | None -> new BlockingCollection<'a>()
    let buffer = createBlockingCollection capacity
    let routes = ref List.empty<IPipelineInput<'b>>
    let queuedOrRunning = ref false
    let blocktime =
      match blockingTime with
      | Some b -> b
      | None -> 250
    let consumerLoop = async {
      try
        let rec loop()=
          let item = ref Unchecked.defaultof<_>
          let taken = buffer.TryTake(item, blocktime)
          if taken then
              do !item
              |> processor
              |> Seq.iter (fun z ->
              (match !routes with
               | [] -> ()(*we cant route with no routes*)
               | _ -> do router (!routes, z) |> Seq.iter (fun r -> (r.Insert z ))) )
              loop()
          else ()(*exit nothing to consume in time limit*)
        loop()
      with e -> raise e
      }
    member this.ClearRoutes = routes := []
    interface IPipelineInput<'a> with
      member this.Insert payload =
        let added = buffer.TryAdd(payload, blocktime)
        if added then
          //begin consumer loop
          if not !queuedOrRunning then
            lock consumerLoop (fun() ->
            Async.Start(async {do! consumerLoop })
            queuedOrRunning := true)
          else()
        else
          //overflow here if function passed
          match overflow with
          | Some t ->  payload |> overflow.Value
          | None -> ()
    interface IPipelineConnection<'b> with
      member this.Attach (stage) =
        let current = !routes
        routes := stage :: current
      member this.Detach (stage) =
        let current = !routes
        routes := List.filter (fun el -> el <> stage) current
    static member Attach (a:IPipelineConnection<_>) (b) =
      a.Attach b ;b
    static member Detach (a: IPipelineConnection<_>) (b) =
      a.Detach b ;a
    static member (++>) (a:IPipelineConnection<_>, b) =
      a.Attach (b) ;b
    static member (-->) (a:IPipelineConnection<_>, b) =
      a.Detach b ;a
    static member (<<--) (a:IPipelineInput<_>, b:'b) =
      a.Insert b
    static member (-->>) (b,a:IPipelineInput<_>) =
      a.Insert b

Summary.

I only want to summarise the code as I think its fairly straight forward to see whats going on.

Interfaces

We have two main interfaces defined IPipelineInput<’a> and IPipelineConnection<’a>, as you can tell by the names they are involved with connecting the pipeline together and getting information into the pipeline.  Those two interfaces are merged together in the IPipeline<’a, ‘b> interface, this keeps a nice separation between connecting and inserting into the pipeline, it also makes implementation easier and allows the interfaces to be implemented in other areas of code that need to talk to or connect to a pipeline.

Internals

Inside the pipeline we have the bounded blocking queue which is implemented by the BlockingCollection from TPL. This is used to store the pipeline payloads that are waiting to be processed.

The consumerLoop function is recursive and continually tries to take items from the blocking collection processing and routing each one to the next pipeline stage.

The processor is a function that transforms from type ‘a to type ‘b.

The router is a function that takes a sequence of IPipelineInput<’b> and also the payload ‘b it returns a sequence of IPipelineInput<’b>.  What this effectively means is that we can route by the connected stages (i.e. round robin routing, multi-cast routing.)   Or we could route by payload contents (i.e. if the payload contains a certain bytes sequence we could choose a certain IPipelineInput<’b>.)

Each item taken is passed to the processor and router via pipeline (|>) and Seq operations, recursively calling itself until an item can no longer be retrieved from the buffer.

The implementation of IPipelineInput<’a>.Insert is the counterpart to the previous function. It first tries to inset the item into the bounded blocking queue, if this cannot be done then the overflow function is called if one is present. Next the async consumer loop is started if it is not already running. The idea behind this is that by keeping the payload processing running on the thread pool while there is work to do it will cut down on the number of context switches between threads.  Once an item cannot be taken from the bounding blocking queue the loop will exit.

The rest of the code is pretty standard stuff and should be pretty easy to follow.

I also define some symbolic operations to simply constructing and using the pipeline:

++> Attaches the pipeline stage on the right hand side to the one on the left. –> Detaches the pipelinestage on the right from the one on the left. <<– Inserts a payload on the right into the pipeline stage on the left. –>> Inserts a payload on the left hand side into the pipeline stage on the right.
These help to keep a nice terse description of the pipeline, once things get a little more complex other operators may be required, the now discontinued Axiom had a whole host of these, its a pity Microsoft dropped the language.

Example

Heres a quick sample pipeline showing the pipeline in use:

• Stage 1 takes a string and splits it based on the ‘,’.
• Stage 2 reverses each string.
• Stage 3 reverses the string back to the original.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

module program
  open System
  open Pipeline
  let consoleLock = new obj()
  let split del n (s:string) =
    lock consoleLock (fun() ->
    do printfn "%A:before split %A" n s
    let split = s.Split([|del|])
    do printfn "%A:after: split into: %A" n split
    split |> Array.toSeq)
  let reverse (s:string) =
    new string(s |> Seq.toArray |> Array.rev)
  let oneToSingleton a b f=
    lock consoleLock (fun() ->
      printfn "%A:before reverse %A" a b
      let result = b |> f
      printfn "%A:after reverse %A" a result
      result|> Seq.singleton)
  let OneToSeqRev a b = oneToSingleton a b reverse
  ///Simply picks the first route
  let basicRouter( r, i) =
    let head = Seq.head r
    Seq.singleton head
  let p1 = PipelineStage( split ',' "1", basicRouter)
  let p2 = PipelineStage( OneToSeqRev "2", basicRouter)
  let p3 = PipelineStage( OneToSeqRev "3", basicRouter)
  p1 ++> p2 ++> p3 |> ignore
  let generateCircularSeq (lst:'a list) =
    let rec next () =
      seq {
        for element in lst do
          yield element
        yield! next()
      }
    next()
  for str in ["John,Paul,George,Ringo"]
  |> generateCircularSeq
  |> Seq.take 10
    do  str -->> p1
  let x = Console.ReadKey()

As you can see the assignment of the pipeline stages is pretty simple as is the composition of multiple stages.  This was often one of the most difficult areas while developing a similar pipelines in C# you could often find yourself with a few hundred lines of setup code which was a often a nightmare to debug a few weeks later.

Hopefully I have whet your appetite with pipelines, in a future article I will be combining socket operations with pipeline stages to produce a flexible framework to deal with high throughput network applications.

As always I appreciate any comments, until next time…

I feel I need to backtrack slightly from the previous post, having worked with pipelines for quite some time I have the advantage of knowing all of the details that may be alluded to in these articles without being effected by any omissions I may make, obviously you guys aren’t in that position, so I’m going to try and rectify that a bit now.  If you have any queries then please leave a comment and I will try to address them in further articles. Pipelines are a simple concept but in practice there can be some caveats and things to bear in mind, sometime the whole mindset of development team can be against them unless they can see the bigger picture…

First of all one of the most important things to bear in mind with a pipeline is that you are only going to be as fast as your slowest stage, if one stage is ten times slower than another then it will be waiting for input most of the time, we need to make this more efficient.

Premature Optimisation

Lots of developers out there have the premature optimisation is the root of all evil mindset and will quote this out loud to you when you mention performance early on.  I’m not advocating premature optimisation, in this instance performance is key, if one stage is out of kilter with the rest then we are going to be running at that pace of the slowest stage, if that’s too slow for the requirements then you are screwed.

The more I think about performance the more I believe its an essential part of creating code. There are too many developers these days that will produce sloppy unrefined plain bad code.  I’m a keen believer in producing quality code that you can be proud of, and part of that is having clean code that’s both efficient and works.  I think some of this boils down to a feature driven approach that measures developers solely in terms of features added, take the typical burn down chart that you would use in agile software development:

There is nowhere on this chart that measures whether the code is good or bad or runs to performance requirements.  In the future I may do an article on integrating code quality into your build process, its something I have been thinking about doing for a while now.

While I’m talking about performance you also might want to check out Joe Duffy’s post on The ‘premature optimization is evil’ myth, and also check out Joe’s book on concurrent programming, put it on your wish list if you haven’t already read it, its a great book.

Unbalanced pipelines

Data is received from the network via packets, each packet may contain one or more messages from a business systems or indeed a partial message.  We need to collect the packets either separate or combine them to form individual messages, deserialize them and finally log them.

Here’s a sample pipeline demonstrating an unbalanced pipeline:

1. Stage 1 of the pipeline receives these packets and processes them into individual messages passing them onto Stage 2.
2. We now have a complete message (in this instance the message will be XML) we want to turn it into a .Net type we now deserialize the message and pass it onto Stage 3.
3. To keep this pipeline simple all we are going to do here is log type of message to disk or a database, the pipeline is now complete.

Stage 1 would take 5 seconds to fully utilise stage 2, stage 2 would take 2 seconds to fully utilise stage 3.  You can see this pipeline will only process 100 transactions per second even though stages 2 has 5x the throughput of stage 1 and stage 3 has 2x the throughput of stage 2.  Our efficiency is only about 10% of what it could be, we must be able to do something about that.

Lets look at the following diagram which demonstrate a balanced pipeline:

Balanced pipelines

You can see from this diagram that each stage processes the same number of transactions per second by introducing parallel stages.  This is called a balanced pipeline.  Sometimes you cant get a perfectly balanced pipeline but you should strive to get as close as possible.  Sometimes a certain stage cannot be parallelised because it may have mutable state, or you are using some sort of IOC container for processing services, this might make constructing the various stages in parallel difficult, this can become an art form in itself and can lead to very large initialisation sections in the code.  I hope to address all of these issues in due course.

This poses some interesting thoughts and questions to add to some you may already have:

• How can we easily manage the complexity of parallelism?
• How will the distribution of work be handled?
• How do you baseline the throughput of each stage?
• Can you automate the parallelism of a particular stage?
• How do you manage the complexity of multiple stages?
• What about parallelism and mutable state?

The final point to note is the Distributor/Router must operate at a much higher rate than the processing stages otherwise you will introduce another bottle neck into the system, although you could have a multiple distributors but this would yet another degree of complexity that has to be managed.  You can see that things can quickly become more complicated than they first seemed.

I know I promised lots of funky code but I figured there was a bit more explaining to do before we can get to that.  I want to take a more of an iterative approach to show you the potential pitfalls that can occur during developing such a pipeline and how to avoid them.  I thought this would be a lot more constructive than dropping a load of code and some pretty pictures and hoping for the best.

Next time we will be exploring a simple pipeline stage with a single degree of parallelism and a simple router.  After that we will then start exploring and answering the questions above, adding more features like parallelism, instrumentation, and visualisation.

Hope you enjoyed this even though there was no code!

See you next time.

First up, what’s a pipeline?  Well according to Wikipedia):

A pipeline is a set of data processing elements connected in series, so that
the output of one element is the input of the next one. The elements of a
pipeline are often executed in parallel or in time-sliced fashion; in that
case, some amount of buffer storage is often inserted between elements.  

In essence its a way of dealing with complexity and its also a way of breaking down a process into separate tasks of a similar size.  If they are used correctly then pipelines can be used to increase the overall throughput of a system.

In enterprise systems or in fact in most large systems, a simple idea or program can rapidly become overwhelmingly complex.  The management all of the disparate parts of the system can become a nightmare and the code can quickly becomes a labyrinth, navigating it becomes a skill of only the most accomplished code ninja, and even then your playing Russian roulette with any bug fixes.In an effort to keep things manageable and simple one approach that we can use is a pipeline. The idea is that each stage is connected to one or more other stages and each that each stage deals with a single task before passing the work onto the next stage.  There are many primitive types in the Task Parallel Library (TPL) that you could use to compose a working pipeline, we will be using a lightweight subset taking only a few core ideas and making sure we get a nice slick design that is both powerful and flexible.

Here’s a quick flow diagram of the sort of thing that we will be looking at:

This is a generic asynchronous payload based pipeline.  Each stage is asynchronous and self contained and is connected to one or more other stages. As a payload enters the pipeline it is initially added to a bounding blocking queue.  If the queue is full then the payload is said to have overflowed and is passed to the failure processor where the payload can be processed or transformed in some way before being passed to a failure router which would in turn pass the payload to one or more of the next failure stages.  The same is also true for a successfully queued payload except that the payload is first dequeued, processed, then passed to a router which then passes the payload to one or more stages.  If an exception occurs during processing then the payload is passed to the failure processor and processed like an overflow.  I am purposely missing out any details of asynchronous operation as they will be described in more detail next time.

We will be using a little bit of Language Oriented Programming to construct the pipeline stages, maybe using a little bit of operator overloading too.   I will describe all of this in more detail next time as we dig into the code.   I want this to be just a brief introduction to what we are going to be doing.

Here’s a more detailed description of the components that are involved in each stage:

Bounded Blocking Queue

This is a standard bounded blocking queue from the TPL, its purpose here is to limit the amount of payloads that are waiting to be processed, each queue will have an associated time-out period, if the time-out period passes the payload is passed to the failure processor for processing and then finally to the the failure router to be passed to one or more failure stages.

Processors

Each pipeline processor has a primary Processor<T,U> and a failure processor<T,V>.

The primary processors job is to convert type T to type U, both types can be the same if you wish, you may well be thinking why would I want a processing stage that essentially leaves the type unchanged?  In this case the processor acts as a simple a pass through but using this you to do some custom routing. This can be very be useful in some scenarios and I will describing this in more detail in a further post.

Each pipeline stage also has a failure processor<T,V>.  The failure processor acts on the payload to produce the desired type and passes it onto the failure router.  The reasoning behind this scheme rather than a simplistic exception logger is simply flexibility.  Having spent a lot of time with this kind of API in a more locked down format I have found that you can end up wanting a bit more flexibility especially when some developers try to get a bit creative with the API or start state to the payload.  A good example of having some flexibility is during overflow:  If the bounded blocking queue fills up and blocks for the time-out period then the payload could be passed to a failure failure processor in which types T and V are the same.  This would allow us to pass the payload to another stage and retry later on by attaching some sort of delayed forwarding pipeline stage.

Routers

The router is responsible for getting the payload to the next pipeline stage, it can be implemented as a simple predicate function operating on the type directly or even some outside influence if you wish.   An example of this might be a simple duplicating stage where the payload is passed to multiple output stages rather than just one, or a time based router where one stage is passed the payload during the day and another at night.   When you start to think about the possibilities the Processor / Router combination can be really really flexible.

Each pipeline stage also has a corresponding has a failure router, this can be used for all sorts of purposes like routing the failed payload to a logging component, routing to a delayed retry mechanism, or saved to a database etc.

Thats all for now, we will be digging into some code and more detail next time, and I will be describing a few different types of pipelines so you can get a feel of how to use them and the overall structure.

Another interesting aspect of these pipelines is that once constructed they can be composed into single reusable blocks that as a whole, represent a single pipeline stage.  These composite stages can then be connected together to form a super pipeline stage, complexity is only visible when you start to drill down and becomes almost fractal like…

As always please leave any comments or suggestions.

If you were looking forward to some exciting new F# code this time your going to be disappointed, however if you are like me and like looking at graphs and stats and digging in deeper into the code then your going to enjoy this, lets get started…

I set up a 5 minute test with 50 clients connecting to the server with a 15ms interval between each one.  Once connected each client receives a 128 byte message from the server every 100ms so this will be a 500 messages per second test.  I am going to be using an excellent product called YourKit Profilerfor .NET it can do both memory and CPU profiling as well as displaying telemetry for things like thread count, stack contents, memory allocations etc.  It can be configured to be a lot less intrusive than a lot of other profilers and I have had a lot of success using it.  You can download a demo from their site using the link above.  I will be doing some other articles on using profiling and analysis tools later on so stay tuned for those too.  All of the graphs and information gathered in this post come from YourKits output during CPU and memory profiling.

Before we start here’s a reminder of what the client code looks like, this is a simple test client using Brian’s code as mentioned in Part1 I have highlighted the lines that have changed below.