All posts by kerry

DWX13 Follow-up: Detecting jumps with Kinect and Reactive Extensions

At Developer Week 2013 I gave a talk (in German) on Reactive Extensions. The slides are online on Slideshare. During the talk, I live-coded a part of a console-based jump and run game that uses Kinect as input. The first task was to detect when the player (the volunteer on stage) jumped. Here’s the code from the demo:

This code contains a lot of simplifications for presentation purposes but it can still be used to explain some Reactive Extension concepts. I’ll walk you through it and also spell out the vars:

This retrieves the single Kinect that is attached to the PC (or throws an exception if there is not exactly one Kinect connected).

This creates an IObservable from the classic .NET event SkeletonFrameReady by telling Rx how to subscribe and unsubscribe from the event.

Interestingly, the class SkeletonFrameReadyEventArgs doesn’t contain any properties at all, just one method public SkeletonFrame OpenSkeletonFrame(); that can be used to access the skeleton data. The SkeletonFrame instance must then be disposed within 1/30 second.

We now have an IObservable that publishes a list of skeletons whenever there are people in front of the Kinect sensor.

This code extracts the joints from the single skeleton if it has a tracked state.

This extracts the average vertical position of the left and right foot. This is a simplification as it would be possible to trick the algorithm by just lifting one foot up twice as high as both feet would need to jump up. To fix this, you could Select() both feet in this statement.

This is the actual jump detection code. The idea is that to have jumped, the player would have to have his feet low, then high and then low again in a short timespan. To analyze this, we’re looking at samples from a time frame of one second. After this, we’ll move forward by 200 milliseconds and analyze again. This magic is provided by the Buffer() extension method. We’ll look for the maximum height of the feet within the time frame and see if the first and last samples are both lower than the maximum minus a hard-coded magic number (for simplification again). If the algorithm matches, the IObservable outputs the string “jumped”, else the string “didn’t jump”.

If you add jumped.Subscribe(Console.WriteLine); at this point you will see a stream of “didn’t jump” interrupted by a few instances of “jumped” whenever the player jumped.

The call to DistinctUntilChanged() will only output the string if it differs from the previously output string. The next line simply filters to only output “jumped” whenever the player jumps.

This line will output the resulting IObservable to the console.

This starts the Kinect sensor.

The presentation was a lot of fun. Thanks again to the volunteer! I’ll be back at Developer Week again this year talking about cross-platform mobile using C#.

Animation fun with UIView.AnimateAsync()

Xamarin.iOS 6.4 brought async/await support to iOS devices. Apart from supporting the idiom with the Mono 3.0 compiler, Xamarin also invested into adding awaitable versions of all long-running calls in Apple’s iOS API. One of my favorites is

UIView.AnimateAsync()

taking an Action or lambda to animate asynchronously without blocking the UI thread.

The fun begins when you use this to chain animations. Here’s an example of doing a horizontal shake animation that could be used to indicate an invalid entry: