At Developer Week 2013 I gave a talk (in German) on Reactive Extensions. The slides are online on Slideshare. During the talk, I live-coded a part of a console-based jump and run game that uses Kinect as input. The first task was to detect when the player (the volunteer on stage) jumped. Here’s the code from the demo:
This code contains a lot of simplifications for presentation purposes but it can still be used to explain some Reactive Extension concepts. I’ll walk you through it and also spell out the vars:
This retrieves the single Kinect that is attached to the PC (or throws an exception if there is not exactly one Kinect connected).
This creates an IObservable from the classic .NET event SkeletonFrameReady by telling Rx how to subscribe and unsubscribe from the event.
Interestingly, the class SkeletonFrameReadyEventArgs doesn’t contain any properties at all, just one method public SkeletonFrame OpenSkeletonFrame(); that can be used to access the skeleton data. The SkeletonFrame instance must then be disposed within 1/30 second.
We now have an IObservable that publishes a list of skeletons whenever there are people in front of the Kinect sensor.
This code extracts the joints from the single skeleton if it has a tracked state.
This extracts the average vertical position of the left and right foot. This is a simplification as it would be possible to trick the algorithm by just lifting one foot up twice as high as both feet would need to jump up. To fix this, you could Select() both feet in this statement.
This is the actual jump detection code. The idea is that to have jumped, the player would have to have his feet low, then high and then low again in a short timespan. To analyze this, we’re looking at samples from a time frame of one second. After this, we’ll move forward by 200 milliseconds and analyze again. This magic is provided by the Buffer() extension method. We’ll look for the maximum height of the feet within the time frame and see if the first and last samples are both lower than the maximum minus a hard-coded magic number (for simplification again). If the algorithm matches, the IObservable outputs the string “jumped”, else the string “didn’t jump”.
If you add jumped.Subscribe(Console.WriteLine); at this point you will see a stream of “didn’t jump” interrupted by a few instances of “jumped” whenever the player jumped.
The call to DistinctUntilChanged() will only output the string if it differs from the previously output string. The next line simply filters to only output “jumped” whenever the player jumps.
This line will output the resulting IObservable to the console.
This starts the Kinect sensor.
The presentation was a lot of fun. Thanks again to the volunteer! I’ll be back at Developer Week again this year talking about cross-platform mobile using C#.