Microsoft’s Kinect is a powerful sensor for motion tracking and analysis. There are different applications that take advantage of its functionality of 3D motion capture. In the medical field, for example, the sensor offers great possibilities to the treatment and prevention of disease, illness or injury, as we discussed in this post.
The Kinect can be used in a fall detection system to detect when an individual is walking and suddenly falls. Its implementation is quite easy using the framework for skeleton tracking. However, we designed a system to detect fall events using a smartphone and we want to use the Kinect for verification after a fall. This verification consists of detecting if the individual is lying down in the floor. In this post we will discuss three different approaches for the verification of the fall event and its associated problems.
Skeleton tracking with the Microsoft SDK
The fall verification could consist of detecting some joints (head and hands, for example) using the skeleton tracking framework (included in the Microsoft SDK) and calculating the distance from the floor.The fall will be considered detected if the distance from the floor is almost zero from all joints.
We performed several experiments implemented with the official Microsoft SDK. The main challenge is to detect the joints when the Kinect turns on and the individual is already lying on the floor. The algorithm gives good results after small movements of the individual, but sometimes the person remains unconscious after a fall which makes this approach not useful.
Skeleton tracking with OpenNI
OpenNI is the open source SDK for the Kinect. As we discussed in this post, it has some advantages and disadvantages but is always an alternative to develop Kinecting applications. Since the first approach presented some problems to detect the individuals joints when the Kinect turns on and the individual is already lying on the floor, we decided to try with this SDK. Using this SDK we obtain better results in terms of accuracy of detection but not enough for a reliable verification of the fall.
User selection using depth data with OpenNI
We also performed some experiments using OpenNI and open source libraries. The fall verification consist of detecting the individual using the depth data to segment the user from the background. Once the individual is selected, we check if the individual’s bounding box is less tall than a threshold value and the position of the highest point is lower than a threshold, which means than the user is lying on the floor. This approach has the same limitation: pickup the person if is already lying on the floor when the Kinect turns on. This is the same problem we had with the previous approaches.
After all the experiments using both SDKs for the Kinect and different methodologies, we realized the Kinect has an important limitation to track joints when the individual is motionless. All SDKs present good accuracies after small movements but this is not useful in our system, where we want to detect if an individual is lying on the floor. Any ideas or suggestions about how to implement it?