Current markerless motion capture systems are capable of acquiring faithful biomechanical abstractions of the skeletal system, also known as stick figure. Despite their applicability, conventional markerless motion capture systems perform improper limb orientation tracking. Marker­-based motion capture systems, such as Vicon and Optitrack, perform an improved limb orientation tracking with counterpart of being less cost effective and more intrusive when compared to markerless approaches. To advance Human-Computer Interaction in physical activity settings, it is necessary to have a richer body representation that adds segment orientation information to skeletal tracking. This project proposal envisions to accurately track body segment orientations, for a given arbitrary posture, using machine learning techniques. By the end of the project, we expect to develop a markerless, cost effective, and reliable motion capture platform composed by an array of Kinect sensors that runs our algorithm in order to build plausible skeletons with anatomically correct body segments. The potential economic value of the technology resides mostly in the video game and fitness industries as well as rehabilitation and motion capture laboratory niches.