Leap Motion for 2D Pointing Tasks

leapmotion; hci 07 May 2015

This post is a summary of a study we did to evaluate the Leap Motion device for two-dimensional pointing and selecting tasks. The study was implemented by Manuel Seixas during his MSc degree.

Original papers

Seixas, M. C. B., Cardoso, J. C. S., & Dias, M. T. G. (2015). The Leap Motion Movement for 2D Pointing Tasks - Characterisation and Comparison to Other Devices. In Proceedings of the 5th International Conference on Pervasive and Embedded Computing and Communication Systems (pp. 15–24). SCITEPRESS Science and Technology Publications, Lda. doi:10.5220/0005206100150024 https://www.mendeley.com/research/leap-motion-movement-2d-pointing-tasks-characterisation-comparison-other-devices-1/

Seixas, M. C. B., Cardoso, J. C. S., & Dias, M. T. G. (2015). One Hand or Two Hands? 2D Selection Tasks With the Leap Motion Device. In ACHI 2015 : The Eighth International Conference on Advances in Computer-Human Interactions (pp. 33–38). Lisbon, Portugal, Portugal: IARIA. Retrieved from http://www.thinkmind.org/index.php?view=article&articleid=achi_2015_2_20_20020

Introduction

The Leap Motion controller (Figure 1) is a new 3D sensing device for hand gesture interaction with a computer. It is capable of sensing the position and orientation of the fingers of the hands, as well as the palm orientation and curvature. The LM is a small device that sits on top of the computer desk and is operated by positioning the hands over the device. The controller can be used to point to a computer screen with a finger or with a tool (a pen or pencil, for example), or perform other hand gestures.

Figure 1. The Leap Motion controller device.

Although not meant to replace the mouse, many of the interactions with the LM involve pointing and selecting targets on a computer screen. Many applications in the Leap Motion App Store are OS control apps meant to give users various degrees of control over the computer, from selecting and launching predefined applications and settings to scrolling content on webpages. Some applications even emulate the mouse, allowing cursor control and mouse actions. Most applications that take advantage of the LM device still require users to perform typical WIMP tasks at some point (in many cases giving users the option of using the mouse or the LM device). For example, in many games users still need to select options and activate buttons; some software for surgery rooms also provides cursor control for specific functions.

If we assume that the LM device gains commercial traction and becomes embedded in additional laptop computers and desktop keyboards (it is already embedded in one HP laptop and keyboard), we must also assume that it will become an additional alternative to typical WIMP tasks. In a situation where the user is operating the LM device in a specific task, it may be faster to perform a WIMP task also with the LM, instead of moving the hand to operate the mouse.

However, up until now, there have been no studies about the performance of the LM device for 2D pointing tasks.

Experiments

We performed two experiments with the LM device in a multi-directional selection test. In this multi-directional selection test, participants are required to move the cursor across a circle to sequentially numbered targets (see Figure 2). The targets are equally spaced around the circumference of the circle and the sequence of targets to select is such that the movements are nearly equal to the diameter of the circle.

Figure 2. The Multi-directional tapping test.

In the first experiment, we compared the LM device with a mouse and a touchpad. To select the targets with the LM, we used the Touchless software for Mac, which emulates the mouse.

In the second experiment, we compared two different selection gestures with the LM: the screen tap (the same used in the first experiment), and the hand grab gesture (Figure 3). The hand grab uses two hands for interaction. The dominant hand performs the cursor movement, and the auxiliary hand performs the “click”. In this experiment, we also compared both gestures to the mouse, to have a baseline.

Figure 3. Gestures used in the second experiment.

Results

In both experiments, we calculated various standard metrics (speed, error rate, throughput) as well as other metrics proposed by MacKenzie et al.. In this post, we only show the standard metrics.

The first experiment made it obvious that the LM device is not a match for the mouse or touchpad. Figures 4 and 5 show the movement time, throughput, and error rate, for the LM, mouse, and touchpad. It is obvious from these figures that participants are a lot slower when using the LM (it takes more than twice the time of the mouse). Consequently, the throughput for the LM is a lot lower. We also observed more errors (clicks outside the target) with the LM than with any of the other devices.

Figure 4. Movement time - first experiment.

Figure 5. Throughput and Error rate - first experiment.

The second experiment confirmed the results that compare the LM to the mouse, but it also showed that performing the selection gesture with another hand improves the selection time. We can see in Figures 6 and 7 that the time to select is a bit lower when using the hand grab gesture. However, probably due to our implementation of the gesture, it generated more errors than the simpler screen tap gesture (Figure 7).

Figure 6. Movement time - second experiment.

Figure 7. Throughput and Error rate - second experiment.

Posture

During the experiments, we also analysed the participants’ postures’ and asked them how comfortable the devices were. One of the main problems with the LM was that participants would get tired very easily. When interacting with a desktop, the LM is has the same problem as vertical touch-screens: the gorilla arm effect. Users have to keep their arm extended to operate the device which, after some time, causes the arm to feel very heavy and leads to less accuracy.

Figure 8. Gorilla arm effect.

Conclusion

• For desktop 2d point and select tasks, the LM performs poorly.
• The LM causes physical fatigue very quickly in this setting.
• A gesture with two hands performs a bit better than using only one hand to point and select in terms of throughput.
• We evaluated the LM in a very particular setting. This does not mean the device is not good for many other tasks!