Neurophysiological Constraints on Human-Robot Interfacing for Augmentation

Research on neurotechnology has been recently expanded to the realm of robotic augmentation. In particular, augmentation by wearing supernumerary (i.e. additional to the natural number of) robotic limbs has been the focus of neuroscientific research regarding human capability for controlling these devices. Specifically, supernumerary robotics fingers working in collaboration with natural fingers are a relevant testbed to study neurocognitive phenomena. Within this context, the choice of an interface modality to control augmentative devices has effects in their embodiment and in the motor learning process behind controlling them successfully [1-3].

One of the most common interfacing approaches to control supernumerary robotic limbs is substitution control, in which the extra limb is controlled by the kinematic output originally intended for other body parts. For instance, in our previous study, participants were trained to use an additional robotic finger attached to the side of the right hand and controlled by foot movements [4]. Moreover, they learned to use it in a piano playing task in under 30 minutes, which showcases human capability to quickly integrate augmentation devices in real-world tasks [4]. Crucially, it was also demonstrated that participants with better foot coordination skills also performed better in piano playing with the foot-controlled SRF [4]. This supports the idea that motor coordination metrics, in particular the ones related to the control interface, are predictive of success in robotic augmentation [4]. This can have implications for other types of interfaces with higher cognitive demands such as the ones based on neurophysiological signals decoding, in which subjects could benefit from tailored training in the modulation of these signals.

Substitution interfaces are an effective approach for short-latency real-time control. However, because essentially the degrees of freedom of one body part are transferred to the SRL, this strategy could be deemed as not being “true augmentation”, which instead implies simultaneous volitional control of natural and artificial limbs without hindering the control of the formers [5]. However, in able-bodied people, neural motor commands on the efferent pathways are already tasked with controlling all muscles. Hence, there is a shortage of signals available to voluntarily control augmentative devices without compromising the control of the biological body [6].

There have been some meaningful steps towards understanding the mechanisms for successful augmentation beyond natural motor constraints. Recent advances in non-invasive recording of individual motoneurons activity have facilitated the study of the human potential for controlling a subset of these signals independently from the motor unit pool that determines muscle force [7,8]. If attained, flexible control of spinal motor neurons will lead to the increase in the number of independently controlled neurophysiological signals available (neural output) required for true human augmentation.

References

[1] Kieliba, P., Clode, D., Maimon-Mor, R. O., & Makin, T. R. (2021). Robotic hand augmentation drives changes in neural body representation. Science Robotics, 6(54), 1–14.

[2] Zhu, Y., Ito, T., Aoyama, T., & Hasegawa, Y. (2019). Development of sense of self-location based on somatosensory feedback from finger tips for extra robotic thumb control. ROBOMECH Journal, 6(1), 0–9.

[3] Amoruso, E., Dowdall, L., Kollamkulam, M. T., Ukaegbu, O., Kieliba, P., Ng, T., Dempsey-Jones, H., Clode, D., & Makin, T. R. (2022). Intrinsic somatosensory feedback supports motor control and learning to operate artificial body parts. Journal of Neural Engineering, 19(1), 016006.

[4] Shafti, A., Haar, S., Mio, R., Guilleminot, P., & Faisal, A. A. (2021). Playing the piano with a robotic third thumb: assessing constraints of human augmentation. Scientific Reports, 11(1), 1–14.

[5] Eden, J., Bräcklein, M., Ibáñez, J., Barsakcioglu, D. Y., di Pino, G., Farina, D., Burdet, E., & Mehring, C. (2022). Principles of human movement augmentation and the challenges in making it a reality. Nature Communications, 13(1), 1345.

[6] Dominijanni, G., Shokur, S., Salvietti, G., Buehler, S., Palmerini, E., Rossi, S., Vignemont, F. de, Avella, A., & Makin, T. R. (2021). The neural resource allocation problem when enhancing human bodies with extra robotic limbs. Nature Machine Intelligence, 3(October), 850–860.

[7] Farina, D., & Negro, F. (2015). Common synaptic input to motor neurons, motor unit synchronization, and force control. Exercise and Sport Sciences Reviews, 43(1), 23–33.

[8] Formento, E., Botros, P., & Carmena, J. M. (2021). Skilled independent control of individual motor units via a non-invasive neuromuscular machine interface. Journal of Neural Engineering, 18(6).

Recommended citation: R. Mio and A. A. Faisal. “Neurophysiological Constraints on Human-Robot Interfacing for Augmentation”, Neuroadaptive Technology 2022.