van den Bunt MR, Groen MA, Ito T, Francisco AA, Gracco VL, Pugh KR, Verhoeven L (2017)
Increased Response to Altered Auditory Feedback in Dyslexia: A Weaker Sensorimotor Magnet Implied in the Phonological Deficit,
J Speech Lang Hear Res, 1-14, doi: 10.1044/2016_JSLHR-L-16-0201.
Purpose: The purpose of this study was to examine whether developmental dyslexia (DD) is
characterized by deficiencies in speech sensory and motor feedforward and feedback mechanisms,
which are involved in the modulation of phonological representations.
Method: A total of 42 adult native speakers of Dutch (22 adults with DD; 20 participants who
were typically reading controls) were asked to produce /bep/ while the first formant (F1) of the /e/
was not altered (baseline), increased (ramp), held at maximal perturbation (hold), and not altered
again (after-effect). The F1 of the produced utterance was measured for each trial and used for statistical
analyses. The measured F1s produced during each phase were entered in a linear mixed-effects model.
Results: Participants with DD adapted more strongly during the ramp phase and returned to
baseline to a lesser extent when feedback was back to normal (after-effect phase) when compared with
the typically reading group. In this study, a faster deviation from baseline during the ramp phase,
a stronger adaptation response during the hold phase, and a slower return to baseline during the
after-effect phase were associated with poorer reading and phonological abilities.
Conclusion: The data of the current study are consistent with the notion that the phonological
deficit in DD is associated with a weaker sensorimotor magnet for phonological representations.
Ito T, Coppola JH, Ostry DJ (2016)
Speech motor learning changes the neural response to both auditory and somatosensory signals,
Sci. Rep., 6:25926, doi: 10.1038/srep25926.
In the present paper, we present evidence for the idea that speech motor learning is accompanied
by changes to the neural coding of both auditory and somatosensory stimuli. Participants in our
experiments undergo adaptation to altered auditory feedback, an experimental model of speech motor
learning which like visuo-motor adaptation in limb movement, requires that participants change their
speech movements and associated somatosensory inputs to correct for systematic real-time changes to
auditory feedback. We measure the sensory e ects of adaptation by examining changes to auditory and
somatosensory event-related responses. We nd that adaptation results in progressive changes to speech
acoustical outputs that serve to correct for the perturbation. We also observe changes in both auditory
and somatosensory event-related responses that are correlated with the magnitude of adaptation.
These results indicate that sensory change occurs in conjunction with the processes involved in speech motor adaptation.
Cortical processing associated with orofacial somatosensory function in speech has received limited
experimental attention due to the difficulty of providing precise and controlled stimulation.
This article introduces a technique for recording somatosensory event-related potentials (ERP) that
uses a novel mechanical stimulation method involving skin deformation using a robotic device.
Controlled deformation of the facial skin is used to modulate kinesthetic inputs through excitation
of cutaneous mechanoreceptors. By combining somatosensory stimulation with electroencephalographic recording,
somatosensory evoked responses can be successfully measured at the level of the cortex. Somatosensory
stimulation can be combined with the stimulation of other sensory modalities to assess multisensory interactions.
For speech orofacial stimulation is combined with speech sound stimulation to assess the contribution of
multi-sensory processing including the effects of timing differences to examine the manner in which
the two sensory signals combine. The ability to precisely control orofacial somatosensory stimulation
during speech perception and speech production with ERP recording is an important tool providing new insights
into the neural organization and neural representations for speech.
Suemitsu A, Jianwu D, Ito T, Tiede M (2015)
A real-time articulatory visual feedback approach with target presentation for second language pronunciation learning,
J. Acoust. Soc. Am., 138, EL382. doi: 10.1121/1.4931827.
Articulatory information can support learning or remediating pronunciation of a second language (L2).
This paper describes an electromagnetic articulometer-based visual-feedback approach using an articulatory
target presented in real-time to facilitate L2 pronunciation learning. This approach trains learners
to adjust articulatory positions to match targets for a novel L2 vowel estimated from productions of vowels
that overlap in both L1 and L2. Training of Japanese learners for the American English vowel that included
visual training improved its pronunciation regardless of whether audio training was also included.
Articulatory visual feedback is shown to be an effective method for facilitating L2 pronunciation learning.
Ito T, Gracco VL, Ostry DJ (2014)
Temporal factors affecting somatosensory-auditory interactions in speech processing,
Front. Psychol. 5:1198. doi: 10.3389/fpsyg.2014.01198.
Speech perception is known to rely on both auditory and visual information. However, sound-specific
somatosensory input has been shown also to influence speech perceptual processing (Ito et al., 2009).
In the present study, we addressed further the relationship between somatosensory information and
speech perceptual processing by addressing the hypothesis that the temporal relationship between
orofacial movement and sound processing contributes to somatosensory–auditory interaction in speech
perception. We examined the changes in event-related potentials (ERPs) in response to multisensory
synchronous (simultaneous) and asynchronous (90 ms lag and lead) somatosensory and auditory stimulation
compared to individual unisensory auditory and somatosensory stimulation alone. We used a robotic device
to apply facial skin somatosensory deformations that were similar in timing and duration to those
experienced in speech production. Following synchronous multisensory stimulation the amplitude of
the ERP was reliably different from the two unisensory potentials. More importantly, the magnitude of
the ERP difference varied as a function of the relative timing of the somatosensory– auditory stimulation.
Event-related activity change due to stimulus timing was seen between 160 and 220 ms following somatosensory
onset, mostly around the parietal area. The results demonstrate a dynamic modulation of somatosensory–auditory
convergence and suggest the contribution of somatosensory information for speech processing process is
dependent on the specific temporal order of sensory inputs in speech production.
Ito T, Johns AR, Ostry DJ (2013)
Left lateralized enhancement of orofacial somatosensory processing due to speech sounds,
J Speech Lang Hear Res., 56, 1875-1881, doi: 10.1044/1092-4388(2013/12-0226).
Purpose: Somatosensory information associated with speech articulatory movements affects the perception
of speech sounds and vice versa, suggesting an intimate linkage between speech production and perception
systems. However, it is unclear which cortical processes are involved in the interaction between speech
sounds and orofacial somatosensory inputs. The authors examined whether speech sounds modify orofacial
somatosensory cortical potentials that were elicited using facial skin perturbations. Method: Somatosensory event-related potentials in EEG were recorded in 3 background sound conditions
(pink noise, speech sounds, and nonspeech sounds) and also in a silent condition. Facial skin deformations
that are similar in timing and duration to those experienced in speech production were used for somatosensory
stimulation. Results: The authors found that speech sounds reliably enhanced the first negative peak of the
somatosensory event-related potential when compared with the other 3 sound conditions. The enhancement
was evident at electrode locations above the left motor and premotor area of the orofacial system.
The result indicates that speech sounds interact with somatosensory cortical processes that are produced
by speech-production-like patterns of facial skin stretch. Conclusion: Neural circuits in the left hemisphere, presumably in left motor and premotor cortex,
may play a prominent role in the interaction between auditory inputs and speech-relevant somatosensory
Ito T, Ostry DJ (2012)
Speech sounds alter facial skin sensation.
J Neurophysiol, 107:442-7, doi: 10.1152/jn.00029.2011.
Interactions between auditory and somatosensory information are relevant to the neural processing
of speech since speech processes and certainly speech production in- volves both auditory information
and inputs that arise from the muscles and tissues of the vocal tract. We previously demonstrated that
somatosensory inputs associated with facial skin deformation alter the perceptual processing of speech
sounds. We show here that the reverse is also true, that speech sounds alter the perception of facial
somatosensory inputs. As a somatosensory task, we used a robotic device to create patterns of facial
skin deformation that would normally accompany speech production. We found that the perception of the
facial skin deformation was altered by speech sounds in a manner that reflects the way in which auditory
and somatosensory effects are linked in speech production. The modulation of orofacial somatosensory
processing by auditory inputs was specific to speech and likewise to facial skin deformation.
Somatosensory judgments were not affected when the skin deformation was delivered to the forearm or
palm or when the facial skin deformation accompanied nonspeech sounds. The perceptual modulation that
we observed in conjunction with speech sounds shows that speech sounds specifically affect neural
processing in the facial somatosensory system and suggest the involvement of the somatosensory system
in both the production and perceptual processing of speech.
Ito T, Ostry DJ (2010)
Somatosensory contribution to motor learning due to facial skin deformation.
J Neurophysiol, 104:1230-8, doi: 10.1152/jn.00199.2010.
Motor learning is dependent on kinesthetic information that is obtained both from cutaneous afferents
and from muscle receptors. In human arm move- ment, information from these two kinds of afferents is
largely corre- lated. The facial skin offers a unique situation in which there are plentiful cutaneous
afferents and essentially no muscle receptors and, accordingly, experimental manipulations involving
the facial skin may be used to assess the possible role of cutaneous afferents in motor learning.
We focus here on the information for motor learning pro- vided by the deformation of the facial skin
and the motion of the lips in the context of speech. We used a robotic device to slightly stretch the
facial skin lateral to the side of the mouth in the period immedi- ately preceding movement. We found
that facial skin stretch increased lip protrusion in a progressive manner over the course of a series
of training trials. The learning was manifest in a changed pattern of lip movement, when measured after
learning in the absence of load. The newly acquired motor plan generalized partially to another speech
task that involved a lip movement of different amplitude. Control tests indicated that the primary source
of the observed adaptation was sensory input from cutaneous afferents. The progressive increase in lip
protrusion over the course of training fits with the basic idea that change in sensory input is attributed
to motor performance error. Sensory input, which in the present study precedes the target move- ment, is
credited to the target-related motion, even though the skin stretch is released prior to movement initiation.
This supports the idea that the nervous system generates motor commands on the assumption that sensory
input and kinematic error are in register.
Ito T, Tiede M, Ostry DJ (2009)
Somatosensory function in speech perception.
Proc Natl Acad Sci U S A 106:1245-1248, doi: 10.1073/pnas.0810063106.
Somatosensory signals from the facial skin and muscles of the vocal tract provide a rich source of
sensory input in speech production. We show here that the somatosensory system is also involved in
the perception of speech. We use a robotic device to create patterns of facial skin deformation that
would normally accompany speech production. We find that when we stretch the facial skin while people
listen to words, it alters the sounds they hear. The systematic perceptual variation we observe in
conjunction with speech-like patterns of skin stretch indicates that somatosen- sory inputs affect
the neural processing of speech sounds and shows the involvement of the somatosensory system in the
perceptual processing in speech.
Ito T, Gomi H (2007).
Cutaneous mechanoreceptors contribute to the generation of a cortical reflex in speech.
Neuroreport 18(9): 907-910, doi: 10.1097/WNR.0b013e32810f2dfb.
Owing to the lack of muscle spindles and tendon organs in the perioral system, cutaneous receptors
may contribute to speech sensorimotor processes. We have investigated this possibility in the context
of upper lip reflexes, which we have induced by unexpectedly stretching the facial skin lateral to the
oral angle. Skin stretch at this location resulted in long latency reflex responses that were similar
to the cortical re£exes observed previously. This location reliably elicited the re£ex response, whereas
the skin above the oral angle and the skin on the cheek did not. The data suggest that cutaneous
mechanoreceptors are narrowly tuned to deformation of the facial skin and provide kinesthetic information
for rapid sensorimotor processing in speech.
Ito T, Kimura T, and Gomi H (2005).
The motor cortex is involved in reflexive compensatory adjustment of speech articulation.
Neuroreport 16(16): 1791-4.
Although speech articulation relies heavily on the sensorimotor processing, little is known about its
brain control mechanisms. Here, we investigate, using transcranial magnetic stimulation, whether the
motor cortex contributes to the generation of quick sensorimotor responses involved in speech motor
coordination. By applying a jaw-lowering perturbation, we induced a reflexive compensatory upper-lip
response, which assists in maintaining the intact labial aperture in the production of bilabial fricative
consonants. This reflex response was significantly facilitated by subthreshold transcranial magnetic
stimulation over the motor cortex, whereas a simple perioral reflex that is mediated only within the
brainstem was not. This suggests that the motor cortex is involved in generating this functional
reflexive articulatory compensation.
Ito T, Gomi H, Honda M (2004).
Dynamical simulation of speech cooperative articulation by muscle linkages.
Biol Cybern 91: 275-282, doi: 10.1007/s00422-004-0510-6.
Different kinds of articulators, such as the upper and lower lips, jaw, and tongue, are precisely coordinated
in speech production. Based on a perturbation study of the production of a fricative consonant using the upper
and lower lips, it has been suggested that increasing the stiffness in the muscle linkage between the upper
lip and jaw is beneficial for maintaining the constriction area between the lips (Gomi et al. 2002).
This hypothesis is crucial for examining the mechanism of speech motor control, that is, whether mechanical
impedance is controlled for the speech motor coordination. To test this hypothesis, in the current study we
performed a dynamical simulation of lip compensatory movements based on a muscle linkage model and then evaluated
the performance of compensatory movements. The temporal pattern of stiffness of muscle linkage was obtained
from the electromyogram (EMG) of the orbicularis oris superior (OOS) muscle by using the temporal
transformation (second-order dynamics with time delay) from EMG to stiffness, whose parameters were experimentally
determined. The dynamical simulation using stiffness estimated from empirical EMG successfully reproduced the
temporal profile of the upper lip com- pensatory articulations. Moreover, the estimated stiffness variation
significantly contributed to reproduce a functional modulation of the compensatory response. This result supports
the idea that the mechanical impedance highly contributes to organizing coordination among the lips and jaw.
The motor command would be programmed not only to generate movement in each articulator but also to regulate
mechanical impedance among articulators for robust coordination of speech motor control.
Ito T, Murano EZ, Gomi H (2004).
Fast force generation dynamics of human articulatory muscles.
J Appl Physiol 96: 2318-2324, doi: 10.1152/japplphysiol.01048.2003.
To explore the mechanisms of speech articulation, which is one of the most sophisticated human motor skills
controlled by the central nervous system, we investigated the force-generation dynamics of the human speech articulator
muscles [orbicularis oris superior (OOS) and inferior (OOI) muscles of the lips]. Short-pulse electrical stimulation (300 s)
with approximately three or four times the sensation threshold intensity of each subject induced the muscle response.
The responses of these muscles were modeled as second-order dynamics with a time delay (TD), and the model parameters
[natural frequency (NF), damping ratio (DR), and TD] were identified with a nonlinear least mean squares method.
The OOS (NF: 6.1 Hz, DR: 0.71, TD: 14.5 ms) and OOI (NF: 6.1 Hz, DR: 0.68, TD: 15.6 ms) showed roughly similar characteristics
in eight subjects. The dynamics in the tongue (generated by combined muscles) also showed similar characteristics
(NF: 6.1 Hz, DR: 0.68, TD: 17.4 ms) in two subjects. The NF was higher, and the DR was lower than results measured for
arm muscles (NF: 4.25 Hz, DR: 1.05, TD: 23.8 ms for triceps long head), indicating that articulatory organs adapt for
more rapid movement. In contrast, slower response dynamics was estimated when muscle force data by voluntarily contraction
task were used for force-generation dynamics modeling. We discuss methodological problems in estimating muscle dynamics
when different kinds of muscle contraction methods are used.
Ito T, Gomi H, Honda M (2003).
Articulatory coordination by muscle-linkage during bilabial utterances.
Acoust. Sci. & Tech. Acoustical letter 24(6): 391-393, doi: 10.1250/ast.24.391.
Previous studies showed that the upper lip compensatory movement against the downward perturbation to the jaw is effective
in maintaining labial constriction or in attaining labial contact for the production of bilabial utter- ances. We measured
the stiffness of the muscle-linkage between the upper lip and jaw and proposed a model in which the muscle-linkage plays
key role in coordinating articulators during speech. To further examine the compensatory mechanism using muscle-linkage,
we here investigate the coordination change of the lip-jaw system under the upward perturbation condition, and examine
the directional difference between the upward and downward perturbation. Based on the experimental observations, we propose
an extended lip-jaw model, which could explain the directional difference.
Ito T, Gomi H, Honda M (2003).
Articulatory coordination using mechanical linkage between upper lip and jaw examined by jaw perturbation.
The IEICE trans. on Infor. & System, PT.2 J86-D-II(2): 333-341 (in Japanese).
[Abstract (in Japanese)]
Gomi H, Ito T, Murano EZ, Honda M (2002).
Compensatory articulation during bilabial fricative production by regulating muscle stiffness.
J Phonetics 30(3): 261-279, doi: 10.1006/jpho.2002.0173.
The cooperative mechanisms in articulatory movements were examined by using mechanical perturbations during bilabial phonemic tasks.
The first experiment compares the differences in compensatory responses during sustained productions of the bilabial fricative /φ/
for which lip constriction is required, and /a/, for which the lips and jaw are relatively relaxed.
In the second experiment, we perturbed jaw movement with different load-onsets in the sentence ‘‘kono /aφaφa/ mitai’’.
In both experiments, labial distances were recovered partly or fully by the downward shifts of the upper lip.
The upper lip response was frequently prior to the EMG response observed in the sustained task. Additionally, initial downward
displacement of the upper lip was frequently larger when the load was supplied during /φ/ than when it was supplied during
/a/ in the sustained and sentence tasks, respectively. The stiffness variation estimated by using a muscle linkage model indicates
that the stiffness increases for the bilabial phonemic task in order to robustly configure a labial constriction.
The results suggest that the change in passive stiffness regulated by the muscle activation level is important in generating
quick cooperative articulation.
Fukao Y, Nonami K, Ohnuki O, Ito T, Fujimoto T, Naruke SA (2000).
Sensorless positioning control of two link robot arm by means of parameter identification.
Transactions of the Japan Society of Mechanical Engineers (C) 66(648): 190-198, doi: 10.1299/kikaic.66.2660 (in Japanese).
In this study,we have extended the previous sensorless positioning control of single-link robot arm to a two-link robot arm system.
Sliding mode control is useful for multiple-link robot arm which contains the nonlinear characteristics at the drive system.
And, in the sensorless control, a variation of coil resistance which is used as an uncertain parameter of the actuator.
Such uncertain parameter has been estimated by using parameter identifier. Moreover, it has been verified that our provde control
scheme is very useful for the sensorless control system with unknown parameters. We proved the usefulness by experiments.
Ito T, Nonami K (1999).
Application of sliding mode control with robust hyperplane to flexible structure.
Transactions of the Japan Society of Mechanical Engineers (C) 65(629): 161-166 doi: 10.1299/kikaic.65.161 (in Japanese).
Sliding mode control theory is nonlinear robust control theory and one of variable structure control.
This theory has good performance against uncertainty satisfied matching condition. But the conventional
sliding mode control system often becomes unstabled due to high frequency vibration caused by unmodelled
dynamics. In this paper, we consider application of Frequensy-Shaped Sliding Mode Control (FSSMC) to
flexible structure. Flexible structure has uncertainty on the control input side, e.g. time delay,
friction, and so on. It is difficult for control of flexible structure because of these uncertainty,
when we applied a linear control. Applying sliding mode conrol, we can design a eternal system against
these uncertainty and dynamics for sake of mating condition satisfied. Moreover, closed loop system does
not become unstabled due to ummodelled dynamics for sake of robust hyperplane designed
using H∞ theory.
For example of application, we have just applied to the fiexible truss structure with control momentam
gyro (CMG). We have verified from simulation and experiment that this method has good performance.
Ito T, Nonami K (1997).
Sliding mode control with frequency shaping to suppress spillover.
Transactions of the Japan society of mechanical engineers (C) 63(611): 120-126, doi: 10.1299/kikaic.63.2308 (in Japanese).
This paper deals with a new design method for sliding mode control with frequency shaping.
The concept is suppression of spillover phenomena for a flexible system. A conventional hyperplane
consists of a desired reference model without dynamics. Therefore, the sliding mode control system
often becomes unstable due to chattering and spillover phenomena in the high-frequency region.
This method aimes to suppress a control input in a high-frequency region by filtering control input
or measured output using a low-pass filter. The new sliding mode control system is designed as an
augmented system which consists of a reduced-order model and a low-pass filter. We have applied
this method to a four-story flexible structure. We have verified from simulations and experiments
that this method has good performance and is very useful for suppressing spillover phenomena.
Ito T, Nonami K, Iwamoto K, Nishimura H (1996).
Active vibration control of flexible structure by frequency-shaped sliding mode control with μ synthesis theory.
Transactions of the Japan Society of Mechanical Engineers (C) 62(602): 112-119, doi: 10.1299/kikaic.62.3850 (in Japanese).
This paper proposes a new design method for a sliding mode control system using μ synthesis theory.
This concept is based on a frequency-shaped approach. A general hyperplane consists of a reference model
without dynamics. Therefore, a conventional sliding mode control system often becomes unstable due to
spillover phenomena of high frequency caused by high-speed switching. The proposed design method suppresses
such spillover phenomena because of frequency shaping. Also, it has good robustness on the hyperplane
to realize a minimum value of H∞ norm and the structured singular value from some
noise to state variables.
We have just applied this new method to the flexible structure of a test rig with four stories such as a
high-rise building. We have obtained good performance from simulations and experiments.
Nonami K, Ito T (1996).
μ Synthesis of flexible rotor-magnetic bearing systems.
IEEE, Transactions on Control Systems Technology 4(5): 503-512 doi: 10.1109/87.531917.
The μ synthesis design method was evaluated for a flexible rotor magnetic bearing system with a
five-axis-control system using both simulations and experiments. After modeling the full-order system
using the finite element method, we obtained a reduced-order model in the modal domain by truncating
the flexible modes. After choosing appropriate weighting functions with respect to frequency, we designed
the μ-synthesis control system using the μ-toolbox in MATLAB. We then carried out simulations of
the control system for a flexible rotor magnetic bearing system with a five-axis-control system and
obtained good performance. Next, we conducted experiments to verify the robustness of the controllers
on a test rig during initial levitation. The controllers provided robust stability and performance over
a wide range of parameter variations in the test rig.
Ito T, Nonami, K (1995).
μ Synthesis of flexible rotor-magnetic bearing systems.
Transactions of the Japan Society of Mechanical Engineers (C) 61(584): 173-178. (in Japanese).
The H∞ control theory, which is the most powerful method for robust control
theory to date, is applied to the flexible rotor-magnetic bearing system.
However, the H∞ control system has some disadvantages in that it is a
conservative system and can not be used to treat with robust performances. This is due to its
maximum singular value. Doyle proposed a structured singular value instead of maximum singular
value. This is called the μ synthesis theory which treats robust performances using D-K iteration.
This paper concerned with the μ control of a flexible rotor-magnetic bearing system (FR-MBS).
Plant dynamics, consisting of actuator dynamics and flexible rotor dynamics, are described.
The μ controller for the reduced-order model is designed by D-K iteration, and its robust
performances are evaluated with several experiments. The relationships between μ control,
H∞ control and their robust performances are discussed for the flexible
rotor-magnetic bearing system.
Ito, T, Orofacial cutaneous function in speech motor control and learning,
Handbook of Speech Production , Wiley-Blackwell, Edited by Melissa A. Redford, April 2015.
Somatosensory signals from facial skin can provide a rich source of sensory input. However, it is unknown
yet how cutaneous input works on speech motor control and learning. This chapter introduces a kinesthetic
role of orofacial cutaneous afferents in speech processing. We argue for specificity of the orofacial
somatosensory system from anatomical and physiological perspectives. The contribution of cutaneous afferents
to speech production is evident in neurophysiological and psychophysical findings. Somatosensory modulation
associated with facial skin deformation induces a reflex for articulatory motion adjustment in speech
production and also an adaptive motion change in speech motor learning. In addition, cutaneous mechanoreceptors
are narrowly tuned at the skin lateral to the oral angle. An intriguing function of somatosensory inputs
associated with facial skin deformation is to interact with the processing of speech perception.
Taken together, orofacial cutaneous afferents play an important role in both speech production and perception.