I never saw anyone whisper into the ear of the Messenger of Allah ﷺ and have him pull his head away. Rather, the man himself would be the one to pull his head back. And I never saw anyone take his hand and have him withdraw it. Rather, the man himself would be the one to let go of his hand.
حدثنا أحمد بن منيع حدثنا أبو قطن أخبرنا مبارك عن ثابت
ما رأيت رجلا التقم أذن رسول الله ﷺ فينحي رأسه حتى يكون الرجل هو الذي ينحي رأسه وما رأيت رجلا أخذ بيده فترك يده حتى يكون الرجل هو الذي يدع يده
Its chain of transmission is daif. Bayhaqi narrated it in Dalail Nubuwwah (1/320) through Abu Dawud's route, and Ibn Hibban graded it sahih in Al-Ihsan, no. 6401.Mubarak bin Fadala did not explicitly state the chain of hearing throughout, and he used to practice tadlis al-taswiya. Part of the hadith has a daif corroborating witness in Ibn Majah, no. 3716, without mention of the ear.