Saad bin Abu Waqqas reported,
We were with the Prophet ﷺ, six men in all, and the polytheists said to the Prophet ﷺ, "Drive these people away, they must not presume to sit with us."The narrator said, "There was myself, Ibn Masud, a man from Huzayl, Bilal, and two others whom I will not name. Something then stirred in the heart of the Messenger of Allah ﷺ as Allah willed it to stir, and he turned it over in his mind. So Allah 
revealed, 'And do not drive away those who call upon their Lord morning and evening, seeking His face. You are not accountable for them in any way' [Anam 6:52]."
Azami 12306
Sahihعن سعد قال
كنا مع النبي ﷺ ستة نفر، فقال المشركون للنبي ﷺ: اطرد هولاء لا يجترؤون علينا، قال: وكنت أنا وابن مسعود ورجل من هذيل وبلال ورجلان لست أسميهما، فوقع في نفس رسول الله ﷺ ما شاء الله أن يقع فحدث نفسه، فأنزل الله عز وجل {ولا تطرد الذين يدعون ربهم بالغدوة والعشي يريدون وجهه ما عليك من حسابهم من شيء}.
Muslim narrated it in the Merits of the Companions (46:2413), from Abu Bakr bin Abu Shayba, who said, Muhammad bin Abdullah Asadi told us, from Israil, from Miqdam bin Shurayh, from his father, from Saad, who related it.Muslim also narrated it in the Merits of the Companions (2413:45), from Zuhayr bin Harb, who said, Abdul-Rahman told us, from Sufyan, from Miqdam bin Shurayh, from his father, from Saad, concerning the revelation of "Do not drive away those who call upon their Lord morning and evening" [Anam 6:52]. He said, It was revealed concerning six men, myself and Ibn Masud among them. The polytheists had said to him, "Would you bring these people near to you?" And his statement, "I will not name the two of them." In another version, "I have forgotten their names."Something similar has been reported from Ibn Masud. Ahmad narrated it (3985), and Ibn Jareer (9/258), and it contains Ashath, who is Ibn Sawwar Kindi, and he is daif. Except that Ahmad mentioned the verse preceding it, which is His statement, "And warn with it those who fear that they will be gathered..." [Anam 6:51–58], meaning that the leaders of Quraysh used to mock those who had believed, saying, "Are these the ones among us whom Allah has favored?" [Anam 6:53], and they also said, "If it were good, they would not have preceded us to it" [Ahqaf 46:11].