He addressed us, and Mura's version says, the Messenger of Allah ﷺ addressed us and said, "People, Allah has made the pilgrimage obligatory upon you, so perform the pilgrimage."A man asked, "Every year, Messenger of Allah?"He remained silent until the man had asked it three times, then the Messenger of Allah ﷺ said, "Had I said yes, it would have become obligatory, and you would not have been able to bear it."Then he said, "Leave me alone as long as I leave you alone. Those before you were destroyed only because of their excessive questioning and their disagreements with their prophets. So when I command you to do something, do as much of it as you can, and when I forbid you from something, abandon it."
حدثنا يزيد أخبرنا الربيع بن مسلم القرشي عن محمد بن زياد
خطبنا وقال مرة خطب رسول الله ﷺ فقال أيها الناس إن الله عز وجل قد فرض عليكم الحج فحجوا فقال رجل أكل عام يا رسول الله فسكت حتى قالها ثلاثا فقال رسول الله ﷺ لو قلت نعم لوجبت ولما استطعتم ثم قال ذروني ما تركتكم فإنما هلك من كان قبلكم بكثرة سؤالهم واختلافهم على أنبيائهم فإذا أمرتكم بأمر فأتوا منه ما استطعتم وإذا نهيتكم عن شيء فدعوه
Its isnad is sahih. Muslim reported it, 2/975, no. 1337, and Nasai, 5/111, and Ibn Majah, 2/963, no. 2885, all of them in the chapter on Hajj.