People used to depart in every direction, so the Messenger of Allah ﷺ said, "No one should leave until his final act is at the House."
حدثنا سفيان عن سليمان عن طاوس
كان الناس ينصرفون في كل وجه فقال رسول الله ﷺ لا ينفر أحد حتى يكون آخر عهده بالبيت
Its isnad is sahih. It was also narrated by Muslim, Abu Dawud, and Ibn Majah, and Bukhari narrated something similar in meaning, as in Muntaqa 2669, 2670.