The Prophet ﷺ said, "Children of Abd Manaf, do not prevent anyone from circumambulating this House and praying at any hour they wish, by night or by day."
أخبرنا محمد بن منصور قال حدثنا سفيان قال سمعت من أبي الزبير قال سمعت عبد الله بن باباه يحدث
أن النبي ﷺ قال يا بني عبد مناف لا تمنعوا أحدا طاف بهذا البيت وصلى أية ساعة شاء من ليل أو نهار
Its isnad is sahih. Abu Dawud narrated it in the Book of Hajj Rites, chapter on tawaf after Asr, hadith no. 1894, and Tirmidhi in the Book of Hajj, chapter on what has been said about prayer after Asr, and so on, hadith no. 868, and Ibn Majah in the Book of Establishing Prayers, chapter on what has been said about the concession to pray in Makkah at any time, hadith no. 1254, from the hadith of Sufyan bin Uyayna as narrated. It is also found in al-Kubra, hadith no. 1561. Tirmidhi said, hasan sahih, and Hakim graded it sahih according to the criterion of Muslim, 1/448, and Dhahabi agreed with him.