During the time of the Prophet ﷺ, there was no wall around the Kaaba. People would pray around the Kaaba until Umar came and built a wall around it.Ubaydullah said, "Its wall was low, so Ibn Zubayr rebuilt it."
لم يكن على عهد النبي ﷺ حول البيت حائد، كانوا يصلون حول البيت، حتى كان عمر فبنى حوله حائطا. قال عبيد الله: جدره قصير، فبناه ابن الزبير.
Bukhari narrated it in the Virtues of the Ansar (3830) from Abu Numan, who said, Hammad bin Zayd narrated to us from Amr bin Dinar and Ubaydullah bin Abu Yazid, who both said, and then they related it.His saying, "jidarah" with a fatha on the jeem and no vowel on the dal, means the wall, its plural being jidran.