Ayesha reported,
The Prophet ﷺ said, "Do you not see that when your people built the Kaabah, they fell short of the foundations of Ibrahim?"Ayesha said, "I asked, 'Messenger of Allah, will you not restore it to the foundations of Ibrahim?'"The Messenger of Allah ﷺ replied, "Were it not that your people are recent in their abandonment of unbelief, I would have done so."Abdullah bin Umar said, "If Ayesha truly heard this from the Messenger of Allah ﷺ, then I do not think the Messenger of Allah ﷺ refrained from touching the two corners adjoining the Hijr except because the House had not been completed upon the foundations of Ibrahim."
Azami 8381
Sahihعن عائشة
أن النبي ﷺ قال: ألم تري أن قومك حين بنوا الكعبة اقتصروا عن قواعد إبراهيم قالت: فقلت: يا رسول الله أفلا تردها على قواعد إبراهيم فقال رسول الله ﷺ: لولا حدثان قومك بالكفر لفعلت قال: فقال عبد الله بن عمر: لئن كانت عائشة سمعت هذا من رسول الله ﷺ ما أرى رسول الله ﷺ ترك استلام الركنين اللذين يليان الحجر إلا أن البيت لم يتمم على قواعد إبراهيم.
Malik narrated it in Hajj (807) from Ibn Shihab, from Salim bin Abdullah, that Abdullah bin Muhammad bin Abu Bakr Siddiq informed Abdullah bin Umar, from Ayesha, and he mentioned it. Bukhari narrated it in Hajj (1583), and Muslim in Hajj (1333:399), both through the route of Malik with the same wording.Bukhari narrated it in Hajj (1586) from Bayan bin Amr, who said, Yazid told us, who said, Jareer bin Hazim told us, who said, Yazid bin Ruman told us, from Urwa, from Ayesha, that the Prophet ﷺ said to her, "Ayesha, were it not that your people are so recently removed from paganism, I would have ordered that the House be torn down, and I would have included in it what was left out of it, and I would have brought it level with the ground, and I would have made for it two doors, an eastern door and a western door, and I would have built it up to the foundation of Ibrahim."That is what led Ibn Zubayr 
to tear it down. Yazid said, I witnessed Ibn Zubayr when he tore it down and rebuilt it, and included the Hijr within it. I saw the foundation of Ibrahim, stones like the humps of camels.Jareer said, So I asked him, "Where is its place?" He said, "I will show it to you now." So I entered the Hijr with him, and he pointed to a spot and said, "Here." Jareer said, So I estimated from the Hijr about six cubits or thereabouts.