Muhammad bin Jubayr bin Mutim used to relate that news reached Muawiya, while he was with a delegation from Quraysh in his presence, that Abdullah bin Amr bin As was reporting that there would come a king from Qahtan. Muawiya became angry. He stood up, praised Allah as He deserves to be praised, then said,"To proceed. It has reached me that some men among you are relating reports that are neither in the Book of Allah nor transmitted from the Messenger of Allah ﷺ. Those are the ignorant among you. Beware of false hopes that lead their holders astray, for I heard the Messenger of Allah ﷺ say, 'This authority belongs to Quraysh. No one will contend with them over it without Allah casting him down on his face, so long as they uphold the religion.'"
حدثنا بشر بن شعيب بن أبي حمزة قال حدثني أبي
كان محمد بن جبير بن مطعم يحدث أنه بلغ معاوية وهو عنده في وفد من قريش أن عبد الله بن عمرو بن العاص يحدث أنه سيكون ملك من قحطان فغضب معاوية فقام فأثنى على الله عز وجل بما هو أهله ثم قال أما بعد فإنه بلغني أن رجالا منكم يحدثون أحاديث ليست في كتاب الله ولا تؤثر عن رسول الله ﷺ أولئك جهالكم فإياكم والأماني التي تضل أهلها فإني سمعت رسول الله ﷺ يقول إن هذا الأمر في قريش لا ينازعهم أحد إلا أكبه الله على وجهه ما أقاموا الدين
Its isnad is sahih, its narrators have already been mentioned. The hadith is found in similar wording in Bukhari, 13/113, no. 7139, in Fath al-Bari, in the Book of Rulings, chapter "Rulers are from Quraysh," and in Tabarani's Al-Kabir, 19/338, no. 781, in wording similar to Ahmad's.However, the Companions did not concede this view to Muawiya, despite the soundness of the hadith he cited as evidence, because the hadith itself refutes his position. It stipulated that they must uphold the Book of Allah, so if they neglected it, authority would slip from their hands. This is indeed what happened when the Abbasid state fell.