I attended the Eid with the Messenger of Allah ﷺ. He led us in the Eid prayer, then said, "We have completed the prayer. Whoever wishes to stay for the sermon may stay, and whoever wishes to leave may leave."
حدثنا هدية بن عبد الوهاب وعمرو بن رافع البجلي قالا حدثنا الفضل بن موسى حدثنا ابن جريج عن عطاء
حضرت العيد مع رسول الله ﷺ فصلى بنا العيد ثم قال قد قضينا الصلاة فمن أحب أن يجلس للخطبة فليجلس ومن أحب أن يذهب فليذهب
[Sahih] Abu Dawud narrated it in "the book of Prayer, the chapter on sitting for the sermon" (no. 1155) from the hadith of Fadl, and Ibn Khuzayma, Hakim, and Dhahabi graded it sahih. It has been criticized on a point that does not undermine it.