I love miss Shirley's. The Annapolis one is pretty big so they handle groups well.
I asked the local nest FB group for recs because I've been out of the loop since I left my job down there 4 years ago and got the following:
Iron Rooster - opens at 7. Annnapolis Smokehouse - doesn't start brunch till 11 on Saturdays BarOak - Brunch starts at 10 Main Ingredient - opens at 7:30
Gorelick has a review of Iron Rooster in the Sun where he practically makes reservations for you he likes it so much, so there's that. I've had Main Ingredient catering before and was impressed, but I've never eaten at any of these places. BarOak and Iron Rooster are both right up my alley in terms of things I'll drool on, but any of them (or Miss Shirley's) look good to me.