It was a lot easier to remember when I was on BC. I am not a big fan of going to the doctor. At least my last obgyn so I think this is why I kinda just forgot every year. But I do remember getting notice that she was no longer practicing. I had planned to go in September (before TTC) but I just didn't get around to it.
I heard that if you're in a committed relationship and don't have any other serious issues, then the yearly exam is overkill. I think they recommended once every 2-3 years.