DH's crazy season at work is during the summer -- it's all summer long, and he's barely ever home. He tries to make it home before the kids go to bed at least once or twice a week (we have two under 6), but he'll often work 24 hours in a row, sleep for 5, and then work another 18, before collapsing for 12 hours on a weekend.
As a result, I don't plan for him to be a part of any activities. If he can, great, but if not, no disappointment this way.
I'm going to visit my best friend who lives 6 hours way next month. We only see each other about 1x a year. I've had this trip planned for months: We're going to drive up halfway on Thursday evening after work, stay overnight in a hotel in a fun town, explore the town on Friday, then drive the rest of the way and be at her lake house Friday afternoon. Drive home Sunday morning (friends are leaving their house Sunday morning too, so staying that day isn't an option).
DH first said he would definitely go. Now he's leaning toward no. Now he wants us to drive up Friday "late afternoon", get to their lake house super late Friday and leave Sunday morning. I think this sucks, will be way too much time in the car for the kids, and just isn't nearly as much fun as the way I'd planned it. Friday traffic where we live SUCKs and it will take us forever.
I know he feels guilty that he's barely home in the summer and that he'll be missing what is essentially our only fun "vacation" this summer, but I don't think it's fair that I have to basically cut it in half. Plus, when he says we'll leave "late afternoon" on Friday, I know that it will likely not be until the evening, and I'll be pissed with him as he's making us wait. I was fine with leaving Thursday night because I don't really care if we get to the hotel late, but I don't want to miss a ton of time with my friend.
Our trip this summer was going that way. Now I'm going solo, and we extended our Christmas excursion by a couple days to compensate.
I think you should try making a plan for something awesome in your area you can all do, with him, a couple weeks after the fact. The zoo, an amusement park, the aquarium, even a fun place to eat you haven't tried. Then he'll still feel left out but at least he has something fun and unique to look forward to.
If you go without him can you leave a couple of days earlier and get even more time with your friend? Then if he wants he can always drive up Friday night and spend the weekend (if the gas wouldn't be ridiculous).
Maybe I'll offer the idea of him driving up Friday night. I know he likely won't go for it, but at least it's an option. I can't go up earlier because I have to work on Thursday.
We always plan fun vacations in the fall, so I don't know why I feel guilty about this.