The only way it would be possibly okay for the lack of communication is if it was presented as, "Hey, we're going to be swimming all weekend, come over any time you want. Just shoot us a text right when you're on your way." Even at that, it's still polite to set a time.