Well, while I think it's okay to keep your options open since you and Gym Guy #1 are not exclusive, I would NOT go out on a date with Gym Guy #2. That would just be all kinds of awkward imho.
If one guy was from the gym and the other from let's say an arts class or something, I would say go for it. Going out with two people from the same place doesn't sound like a really good idea.