I can totally appreciate wanting to share your family values vis a vis music. Us too. DH and I toyed around with the idea of piano when DS was younger, but having seen our youngest niece (who is probably on spectrum, too) turn her back on music after starting Suzuki at 2 1/2 we decided to wait. Niece quit music in 7th grade to concentrate on the tennis that got her into one of the service academies.
I can also appreciate not wanting to allow him to take time out from music education in a period where it's not going well and have him perceive it as a victory of manipulation that could be globalized to other battles.
We decided to wait until DS was older. In the meantime, we dragged DS to lots of concerts and plays. The Philly Orchestra has a children's and family programs we did when he was little. We also did summer orchestra and pops concerts at the Mann. My godchild is an actress and we hauled him to see her in countless musicals.
On the advice of DS's psych, we focused on band rather than orchestra. Locally, band is the stronger and more social program. The band room became a smaller community within the huge public school he attended. Membership in band gave him access to a readymade social group, the protection of that group and the experience of being a part of a team. Where we live, orchestra and choral music don't offer the same social opportunities.
We allowed him to select his instrument and we made it a non-negotiable part of his education as part of our family values. DS is something of a rule boy, so we made a rule that he would participate in one in-school and one community activity to help him be well rounded and to give him social skills practice. He chose band and scouts which have a lot of overlap.
I suspect that the relationship between music and math talent as much inborn as a function of working at either skill.
For us, backing off until DS could see the benefits of music and could commit made sense. I don't think he's touched his trombone since he graduated from high school, but he plays his guitar a lot and has a comprehensive knowledge and appreciation of all kinds of music.
I'd compromise, I'd set an end point (end of summer/after the next concert/end of first 6 weeks of school) that he has to complete before he can stop thereby not letting his tantrums end it but you setting an end point. I wouldn't force this because it may turn him off music entirely. I'd do what Auntie suggests and focus on bringing music in through other means for now and revisit when it starts at school. At that point he may be more interested and motivated to participate. And be willing to accept that some kids just aren't into the same things as their parents and that is perfectly OK. One of the worst things my Dad did was try to force my art and music loving brother to be a Jock. It led to a LOT of resentment and while there is a lot more to the story it contributed to my brother having no relationship with our Dad as an adult. find his interest and support and encourage that.