Rice flour? That's gluten free, right? Instead of breadcrumbs my mom used to use smashed-up wheaties (I know, I know, I grew up in Ohio and we ate weird things). Rice Chex are gluten free, aren't they? You could smash them up and use them as a binding agent.
Try rolling them in your hands with as you form them with your hands dusted in gluten-free flour - or roll them in the flour. Then pop them in the freezer for 10 minutes, they should be ok from there, whether frying or baking.
I am gluten free - so fuck the wheatgerm and breadcrumbs! haha.
I bet the fridge thing would help. My balls may be too warm.
Maybe try a bit of tapioca flour? It has a very neutral taste, as opposed to rice flour which is pretty distinctive so it shouldn't interfere with the taste of your balls.