We went to Mallorca for 6 days but didn't actually stay in Palma. For 3 nights we were about a 30 min drive from Palma so we drove into the city for a few hours. I was fine with that--we toured the Cathedral, walked around and had dinner before heading back to our hotel.
I replied on the international board too. As far as 'need,' it obviously depends on what you want to see. We stayed two nights and three days, with part of the time in Palma de Mallorca, one day in Soller, and the rest on the beach/on our terrace overlooking the beach. It was meant to be a bit of R&R.
There are some cool things to see outside of Palma--Miro's studio is there, for instance, and there are many interesting walks and beaches to see. Enjoy it!