Is it possible to set it so that only grenades can damage buildings?

no, if a player uses a weapon then the rule applies to the player and not the weapon.