Garments are useful for PvP since they reduce the magic damage you take. PvE's a bit of a judgement call since you use a glaive. If you kill the mobs very quickly, use a garment. If you kill them more slowly, use protector.
The big reason to use garments for PvE is for the speed boost and reduction of mana cost of skills, but if you're standing toe-to-toe with a mob for any significant amount of time, you start losing more HP and it's not worth it.
A quick-killing glaive user can probably get the best benefit from garments, but as I said, if it takes you a bit longer, use protector. Armor's too slow and drains your mana too quickly, imo. Armor looks cool though, and can offer good protection for a sword user w/o ice.
Another factor is whether or not you use ice or some sort of ranged attack opener. If you use either ice or a nuke to start your fights, you'll probably benefit most from garments since the target will do less damage to you (either from being partially dead, or from being chilled).
Another thing to take into account is your phys/magical balance, since high physical balance reduces your damage intake from physical.
My suggestion is to get a set of each type of armor and try it out. I find that I take too much damage while wearing garments, so I switched to armor. With armor, however, I expend my mana too quickly. Protector is a happy medium for me right now, but that may change later as my balance changes.
