I have a Perrin lightweight crank pulley too and when my A/C is on, I feel a noticeable decrease in power - actually, yeah, to the point where I think I have to give it a little more gas when accelerating from a complete stop. I would guess that everything is probably ok. I really only use my A/C several times a year - when I have other people in my car on a warm day or when I think it's uncomfortably hot (which doesn't happen a whole lot).
07 WRX (Albins 5MT + Blouch 20G)
Jorge Carrillo Tuned (365 whp/340 wtq)