If your car is 100% stock (I'm assuming it is since you didn't list any mods), the absolute #1 best mod you could do, hands down, is: new tires. I'm sure just about everybody will agree that the stock tires that come on the WRX are complete garbage.
If you're looking for power, I'll second what everybody said so far: get a downpipe, cat back, and tune for it. After those specific things, you'll be at "Stage 2." You can put an aftermarket cat back on any time without a tune and be just fine, but when you get the downpipe, that's when you'll really need the tune.
As far as low-budget mods, you could get a short throw shift lever w/ front and rear polyurethane bushings, a lightweight crank pulley (NOT UNDERDRIVE PULLEYS!), perform the "Snorkus Delete" mod for a little more intake sound, K&N panel replacement air filter, rear swaybar w/ endlinks, make sure your transmission fluid and rear differential fluids are fresh (should be changed every 30K miles), get a nice synthetic oil in your engine... the list goes on. There really are tons of things you can do to your car for not a whole lot of money, but there are just as many expensive mods too, if not more.
If you haven't done so already, maybe you should skim through the information here:
Unabomber's Manifesto - NASIOC
Oh, and I almost forgot, welcome to the club!