More often than not, buying a mostly stock to slightly modded wrecked WRX isn't worth the hassle. You're better off cutting your losses and finding another one.
If the car was extremely rare and valuable to collectors or it had all kinds of goodies on it that were worth more than the entire car was worth, then yeah, I'd say it would be worth buying it back. Also keep in mind that used WRX parts aren't as popular as used STi parts so you would probably have to sell the majority of the salvageable parts on the car just to make it worth the buy-back price. Think about the time you would have to spend disassembling/packing everything, finding buyers, negotiating deals, and just storing the car on your property until it's gone. Then you would have to find a way to get rid of the frame and/or whatever remains once you're satisfied that everything worth something is removed.
If, on the other hand, you intend to buy the car back to fix it and continue driving it yourself and there was little to no frame damage and you had the ability to inexpensively repair the damage, then yeah, that might be worth doing. If you go this route, though, you would be driving a car with a salvage title and the only real way to get your money's worth out of that is if you keep it until it completely falls apart because you usually don't get much money from selling a car with a salvage title.
It sounds like you might have some kind of attachment to this car, but let it go Chris... just let it go !Cry!