I've seen this a few times with my car and I asked my mechanic about it.
He explained it like this...
The Subaru's run with positive crankcase pressure. This creates kind of an oil steam in the engine that is run back into the intake system. Like all hot liquids, they will condense when they hit something cool. Since your top mount intercooler is cool, it will gather there. That's why when you pop off the intercooler there is a little bit of oil in the coupling. On start up, this will come out your exhaust b/c your catalytic converter isn't warm yet. Once the cat is warmed up, it will burn off this oil so you don't see it out the tailpipe.
That's how it was explained to me, and it seems to make sense. After about 3k miles I'm about 1/4 quart low on oil. I try to check it every 1000 miles now.
2005 WRX Cobb Stage 2