The TD04 is a small turbo. You're going to be operating at higher flowrates than are optimal from the turbo's perspective. That means it's past it's optimal point in terms of efficiency. That means the pressure will drop. That doesn't mean it's being pushed too hard though, it's designed to work over a variety of flowrates. The flowrate of your car changes over the RPM range anyway to don't fool yourself that if your turbo hits peak efficiency at some point that everything is perfect anyway. 100rpms later, things are different.
To keep things real high level, you just need to stop worrying about pressure and trying to associate more pressure with more power. Focus on flowrate*
*Volumetric flowrate is typically used and is not really what I'm talking about here. It's the number of air molecules you're sucking in. More air volume means more air molecules but so does air under more pressure and so does air at lower temperatures. It's the density as much as it is the volumetric flowrate. It's more the molecular flowrate if that makes any sense to you.
2015 STi: Still Stock
04 STi, 04 WRX: SOLD