I read some articles about how the sway bars work and I'm convinced they do improve the handling. Imagine taking a long sweeping right turn (let's say it's an on-ramp) at about 40 mph... with stock sway bars, the body of the car will lean (body roll) and the outside edge of the front right tire will start to lose contact with the road and more weight will be transferred to the left front tire until it loses traction and you begin to slide off the road nose first (oversteer). With a larger front and rear sway bar, the car will not lean nearly as much going through the turn, keeping more of the outside edge of the front right tire on the road, and allowing more traction there and less weight to be transferred to the left front tire. As a result, you'll be able to go through the turn faster than you would be able to without the larger aftermarket sway bars. I've experienced this first hand. I have a few favorite country roads that I've pushed my car to the point where the tires were squealing going through some turns when there was no traffic and now I can take those same turns a bit faster without the tires squealing. I'd call that an increase in handling.
If you were just talking about the aftermarket endlinks not improving handling, I'd say you were right. They don't increase handling, and, in fact, I'm pretty convinced that the design of the factory front end links is just fine. Maybe they could be a little thicker, but judging by the design, they look pretty good to me. The rears, being plastic, definitely benefit from an upgrade. Like you said, they just help deliver more clear feedback and they're basically just a support piece for the bar. The bars do the real work.