I test-drove a GX470 and a RX 300 on the same day, one after another. The RX has more passenger legroom (width and length) because it's a unibody, not a body-on-frame. The GX driver seat seemed low (like a 4Runner) and it was tough to get thigh support, which is tough on any 4Runner-class truck. I liked the GX, but wasn't blown out of the water by it. The rear-most (cargo-area) seats have zero foot/legroom - not sure who can ride back there. I would remove the seats and toss them in my garage.
I'm looking to replace our '08 Sienna, because it's such a barge to park, and we don't need the capacity. The replacement vehicle will be a wife-mobile, not a 'wheeler. So, I am looking at Highlander, RX 300/330, etc. as well as the GX.
Agreed that if you want a 'wheeler, it won't be anything based on a Camry (RX, etc.) I really liked the GX's shift lever, low range, diff lock, etc. I think it would be fun for reasonable offroad use.