That size is a touch bigger than stock but the extra side wall is worth while in my mind. In 2 yrs or so this vehicle will be converted to weekend car with heavier off road use and I'll likely go up to 265/65r18 in E load at that time, so figured I'd go ahead and get the wheels as I like the look much more over the stocks. For now looking to keep the tire choice a bit more tame in line with my current expected use.
So let me say a few things for you to consider because we don't know your entire picture here. First, how many miles do you expect to roll before making it's change to more off-road/weekend uses and why would that time period then see tracks more rugged than now? I'm not asking how often, but if the use 2 yrs from now will be actually any more rugged?
Also, what makes you think the E load range tire will be that much different in the peppiness from a same size XL? I mean it's heavier by a bit but my guess is the felt difference after a few days driving wouldn't actually be something you'd cry about either. The lr4 is already a lot heavier than the 4runner so in a sense, a slightly heavier tire is a smaller % difference especially when also combining the higher output engine in the lr4. You can feel it, but it's also sort of irrelevant in the overall feel.
The reason I asked about mileage for next two yrs is if it's not much, then why partially use a tire that ought to last far longer? No savings occurs, in fact it's probably more cost effective to just get a tire like the BFG AT Ko2 that can last 60-70-80,000 miles, easily on a 5 tire rotation. I got 75k and still had 1/4" tread on a ko1 set LOL
One way to seriously increase wear life for lr4 tires is to do a "flip" every 25-30k or so. The inside edge wears a lot more due to the suspension design, not unlike a strong camber car in the rear.
Next, there are various load ranges, C D E. The factory rating is fine for an unmolested lr4 with not much added weight but as you add weight, you should do some homework on the load rating and YOUR use case. Mine or someone else on here is somewhat irrelevant because mine weighs more and I use it differently than you will.
If anything though, I would suggest going to the 31.5" size of 265/65-18 for far more reasons than there are against it. There is almost no reason not to except what you mentioned and it's still not going to handle like a sports car but even with an E load 265/65-18 BFG AT KO2, it will still handle better than the 4 Runner did due to the suspension design. The air makes for quite the nice magic carpet ride IMO.
I did use the Nitto briefly. To be honest, all those are 'meh' for the use you describe. I used to think I needed SO much tire, tough, aggressive, etc. Then a few yrs ago I decided to see how little I could use to do how much. 
