I know what you're talking about, and it is extremely irritating. I'm probably the last person on this forum who should attempt to give horse advice, but here goes

:
What has helped me is to say, "Great, lets go back to your pals," and when you get to them, work the crap out of him...make it work when he's around the others. I usually just lope/trot circles around the other riders, stop and turn, change directions, mix it up and get him listening to you. When you head off alone again, he will probably try to pull his trick again, so go back to the other horses and repeat. Obviously set this up when you have time to make it a training session since it may take awhile. Hopefully your horse will clue in that going away from others can be relaxing, and acting like a jerk will only get him more work.
Or, if you don't want to go back to the others, just work the crap out of him where you're at when he tries to pull his crap with you away from the other horses. The whole goal is to get him thinking about you and not anything else. This has worked for me, but I'm certainly NO horse expert

.
As for spurring him, I agree, but I'm too chicken to spur the crap out of one when I'm off alone. But it sounds like you're a bull rider, so I'm betting it wouldn't bother you

.
There are good horse people on this forum, hopefully one of them will speak up and give us both some advice, I'd really like to hear what you're supposed to do

.