Travelling with friends can be a challenge whether or not you are all interested in sketching. Sometimes it can be more difficult to travel with other artists, especially when you're not going with a tour group. Everybody is different, and every one has his individual interests and priorities. If things become ugly and nobody wants to give way, you can go on a trip as friends and return without that friendship intact. Sad reality of life, but it happens.
I was fortunate to have went on my Western Australia trip with two friends who would have been happy to give me time to sketch if I wanted to, and they did on at least 2 occasions. They're not sketchers themselves, but they could find other things to do while I was preoccupied. On my part, I didn't want to keep them waiting all the time (and I also wanted to see too many things to stick on one spot for up to an hour each time), so on many occasions I just took photos and did my drawing at night, sometimes long after they had gone to bed. If you want to sketchpack, sometimes you have to do that. Find a time that doesn't inconvenience others, or make prior arrangements. Sketching can wait. Friendships are more valuable. No point sacrificing that just to produce a few drawings.