I would maybe advise a size up and taking it to an alteration tailor who can take it in without removing material and then you can take it back and have it let out as and when you need to. Another option if you can afford it is made to measure where you can possibly request some material built in to allow it to be let out.
I’ve been really into classic mens fashion and tailoring for about 6 years but there’s still a lot I don’t know so while I’m pretty sure this is what my betters would advise, take it with a grain of salt.
You’re very welcome. Buying up a size can be good for awkward builds too. A good friend of mine is about 6′ 2″ and very skinny with long limbs. He often buys up a size and has it altered. Fortunately for him he discovered that the place he buys from actually has a tailor on staff for alterations.