When music stopped, fast travel was available again
This right here. The game gives absolutely no indication of this and the music doesn't even kick in straight away after dealing with the Order leader. In any other game, this is where the screen would fade to black, disable controls, and force the player to watch a credit roll while the music plays. Instead, it leaves the player baffled and disheartened that they can no longer fast travel, and worse still: it makes players think that their game is bugged, often making them think that reloading to a previous save, sometimes hours before, will fix it. The only indication that fast travel is enabled again is when Eivor comments aloud that he should go talk to Hytham.
Ubisoft, this is bad design, pure and simple. If you're going wrest control away from the player, then you need to clearly indicate that there is a reason for it and that it will be given back once whatever event has completed. This thread is full of examples of what happens when you fail to do this!
TL;DR - After dealing with the Order leader, fast travel will be disabled. Start walking back across the marsh, away from the village. Credit-roll-esque music will start playing as you go about your business. Wait a few more minutes until the music stops. Eivor will comment about talking to Hytham at your settlement. That is when fast travel will be re-enabled.