Bjorn, when assigned to your crew, will stay there until you have to reload for any reason. Once reloaded he is no longer in your crew and you have to reassign him.Here's a Video of it happen.
I have to say i started with 1 antler after about 3 kills however I'm trying to get the 3rd and im on my 60th reindeer at this point. Even the deer hooves stopped spawning. Its definitely bugged on the drop rate.