True refuge definitely protects you from the lower realms. This is one of the advantages of taking refuge.
Thank you for bringing up this more than important subject: the time of death.
A practitioner should have all the knowledge needed to obtain a rebirth with the wonderful base that allows you to work for liberation and enlightenment, the noble human life.
Because what you say it's true, as a possibility, we need to know what to do to avoid it. The bad karma "coming to fruition at the time of death" is not a rule. A practitioner needs to learn how to prevent it. Our holy system gives us all the tools of Dharma needed. The time of death is not a lottery.
We'll talk more about it, thank you again.