Algorithmic Foundations For Safe And Efficient Reinforcement Learning From Human Feedback