I accidently started a conversation, and after a while I decided that I wanted to skip to the end and say goodbye. 

I have found no way to do that... Is there a way to end conversation early? 

I don't think so. It would be impolite to do so.

Other than maybe quickly leaving the game or if on PC, killing it in the task manager. Better would be looking at how far back your last save file is and hopefully it's not too far back. I set mine to save every 3 minutes. That or do a save before talking to everyone.