Staying close to the ship you have one of the biggest tourist attractions in Kent right above you! The word Iconic gets a bit over used but the views of Dover Castle on top of the White Cliffs probably can claim that description!
It has two thousand years of history as the first line of defence for England so it really is the best thing to visit when you’re in Dover and you could easily spend all day there. The port shuttle buses run there direct or you could get a local bus or taxi. If you want to walk one way I suggest you walk down!