Lady Amy had tried to tell her boss that she could not date him or have anything with him but he did not want to listen to her. So she felt that the best thing to do to him was to tease and deny him so that he got it from her physically and practically that she did not want anything to do with him. And that is how she got him to stop being a bother.