Be very, very careful of how you respond to a neighbor such as you have.
Even though you have not and do not present as a threat to him your neighbour could take it into his head to have an Intervention Order made out against you. He doesn't have to prove anything in the first instance either - it will automatically be granted.
When it goes to court as it must for either confirmation or dismissal as frivolous and unsubstantiated you will still have a permanent record of having an IO issued against you. This can have far reaching effects on your future - think employment etc.
Keep your head down, don't respond and keep a diary of events. This may prove to be a very useful defensive tool later on if matters become ugly.
Good luck!

