I just told my neighbour to get a life, am I in the wrong and how can I deal with a difficult neighbour?

You are absolutely in the right, sadly what you're having to deal with is a really primitive, maladapted territorial behaviour, he really believes other people's business and property is his own, and he's somehow defending it. It's ultimately about control (which is why you must resist it), and making you think that they're watching everything, which is a horribly insidious psychological pressure to put on anyone (again it's an imposition), so, as others here advise, let a camera do *your* monitoring for you. It's impersonal, and anyone who's minding their own business (and keeping themselves occupied) shouldn't be bothered. Of course, as you say, he's standing outside his house, which he probably thinks is his right, but having experience with this*, I think of such things as an abuse of property rights (which should be a positive thing). The problem is that we commonly say 'it's theirs, they can do what you want', but that shouldn't excuse any behaviour. If we take the busybody's excuse away, and imagine it was public land, no-one could defend standing menacingly outside for hours, and would probably be charged with loitering. Feel free to order whatever you need (many people now run businesses from their homes, remember), and the more he stamps his feet, the more ridiculous he will look to your other neighbours. *I had a neighbour who did exactly this, standing at his gate, smoking his pipe. It was only when he got older and less chatty that people realized what he was actually doing, and, for the last few years of his life he would just stand in the front window for hours at a time, staring out miserably and making every passer-by feel uncomfortable. Busybodying really is a serious, lifelong psychological problem. To end on a lighter note, the funniest thing was when a stray cat adopted a place on his window sill, to stare back at him in fascination!