On the evolution of harming and recognition in finite panmictic and infinite structured populations