I am working on an ant simulation for my major project and I have encountered a problem. How would I implement a solution for helping a single agent find its neighbouring agents?

(e.g. If my agent is standing at (x,y), find all neighbouring agents within 1 unit of distance from my agents origin).

I would like an algorithmic approach so that I could apply the concept in any language. Please note that my current solution uses a list which contains all the agents in my world. For the concerned agent, I iterate through this list and determine out of all the agents, which ones are within the distance threshold. This however, is not very effective when the number of agents increases to extreme levels and I am convinced it is not very good practice to use this method.

I am simply looking for ideas and advice, if anyone can offer.

What you're looking for is essentially 'spatial partitioning': http://gameprogrammingpatterns.com/spatial-partition.html

The simplest standard approach is a coarse grid overlaid on the world which keeps track of which agents are in which cells. Other approaches include tree structures like quadtrees and octtrees.

Another surprisingly effective approach when agents are similarly-sized is to simply keep the agent list sorted along one of the directional axes. This allows you to only have to iterate a small number of times either side of the agent being queried to find all the other agents that could possibly be nearby.

(Also I edited your thread title to be less shouty and more relevant.)

Thank you my friend, literally panicking here!