Skip to main content
GameDev.net gamedev.net
🔒 Locked

asio: Cannot assign requested address

Started by _Kami_ Dec 7, 2011 at 4:13 PM 2 replies 8.9k views
Original Post
_Kami_
_Kami_
Hi,

I have a program that simulates clients and datatraffic (connect to echo server, send 1k chunks 10 times, verify that it got back what it sent, reconnect)
The program will only have 100 concurrent connections.
after about 10k connections has been established (100 active, rest have been closed) I get "Cannot assign requested address". (takes maybe 5-10 seconds)

Its obvious im exhausting some sort of resource
> cat /proc/sys/net/ipv4/ip_local_port_range
55000 65535

Even when starting up the program shortly after its been terminated i still get the same problem.

Now, If I make the clients less aggressive, add some sleep, reconnect only 50% of the time, the problem does not happen, and I can go way past 10k connections.

Is this an issue with the echo server and sockets not closing properly?
Or some other system resource i am hitting that just dont want me to spam connections?
www.ageofconan.com
hplus0603
hplus0603

Its obvious im exhausting some sort of resource
> cat /proc/sys/net/ipv4/ip_local_port_range
55000 65535


Yes, this is the resource you're exhausting.
The TCP protocol identifies a connection by the four-tuple (source IP, source port, destination IP, destination port)
To avoid late IP packets from an old connection confusing a newly established connection to the same destination host/port, source ports are held in "limbo" for up to 2 minutes before they can get re-used. This is known as the TIME_WAIT state.

Linux is somewhat dumb in this regard -- by default it will "hold back" a port to *any* destination if it's been used for *any* destination -- it really only needs to do this for a particular destination.
Similarly, it only really needs to do it per interface / IP address, so creating more local interfaces with IP addresses, and cycling through which interface you send on, should allow you to greatly exceed the limit of number of possible connections to any particular destination IP/port range.

There are a few things you can do:

1) Substantially increase the ip_local_port_range -- say, from 2000 to 65535
2) Turn on SO_REUSEADDR on sockets before connecting or binding them
3) Perhaps also manually binding sockets to local ports before connecting them to a remote destination, in a cyclic fashion
4) Also, manually bind the outgoing socket to particular interfaces, using multiple local "virtual" interfaces to have many local IP source addresses to choose from
enum Bool { True, False, FileNotFound };
_Kami_
_Kami_
thanks for the tips

I do set

m_Socket.set_option(boost::asio::ip::tcp::no_delay(true));
m_Socket.set_option(boost::asio::socket_base::reuse_address(true));

But only after the socket has been connected. Is this the incorrect way to do it? Should I do it before the socket is connected?
www.ageofconan.com
hplus0603
hplus0603

thanks for the tips

I do set

m_Socket.set_option(boost::asio::ip::tcp::no_delay(true));
m_Socket.set_option(boost::asio::socket_base::reuse_address(true));

But only after the socket has been connected. Is this the incorrect way to do it? Should I do it before the socket is connected?


You need to set re-use address before binding it, or it won't have any effect. Connect will implicitly bind the socket if you haven't already bound it yourself.
enum Bool { True, False, FileNotFound };

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.