Curious difference between epoll, poll, kqueue and select

01 April 2008 (updated 04 March 2015)

Suppose you have a brand new nonblocking socket and you want to do something with it (i have a connect in mind - but it's irrelevant anyway). So what is your obvious choice for that since the socket is non-blocking? Use select or poll or whatever ofcourse. Right? NO.

Sadly they don't work the same with unconnected sockets - I wonder why.

select (win32):

>>> import select, socket
>>> s = socket.socket()
>>> select.select([s], [s], [s], 1)
([], [], [])

select (linux):

>>> import select, socket
>>> s = socket.socket()
>>> select.select([s], [s], [s], 1)
([], [], [])

select (freebsd):

>>> import select, socket
>>> s = socket.socket()
>>> select.select([s], [s], [s], 1)
([], [], [])

Pretty inconsistent eh?

poll (linux):

>>> import select, socket
>>> poll = select.poll()
>>> s = socket.socket()
>>> poll.register(s.fileno(), select.POLLIN | select.POLLOUT)
>>> (fd, ev), = poll.poll(1000)
>>> ev & select.POLLIN
0
>>> ev & select.POLLOUT
4
>>> ev & select.POLLERR
0
>>> ev & select.POLLHUP
16

poll (freebsd):

>>> import select, socket
>>> poll = select.poll()
>>> s = socket.socket()
>>> poll.register(s.fileno(), select.POLLIN | select.POLLOUT)
>>> (fd, ev), = poll.poll(1000)
Traceback (most recent call last):
  File "...", line 1, in ...
ValueError: need more than 0 values to unpack

Oh NOES!!!!1 It's not portable. epoll:

>>> import epoll, socket
>>> efd = epoll.epoll_create(16)
>>> s = socket.socket()
>>> epoll.epoll_ctl(efd, epoll.EPOLL_CTL_ADD, s.fileno(), epoll.EPOLLIN | epoll.EPOLLOUT)
0
>>> (ev, fd), = epoll.epoll_wait(efd, 16, 1000)
>>> ev & epoll.EPOLLHUP
16
>>> ev & epoll.EPOLLOUT
4

kqueue:

>>> import kqueue, socket
>>> kq = kqueue.kqueue()
>>> s = socket.socket()
>>> kq.kevent(kqueue.EV_SET(s.fileno(), kqueue.EVFILT_READ | kqueue.EVFILT_WRITE, kqueue.EV_ADD | kqueue.EV_ENABLE))
>>> kq.kevent(None, 16, 1000000000)
[]

I wonder why are these so inconsistent. One would expect come consistent results for multiplexing a simple thing such a unconnected brand new socket, no?If you think that is horrible - well, I do - check out the tricks to check for connection success on non-blocking sockets - and they don't cover win32:

However, win32 has a much nicer api in the so called overlapped io wich is very different: you don't need to check the sockets before making a operation - you just make the socket call with a overlapped and check for for completed calls (via io completion ports - gqcs). Still, you can use some selects variants (WSAAsyncSelect, WSAEventSelect, select) wich are just as bad. ** **

Unfortunately python has a incomplete binding, pywin32 (not that bad - only ConnectEx and TransmitFile missing) for that and it's actually fairly easy to crash your python interpreter with this.

This entry was tagged as python sockets