[network] UDP Server Listen on 0.0.0.0
Abstract
- UDP is connectionless protocol, there is no way to know real source address at protocol level
- Only way to get real source address is Control Message or BPF which work at socket level
- Soultion for listening on
0.0.0.0: Creating separate udp socket for all avaliable addresses or UsingControl Messagesto deliver incoming datagram’s source address or UsingBPFto change the source address to the right address
Instroduction
UDP is connectionless protocol, it won’t create new socket for any incoming datagrams, so, function like LocalAddr() will return incorrect destination address that associated with incoming datagrams, and response datagram will be routed base on system’s default routing rule, which might not match client’s 5-tuple.
Example:
# basic
1. server:
* has two address: 10.1.2.2(primary) and 10.1.2.3(secondary)
* default route is 10.1.2.2 (default outgoing address)
2. client:
* has one address: 10.1.2.5
* strong end system, like linux
3. server start udp server:
* listen on 0.0.0.0:12345
* without any additional socketopt set
# communication
1. client send data to server with udp:
* 10.1.2.5:23456 -> 10.1.2.3:12345
2. server response:
* 10.1.2.2:12345 -> 10.1.2.5:23456
# result
1. client discard server's response:
* (10.1.2.2:12345, 10.1.2.5:23456) not match (0.1.2.5:23456, 10.1.2.3:12345)
* client awaiting server's response until timeout
In Strong End System (like linux,windows,macos …), udp application only accept response datagram that has same 5-tuple, others will be discarded.
There are three ways to solve this problem:
- don’t listen on
0.0.0.0, create separate udp sockets for every address, and handle all socket’s incoming datagrams withselect,epoll,multi-threadingor some other ways - use
control messagesto deliver incoming datagram’s destination address, and routing response daragram (determine send response datagram with which address) - use
BPF sockmap
example code with Control Messages:
// udp listen 0.0.0.0
func udpServer(network string, addr string) error {
laddr, err := net.ResolveUDPAddr(network, addr)
if err != nil {
return err
}
udpConn, err := net.ListenUDP(network, laddr)
if err != nil {
return err
}
fmt.Printf("[+] listen on %s://%s\n", network, addr)
sysconn, err := udpConn.SyscallConn()
if err != nil {
return err
}
// credit https://github.com/cloudflare/tubular/blob/main/example/main.go
var rerr error
err = sysconn.Control(func(fd uintptr) {
if network != "udp6" {
rerr = unix.SetsockoptInt(int(fd), unix.SOL_IP, unix.IP_RECVORIGDSTADDR, 1)
if rerr != nil {
return
}
}
if network != "udp4" {
rerr = unix.SetsockoptInt(int(fd), unix.SOL_IPV6, unix.IPV6_RECVORIGDSTADDR, 1)
if rerr != nil {
return
}
rerr = unix.SetsockoptInt(int(fd), unix.SOL_IPV6, unix.IPV6_FREEBIND, 1)
}
})
if err != nil {
return err
}
if rerr != nil {
return rerr
}
buf := make([]byte, 512)
oob := make([]byte, unix.CmsgSpace(unix.SizeofSockaddrInet6))
for {
n, oobn, _, remote, err := udpConn.ReadMsgUDP(buf, oob)
fmt.Printf("[+] read %d byte, from %s, error %v\n", n, remote, err)
if err != nil {
return err
}
// oob contains socket control messages which we need to parse.
scms, err := unix.ParseSocketControlMessage(oob[:oobn])
if err != nil {
return err
}
// retrieve the destination address from the SCM.
sa, err := unix.ParseOrigDstAddr(&scms[0])
if err != nil {
return err
}
dAddr := ""
// encode the destination address into a cmsg.
var info []byte
switch v := sa.(type) {
case *unix.SockaddrInet4:
info = unix.PktInfo4(&unix.Inet4Pktinfo{
Spec_dst: v.Addr,
})
dAddr = net.IP(v.Addr[:]).String()
case *unix.SockaddrInet6:
info = unix.PktInfo6(&unix.Inet6Pktinfo{
Addr: v.Addr,
})
dAddr = net.IP(v.Addr[:]).String()
}
fmt.Printf("[+] datagram recv: %s -> %s\n", remote, dAddr)
// reply from the original destination address.
n, _, err = udpConn.WriteMsgUDP(append([]byte("[+] echo: "), buf[:n]...), info, remote)
if err != nil {
return err
}
fmt.Printf("[+] write %d bytes to %s\n", n, remote)
}
//return err
}
FAQ
Q. can we solve this problem with policy routing or nf_conntrack ?
A: No (I have tried and failed).
Q: can we handle ipv4 and ipv6 datagrams with one udp socket ?
A: Yes