Without these changes, library was failing to build on both Windows and
FreeBSD.

Pass struct sockaddr to ngtcp2 so that it knows how to deal with
address change in more detailed manner (e.g., just compare IP address
without port).