Pass struct sockaddr to ngtcp2 so that it knows how to deal with
address change in more detailed manner (e.g., just compare IP address
without port).