For auth we didn't go the non-blocking route with selectors. We ended up with a thread pool and the pool that does the auth work uses non-blocking IO and spinning with sleeps. The sleeps are 400 millis so you get bad behavior if the auth packet doesn't make it in time. That path needs to be rewritten to use a selector and be non-blocking.
Switch to blocking I/O and schedule a task to close the socket. If successful Auth, you can cancel the task.
Test for Auth timeout