Agree.

I started building this out last year to explore web crypto and webrtc but got sidetracked.

Have you looked into webcrypto? Create a new ecdh keypair for each session for each party, keep the private inextricable in memory, trade public keys over webrtc. This (I think) ensures no-one can evesdrop.

Peer auth could occur normally with ecdsa signatures done OUTSIDE of the browser or whatever.

I don't know if this would be considered a legitimate public/private key exchange, but we now have a feature request for integrated OTP functionality: https://github.com/jeremyckahn/chitchatter/issues/2

Hopefully this will address the need for secure room access!

This is an interesting idea. How do you envision this working, broadly speaking?