You could still kinda do it, something like:
- When a new convo opens, hide it.
- If its a one-person convo, show it, finished.
- If theres 3+ people then ask if you want to enter the convo
- If yes, show the convo, if no, close the convo.
Seems too simple to me, so there must be a flaw in it