@quixoticgeek My instance shows a "Private mentions" option on the right-hand side that seems to do this job?

@quixoticgeek Webinterface or a apecific app? Some apps allow for this; the webinterface can as well by appending /conversations to your server URL (most recent version).