I use Deluge to do something similar - deluge-daemon runs on my server and the client (GTK-based) runs on all my computers and acts like any other graphical torrent client. There are a few for rtorrent as well, here's one written in Java. I can't seem to find a Qt or GTK-based one at the moment though, which would be ideal.