Hi Mark,
I ran it the same way as you basically when I started working on wfview. I used x11vnc to my home linux box, also with fldigi. (Forget raw X11 forwarding, it draws each pixel in the waterfall very slowly.)
With the wfview native server-client connection, you can do voice QSOs too.
I do connect to my other computers around the house when I test wfview. In fact, my rigs are in the garage, so I often connect over to them and see how things are.
–E
de W6EL