Most likely due to security concerns, the default configuration of the X11 server is to only listen on local Unix sockets.
Changing this is possible by altering an item in the preferences system, named "nolisten_tcp".
And yes, you can use Interface Builder to add this item in the preference pane so you don't have to fiddle around with the "defaults" command each time you want to change it.