Subject: Bug#1012: Session reconnect doesn't work (x2gobroker)

From: Stefan Baur <[email-masked]>
Date: Tue, 22 Mar 2016 10:42:03 +0100

package: x2gobroker
version: 0.0.3.0

Situation:      two identical nodes, only difference is DNS name + IP
Desired result: load-balanced X2Go systems, where a suspended session
                can be resumed

Actual outcome: Each connect starts a new session, suspended sessions
                are left dangling forever

Questions:
1) Is x2gobroker-daemon needed at all for ssh-only connections to the
   broker?
2) Did I miss any obvious steps?
3) What would be the suggested path to debug this?

A full typescript recording of the installation process is available,
but as no command returned any error messages, let me shorten it down to
the commands that were executed:

# commands executed on both first and second node
apt-get install x2gobroker -y
cp /etc/x2go/x2gobroker.conf /etc/x2go/x2gobroker.conf.orig
vi /etc/x2go/x2gobroker.conf # see diff below
apt-get install x2gobroker-agent -y
cp /etc/x2go/broker/x2gobroker-sessionprofiles.conf
/etc/x2go/broker/x2gobroker-sessionprofiles.conf.orig
vi /etc/x2go/broker/x2gobroker-sessionprofiles.conf # see diff below
x2gobroker-keygen
cp /var/lib/x2gobroker/.ssh/id_rsa.pub /tmp/
cd /tmp/
vi id_rsa.pub # added a blank and the server name to end of file
python -m SimpleHTTPServer 8081 # run temporary web server so second
node can fetch the file

# once both web servers were up, the following commands were executed
# on BOTH nodes:
x2gobroker-pubkeyauthorizer -t http://firstnode:8081/id_rsa.pub
x2gobroker-pubkeyauthorizer -t http://secondnode:8081/id_rsa.pub

# Sadly, no working session reconnect with these command line
# parameters - it always starts a new session:
x2goclient
--broker-url=ssh://accountwithapublickey@firstnode:22/usr/bin/x2gobroker
--broker-autologin
# This is using x2goclient-4.0.5.0-2015.07.31

# taking a closer look at
# http://wiki.x2go.org/doku.php/doc:installation:x2gobroker:
# maybe x2gobroker-daemon is missing? So ...

apt-get install x2gobroker-daemon -y # this also pulls in
x2gobroker-authservice

# Sadly, no change, still no working session reconnect :-(

# ---------------------------------------------------------

diff -u /etc/x2go/x2gobroker.conf.orig /etc/x2go/x2gobroker.conf
--- /etc/x2go/x2gobroker.conf.orig      2016-03-19 18:39:02.034407506 +0100
+++ /etc/x2go/x2gobroker.conf   2016-03-19 19:56:05.781729565 +0100
@@ -241,6 +241,7 @@
 # The agent query mode can be configured on a per-broker-backend basis, the
 # below value is the default.
 #default-agent-query-mode=NONE
+default-agent-query-mode=SSH

 # Probe SSH port of X2Go Servers (availability check)
 #
@@ -254,7 +255,7 @@
 # Per default, we set this to "true" here. The portscan feature can be
 # deactivated on a per-session-profile basis (use:
broker-portscan-x2goservers =
 # false in the session profile configuration).
-#default-portscan-x2goservers = true
+default-portscan-x2goservers = false

 # Use load checker for querying X2Go Servers' loads in regular intervals
 #
@@ -294,13 +295,13 @@
 #   o the session profile does not block queries to the load checker daemon
 #     on a per profile basis
 #
-#default-use-load-checker = false
+default-use-load-checker = true

 # If the x2gobroker-loadchecker daemon gets used, define here how
 # many seconds to sleep between cycles of querying system load from the
 # associated X2Go Servers.
 #
-#load-checker-intervals = 300
+load-checker-intervals = 300

 ###
@@ -345,9 +346,8 @@
 #desktop-shell = KDE

 [broker_inifile]
-#enable = true
-#session-profiles = /etc/x2go/broker/x2gobroker-sessionprofiles.conf
-#use-load-checker = false
+enable = true
+session-profiles = /etc/x2go/broker/x2gobroker-sessionprofiles.conf

 #[broker_ldap] -> MUSIC OF THE FUTURE
 #enable = false

# ---------------------------------------------------------

# ---------------------------------------------------------

diff -u /etc/x2go/broker/x2gobroker-sessionprofiles.conf.orig
/etc/x2go/broker/x2gobroker-sessionprofiles.conf
--- /etc/x2go/broker/x2gobroker-sessionprofiles.conf.orig
2016-03-19 19:19:33.995277777 +0100
+++ /etc/x2go/broker/x2gobroker-sessionprofiles.conf    2016-03-19
19:58:45.407309387 +0100
@@ -63,146 +63,12 @@
 directrdp=false
 user=BROKER_USER

-[localhost-kde]
-name=KDE - localhost
-host=localhost
-command=KDE
+[GloveBox]
+host=firstnode (firstnodeip), secondnode (secondnodeip)
+name=GloveBox
+published=true
 usebrokerpass=true
-
-[localhost-mate]
-name=MATE - localhost
-host=localhost
-command=MATE
-usebrokerpass=true
-
-[localhost-shadow]
-name=SHADOW - localhost
-# don't even try load-balancing here... it makes not sense and won't
work (first given host will be used!)
-host=localhost
-command=SHADOW
-usebrokerpass=true
-
-### EXAMPLES: Below you find some config examples. Adapt them to your
needs or
-### simply write your own session profiles and remove the examples below.
-
-##
-## EXAMPLE: pool-A (staff servers)
-##
-## The pool-A contains three X2Go Servers (server-A, server-B and
server-C).
-
-## The staff of our example institute falls into two groups of users:
-## gnome-users and kde-users.
-## The gnome-users log into server-A or server-B, depending on their client
-## subnet (IP configuration of the client).
-## The kde-users login to server-C (server-C can be reached from the whole
-## intranet).
-##
-## The client IP based split-up of the GNOME users allows some
primitive load
-## balancing.
-##
-## If staff people are members of both groups (kde-users, gnome-users) both
-## session profiles will be shown in X2Go Client.
-##
-
-#[pool-A-server-A]
-#user=
-#host=server-a.pool-a.domain.local
-#name=GNOME - pool-A (srv-A)
-#command=GNOME
-#rootless=false
-#acl-groups-allow=gnome-users,admins
-#acl-groups-deny=ALL
-#acl-clients-deny=ALL
-#acl-clients-allow=10.1.0.0/16
-#acl-any-order=deny-allow
-#broker-session-autologin=true
-
-#[pool-A-server-B]
-#user=
-#host=server-b.pool-a.domain.local
-#name=GNOME - pool-A (srv-B)
-#command=GNOME
-#rootless=false
-#acl-groups-allow=gnome-users,admins
-#acl-groups-deny=ALL
-#acl-clients-deny=ALL
-#acl-clients-allow=10.2.0.0/16
-#acl-any-order=deny-allow
-#broker-session-autologin=true
-
-#[pool-A-server-C]
-#user=
-#host=server-c.pool-a.domain.local
-#name=KDE - pool-A (srv-C)
-#command=KDE
-#rootless=false
-#acl-groups-allow=kde-users,admins
-#acl-groups-deny=ALL
-#acl-any-order=deny-allow
-#broker-session-autologin=true
-
-##
-## EXAMPLE: pool-B (e.g. webserver in the DMZ or on the internet)
-##
-## The pool-B is a single X2Go Server (server-D) that is
-## hosted externally. The server-D has an official internet IP.
-##
-## The session profile for server-D shall be provided to the
-## admins group only.
-##
-## Furthermore, the session profile for server-D shall only get
-## offered to a member of the admins group if the admin is sitting
-## in front of one of the admin client machines.
-##
-
-#[pool-B-server-D-LXDE]
-#user=
-#host=server-d (server-d.domain.internet)
-#name=LXDE - srv-D
-#command=LXDE
-#rootless=false
-#acl-groups-allow=admins
-#acl-groups-deny=ALL
-## make sure hostnames in client ACLs are resolvable via libnss!!!
-#acl-clients-deny=ALL
-#acl-clients-allow=admin-machine1.domain.local,
admin-machine2.domain.local, admin-machine3.domain.local
-#acl-any-order=deny-allow
-
-##
-## EXAMPLE: pool-C (REAL LOAD BALANCING!!!)
-##
-## The pool-C is a server pool for students. Our example institute
-## knows 200-300 students and has to offer working places for
-## every student.
-##
-## The resource limits on these servers are pretty strict, so staff members
-## normally stay away from these machines, anyway. Only two test account
-## get this session profile into their X2Go Clients.
-##
-## The pool-C contains 6 X2Go Servers that serve all students users
together
-## as a load balance server farm. The servers' hostnames are s-E1,
s-E2, ...
-## (as found in /etc/hostname). The hosts, however, are not configured
in DNS
-## so we give their IPs explicitly (also works for IPv6).
-##
-## Make sure to install x2gobroker-agent on all these 6 X2Go Servers.
Also make
-## sure to once run the script x2gobroker-keygen on the broker host and
once
-## the script x2gobroker-pubkeyauthorizer per X2Go Server.
-##
-## All 6 X2Go Servers have to be configured to use the PostgreSQL X2Go
session
-## DB backend.
-##
-
-#[pool-C-XFCE]
-#user=
-#host=s-E1 (10.0.2.11),s-E2 (10.0.2.12),s-E3 (10.0.2.13),s-E4
(10.0.2.14),s-E5 (10.0.2.15)
-#name=XFCE - pool-C
-#command=XFCE
-#rootless=false
-#acl-users-allow=testuser-A,testuser-B
-#acl-users-deny=ALL
-#acl-groups-allow=students,admins
-#acl-groups-deny=ALL
-#acl-any-order=deny-allow
 # this server pool has a special broker setup for SSH authorized_keys
-#broker-session-autologin=true
-#broker-authorized-keys=/var/lib/x2gobroker/ssh/%u/authorized_keys
+broker-session-autologin=true
+broker-authorized-keys=/etc/ssh/authorized_keys.d/%u

# ---------------------------------------------------------

# ---------------------------------------------------------

dpkg -l|grep x2go
ii  cups-x2go
3.0.1.3-0x2go1+git20160127.135+jessie.main.1    all          Virtual
X2Go printer for CUPS
ii  libnx-x11-6:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
client-side library
ii  libnx-xcomposite1:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
Composite extension library
ii  libnx-xdamage1:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
damaged region extension library
ii  libnx-xdmcp6:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
Display Manager Control Protocol library
ii  libnx-xext6:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
miscellaneous extension library
ii  libnx-xfixes3:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
miscellaneous 'fixes' extension library
ii  libnx-xinerama1:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
Xinerama extension library
ii  libnx-xpm4:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
pixmap library
ii  libnx-xrandr2:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
RandR extension library
ii  libnx-xrender1:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
Rendering Extension client library
ii  libnx-xtst6:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
Testing -- Record extension library
ii  libxcomp3:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
compression library
ii  libxcompext3:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
protocol compression extensions library
ii  libxcompshad3:i386
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         nx-X11
shadowing library
ii  nx-x11-common
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 all          nx-X11
(common files)
ii  nxagent
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 i386         NX agent
ii  python-x2gobroker
0.0.3.0-0x2go1+git20160126.999+jessie.main.1    all          X2Go
Session Broker (Python modules)
ii  x2go-keyring
2012.07.23~jessie~main~17~build1                all          GnuPG keys
of all X2Go developers and the X2Go archive
ii  x2goagent
2:3.5.0.32-0x2go1+git20160126.734+jessie.main.1 all          X2Go agent
ii  x2gobroker
0.0.3.0-0x2go1+git20160126.999+jessie.main.1    all          X2Go
Session Broker (executable)
ii  x2gobroker-agent
0.0.3.0-0x2go1+git20160126.999+jessie.main.1    i386         X2Go
Session Broker (remote agent)
ii  x2gobroker-authservice
0.0.3.0-0x2go1+git20160126.999+jessie.main.1    all          X2Go
Session Broker (PAM authentication service)
ii  x2gobroker-daemon
0.0.3.0-0x2go1+git20160126.999+jessie.main.1    all          X2Go
Session Broker (standalone daemon)
ii  x2goserver
4.0.1.19-0x2go2+git20160126.1064+jessie.main.1  i386         X2Go server
daemon scripts
ii  x2goserver-extensions
4.0.1.19-0x2go2+git20160126.1064+jessie.main.1  all          X2Go server
daemon scripts (extensions)
ii  x2goserver-printing
4.0.1.19-0x2go2+git20160126.1064+jessie.main.1  all          X2Go server
daemon scripts (printing)
ii  x2goserver-xsession
4.0.1.19-0x2go2+git20160126.1064+jessie.main.1  all          X2Go server
daemon scripts (Xsession runner)

# ---------------------------------------------------------

cat /etc/apt/sources.list
# Security Updates
deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free

# Backports
deb http://ftp.debian.org/debian/ jessie-backports main contrib non-free

# ClamAV etc.
deb http://ftp.debian.org/debian jessie-updates main contrib non-free
deb-src http://ftp.debian.org/debian jessie-updates main contrib non-free

# Stable
deb http://ftp.de.debian.org/debian/ jessie main contrib non-free
deb-src http://ftp.de.debian.org/debian/ jessie main contrib non-free

# X2go Repository
deb http://packages.x2go.org/debian jessie main
# X2go Repository (sources)
deb-src http://packages.x2go.org/debian jessie main

# ---------------------------------------------------------

# ---------------------------------------------------------

cat /etc/debian_version
8.3

# ---------------------------------------------------------

Kind Regards,
Stefan Baur

-- 
BAUR-ITCS UG (haftungsbeschränkt)
Geschäftsführer: Stefan Baur
Eichenäckerweg 10, 89081 Ulm | Registergericht Ulm, HRB 724364
Fon/Fax 0731 40 34 66-36/-35 | USt-IdNr.: DE268653243

---

From: Stefan Baur <[email-masked]>
Date: Tue, 22 Mar 2016 11:05:48 +0100

Wait ... is the step regarding the Postgres DB listed at

http://wiki.x2go.org/doku.php/wiki:advanced:x2gobroker:loadbalancing

and

http://wiki.x2go.org/doku.php/wiki:advanced:multi-node:x2goserver-pgsql

required even for the python broker?

If so:
a) Would it be possible to install the database server on one of the
   X2Go server nodes?
b) How can it be avoided that this database server becomes a single
   point of failure?  Does Postgres support replication in a multi-
   master way, so the database could reside on each node and they'd
   sync their state?

-Stefan

---

From: Mike Gabriel <[email-masked]>
Date: Tue, 22 Mar 2016 10:58:34 +0000

Control: tags -1 not-a-bug
Control: severity -1 wishlist

Hi Stefan,

> required even for the python broker?

Yes and no. If you don't have a multi-server farm (identical servers  
for load-balancing), then you don't need the PgSQL Setup at all.

> a) Would it be possible to install the database server on one of the

The database can be installed on any machine that is reachable from  
the X2Go Servers.

>    master way, so the database could reside on each node and they'd

Big DB implementations like PgSQL or MariaDB surely support clustering  
and replication. Never done that, but it is a non-X2Go task.

> -Stefan

Apart from the bug title, you don't provide a description about what  
your issue really is. Thus, tagging this bug with "not-a-bug".

Mike

-- 

DAS-NETZWERKTEAM
mike gabriel, herweg 7, 24357 fleckeby
fon: +49 (1520) 1976 148

GnuPG Key ID 0x25771B31
mail: [email-masked], http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/mailxchange/kronolith/fb.php?u=m.gabriel%40das-netzwerkteam.de

---

From: Stefan Baur <[email-masked]>
Date: Tue, 22 Mar 2016 12:00:37 +0100

> your issue really is. Thus, tagging this bug with "not-a-bug".

Could it be that you only read the second e-mail belonging to this bug?
The first one was rather lengthy and detailed.  Should you have missed
that, please undo the not-a-bug and wishlist tagging.

Kind Regards,
Stefan

---

From: Mike Gabriel <[email-masked]>
Date: Tue, 22 Mar 2016 11:14:57 +0000

Hi Stefan,

ah, sorry, missed your initial post around #1012.

> 1) Is x2gobroker-daemon needed at all for ssh-only connections to the

No. But I recommend starting with x2gobroker-daemon, because it allows  
much easier debugging.

> 2) Did I miss any obvious steps?

What were your steps? (ah... ok... posted below...)

> 3) What would be the suggested path to debug this?

Use http brokerage for debugging. Run

   x2gobroker-daemon-debug

as root on the cmdline and post suspicious messages here.

Where do you actually have the X2Go Broker installed? On both X2Go  
Servers? This is a non-recommended setup.

The recommended setup is:

   on broker machine, several X2Go Servers

or

   two broker machines (with DNS round robin), several X2Go Servers

> A full typescript ecording of the installation process is available,

> # This is using x2goclient-4.0.5.0-2015.07.31

You could play with the x2gobroker-testagent script and investigate  
the calls to the X2Go Server side x2gobroker-agent.

You could also check if you can use the broker-autologin feature on a  
per session basis. x2gobroker-daemon-debug will be your friend...

> # maybe x2gobroker-daemon is missing? So ...

It depends on what brokerage you want to use: x2gobroker-ssh -> SSH  
brokerage, x2gobroker-daemon or x2gobroker-wsgi -> http brokerage.

> apt-get install x2gobroker-daemon -y # this also pulls in

Yes. It is required for the pam authmech inside the broker daemon. If  
you authenticate directly against LDAP or HTTPS, then  
x2gobroker-authservice is not needed.

> # Sadly, no change, still no working session reconnect :-(

Do you have x2gobroker-loadchecker install on the broker server?  
Shouldn't cause your failure, but still...

> +load-checker-intervals = 300

Please consider contracting me, on such a project. The X2Go Broker's  
code says it all. If reading the code is problematic, consider  
contracting me for supervision.

Mike
-- 

DAS-NETZWERKTEAM

GnuPG Key ID 0x25771B31

freeBusy:

---

From: Mike Gabriel <[email-masked]>
Date: Tue, 22 Mar 2016 11:16:01 +0000

Control: severity -1 normal
Control: tags -1 - not-a-bug

> that, please undo the not-a-bug and wishlist tagging.

Yeah, indeed that was the case.

Sorry,
Mike
-- 

DAS-NETZWERKTEAM

GnuPG Key ID 0x25771B31

freeBusy:

---

From: Stefan Baur <[email-masked]>
Date: Wed, 23 Mar 2016 09:05:40 +0100

>   two broker machines (with DNS round robin), several X2Go Servers

What's the reason for this?

Our idea was to install the broker on all X2Go Servers, and have one
Round-Robin-DNS entry for the broker connection, as well as separate
names for the servers themselves.

broker.example.com -> 192.168.0.10, 192.168.0.20 #RRDNS
primarynode.example.com -> 192.168.0.10
secondarynode.example.com -> 192.168.0.20

Is this still a bad idea, and if so, why?

Kind Regards,
Stefan Baur

---

From: Mike Gabriel <[email-masked]>
Date: Wed, 23 Mar 2016 09:42:28 +0000

Hi Stefan,

> Is this still a bad idea, and if so, why?

If you setup the complete broker <-> broker-agent functionality, the  
broker becomes quite powerful. The software design should be safe  
regarding privilege handling.

However, I personally prefer to have the broker on a machine where  
users won't get a login shell. It is just a gut feeling. In theory, it  
should be safe having the broker on X2Go Servers. But still...

Mike

-- 

DAS-NETZWERKTEAM

GnuPG Key ID 0x25771B31

freeBusy:

---

From: Melvin Vermeeren <[email-masked]>
Date: Mon, 14 Oct 2019 17:21:27 +0200

I have recently installed a complete x2go setup on a single server with the 
standalone http broker and am encountering the same problem. For me too it 
appears like not checking for sessions to resume in x2goclient is intentional, 
as message #80 by Oleg indicates.

Now I read on the wiki that if the broker is used the broker is responsible 
for session resumption, which makes sense to me. The x2goclient then just does 
whatever the broker tells it to.

This results in the following question, which I haven't been able to find an 
answer for: How to configure the reference broker to resume sessions 
automatically instead of always spawning a new session?

I have only a single session type configured in the inifile, and the intention 
is that when a client connects, selecting this selection the broker should 
resume an existing session of the same type if one exists. At no point should 
there be more than one sessions of the same type.

If I add the SHADOW session type to the inifile I can actually see the 
suspended sessions and interact with them properly. But even if there are 
sessions suspended, selecting the KDE session results in a new session, 
always.

Also seems that x2gobroker.conf default-session-autologin=true does not work 
for SHADOW session type, it uses my regular SSH key. This is a different issue 
though which will post elsewhere shortly.

Thanks.

---

From: Melvin Vermeeren <[email-masked]>
Date: Tue, 15 Oct 2019 20:08:28 +0200

Found part of the issue. To secure the services I use OpenVPN, even for SSH. 
For this example the "public hostname" is example.com. When you connect to the 
OpenVPN service example.com you can now also reach example.com through the 
secure VPN with IP 10.8.0.1.

The broker only listens on 10.8.0.1, so this means you need both a device 
holding the OpenVPN keys and username+password in the broker to log in. This 
way external, untrusted devices cannot try to log in at all. SSH daemon 
requires public keys, password auth is disabled.

In session profiles inifile, previously the value was host=10.8.0.1. This way 
the SSH traffic gets tunnelled through the VPN too. Now that this is changed 
to host=example.com, I can see status "(suspended)" or "(running)" in 
x2goclient, depending on the server state.

X2Go always uses the system's real $HOST to determine the session hostname. So 
even if inifile host=10.8.0.1 x2golistsessions will lists it as:
12345|foobar-50-....|50|example.com|...

Even though suspended/running status now works clicking on the suspended 
session in x2goclient still starts up a new session. The broker log:
> base_broker.X2GoBroker.select_session(): no X2Go Server could be contacted, 
> session startup will fail, tried these hosts: ['example.com']

The fix was to use the host syntax documented in load balancing mode:
host=example.com (10.8.0.1)

With this change everything works perfectly. I think the documentation, 
perhaps in the inifile itself, should add some IMPORTANT note regarding this. 
If the host field is not properly configured many things don't work properly.

tl;dr The host name MUST match whatever is listed in x2golistsessions and the 
host IP/target MUST match whatever the broker is bound to.