Subject: select_session offers offline servers to X2Go Client

From: Sergey Savko <[email-masked]>
Date: Fri, 5 Dec 2014 20:14:51 +0400 (MSK)

This patch work after patch from http://bugs.x2go.org/cgi-bin/bugreport.cgi?bug=686

-- 
С уважением,
Сергей Савко,
начальник IT отдела.
+7-931-361-04-02

From 5f8a636aa7029baf5ff5df779538d22062b3ee74 Mon Sep 17 00:00:00 2001
From: Sergey Savko <[email-masked]>
Date: Fri, 5 Dec 2014 19:09:34 +0300
Subject: [PATCH] Gives a coefficient of 1 for the server, if the server is
 down.

---
 x2gobroker/agent.py               |  5 +++++
 x2gobroker/brokers/base_broker.py | 38 +++++++++++++++++++++++++++-----------
 2 files changed, 32 insertions(+), 11 deletions(-)

diff --git a/x2gobroker/agent.py b/x2gobroker/agent.py
index e4bef4d..ba9662b 100644
--- a/x2gobroker/agent.py
+++ b/x2gobroker/agent.py
@@ -172,6 +172,8 @@ def _call_remote_broker_agent(username, task, cmdline_args=[], remote_agent=None
         '{username}'.format(username=username),
         '{task}'.format(task=task),
     ]
+    
+    
 
     for cmdline_arg in cmdline_args:
         cmd_line.append('"{arg}"'.format(arg=cmdline_arg))
@@ -183,6 +185,9 @@ def _call_remote_broker_agent(username, task, cmdline_args=[], remote_agent=None
     else:
         remote_port = 22
 
+    if icmp_ping(remote_hostname) is False:
+	return (False, [])
+
     # now, connect and use paramiko Client to negotiate SSH2 across the connection
     try:
         client = paramiko.SSHClient()
diff --git a/x2gobroker/brokers/base_broker.py b/x2gobroker/brokers/base_broker.py
index fc0f44f..270bab7 100644
--- a/x2gobroker/brokers/base_broker.py
+++ b/x2gobroker/brokers/base_broker.py
@@ -1220,19 +1220,32 @@ class X2GoBroker(object):
             busy_servers = None
 	    busy_each_server = dict()
 	    while server_list :
-		remote_agent_server = server_list[-1]
-		remote_agent_port = profile[u'sshport']
-		remote_agent = {u'hostname': remote_agent_server, u'port': remote_agent_port, }
-		try:
-		    success, busy_each_servers = x2gobroker.agent.find_busy_servers(username=username, remote_agent=remote_agent)
-		except x2gobroker.x2gobroker_exceptions.X2GoBrokerAgentException:
-		    pass
-		if busy_servers is None:
-		    busy_servers = busy_each_servers
-		else :
-		    busy_servers.update(busy_each_servers)
+		if x2gobroker.agent.icmp_ping(server_list[-1]):
+		    remote_agent_server = server_list[-1]
+		    remote_agent_port = profile[u'sshport']
+		    remote_agent = {u'hostname': remote_agent_server, u'port': remote_agent_port, }
+		    try:
+			success, busy_each_server = x2gobroker.agent.find_busy_servers(username=username, remote_agent=remote_agent)
+		    except x2gobroker.x2gobroker_exceptions.X2GoBrokerAgentException:
+			pass
+		    if busy_servers is None:
+			busy_servers = busy_each_server
+		    else :
+			busy_servers.update(busy_each_server)
+		else:
+		    busy_each_server.update({ server_list[-1] : 1 })
+		    if busy_servers is None:
+			busy_servers = busy_each_server
+		    else :
+			busy_servers.update(busy_each_server)
 		server_list = server_list[0:-1]
 
+#            try:
+#                success, busy_servers = x2gobroker.agent.find_busy_servers(username=username, remote_agent=remote_agent)
+#		print "server load"
+#            except x2gobroker.x2gobroker_exceptions.X2GoBrokerAgentException:
+#                pass
+
             if busy_servers is not None:
 
                 # if we do not get here, we failed to query a valid agent...
@@ -1271,6 +1284,9 @@ class X2GoBroker(object):
 
                 busy_server_list = [ (load, server) for server, load in busy_servers.items() ]
                 busy_server_list.sort(reverse=True)
+		
+		if busy_server_list[0][0] == 1 :
+		    logger_broker.warning('base_broker.X2GoBroker.select_session(): no broker agent could be contacted, this does not look good. We tried these agent hosts: {agent_hosts}'.format(agent_hosts=unicode(server_list)))
 
                 logger_broker.debug('base_broker.X2GoBroker.select_session(): load balancer analysis: {server_load}'.format(server_load=unicode(busy_server_list)))
 
-- 
1.9.1

---

From: Mike Gabriel <[email-masked]>
Date: Sat, 06 Dec 2014 22:56:05 +0000

Control: clone -1 -2 -3
Control: reassign -2 x2goclient
Control: reassign -3 python-x2go
Control: retitle -1 add exclude-hosts parameter to selectsession task
Control: retitle -2 request another server from broker provided server is down
Control: retitle -3 request another server from broker provided server is down
Control: severity -1 wishlist
Control: severity -2 wishlist
Control: severity -3 wishlist
Control: block -2 by -1
Control: block -3 by -1
Control: tag -1 - patch

Hi Sergey,

On  Fr 05 Dez 2014 17:14:51 CET, Sergey Savko wrote:

> This patch work after patch from  
> http://bugs.x2go.org/cgi-bin/bugreport.cgi?bug=686

After thinking this through a little, I come to the conclusion that  
the broker cannot decide if a machine is down or not.

We have to think very generically. There may be a scenario where the  
broker machine may be on an network segment where it cannot ping/reach  
the X2Go Servers.

The X2Go Clients can reach the X2Go Broker. The broker provides an  
X2Go Server address on the "selectsession" broker task to the X2Go  
Client. Then the X2Go Client should test if that X2Go Server address  
works (via a simple ping6/ping command, machines should always be  
pingable!!!). If the ping fails, X2Go Client should go back to the  
broker and say: hey, that server failed for me, give me another one  
(but not the one you already gave me).

I fear we need to do four things for this bug to get fixed:

   1. extend broker/client communication protocol (second/third/...  
selectsession
      call with a list of hosts that did not work on previous attempts)
   2. extend X2Go Session Broker with an exclude-hosts (or so)  
parameter for the
      selectsession task
   3. Adapt X2Go Client: ping X2Go Server, go back to the broker if  
server is down
      and request another server
   4. Adapt Python X2Go: dito

Regards,
Mike

-- 

DAS-NETZWERKTEAM
mike gabriel, herweg 7, 24357 fleckeby
fon: +49 (1520) 1976 148

GnuPG Key ID 0x25771B31
mail: [email-masked], http://das-netzwerkteam.de

freeBusy:
https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb

---

From: Stefan Baur <[email-masked]>
Date: Sun, 07 Dec 2014 00:17:22 +0100

Am 06.12.2014 um 23:56 schrieb Mike Gabriel:
> Then the X2Go Client should test if that X2Go Server address works
> (via a simple ping6/ping command, machines should always be
> pingable!!!)

Chiming in here:
Even if they aren't pingable - Port 22 (or whichever you've set in the
config as SSH port to be used) must be accepting connections.
You can test that on Linux with

nc -z ip.goes.he.re port_goes_here && echo "is reachable"

... and I'm sure there are ways to do that inside the client code, too.
Check if you can get a TCP handshake going within a set time frame (1
second? 2 seconds? 5 seconds?), then disconnect and proceed depending
on the result.

Actually ... simply lowering the timeout value for the currently
existing code that handles the connection, when called in broker
client mode, might already work.

- -Stefan

- -- 
BAUR-ITCS UG (haftungsbeschränkt)
Geschäftsführer: Stefan Baur
Eichenäckerweg 10, 89081 Ulm | Registergericht Ulm, HRB 724364
Fon/Fax 0731 40 34 66-36/-35 | USt-IdNr.: DE268653243

---

From: Sergey Savko <[email-masked]>
Date: Sun, 7 Dec 2014 03:45:28 +0400 (MSK)

If the server will give the address to which it can not connect, there will be no load balancing works. 
Since the server is connected to receive the coefficient of loading.

----- Исходное сообщение -----
От: "Mike Gabriel" <[email-masked]>
Кому: "Sergey Savko" <[email-masked]>, [email-masked]
Отправленные: Воскресенье, 7 Декабрь 2014 г 1:56:05
Тема: Re: [X2Go-Dev] Bug#684: select_session offers offline servers to X2Go Client

Control: clone -1 -2 -3

Hi Sergey,

Regards,
Mike

-- 

DAS-NETZWERKTEAM

GnuPG Key ID 0x25771B31

freeBusy:

---

From: Mike Gabriel <[email-masked]>
Date: Mon, 08 Dec 2014 07:23:39 +0000

clone #684 -1
retitle -1 select_session offers offline servers to X2Go Client
thanks

Hi Sergey,

> Since the server is connected to receive the coefficient of loading.

Actually, after a third or fourth though: in cases where we use SSH to  
connected from broker server to broker agent, there we can evaluate  
the online status of the X2Go Server. So, in those cases we should  
filter out, if a server is down or not and exclude that server from  
the list of possible X2Go Servers.

Plus, I still think, that X2Go Client should request another machine,  
if the provided server was offline and more than one server is  
configured in the broker's session profile.

So, cloning this bug again...

   #684: for tracking the new feature of re-requesting a server address
   new bug: filter out offline servers on the broker side already

Greets,
Mike

-- 

DAS-NETZWERKTEAM

GnuPG Key ID 0x25771B31

freeBusy: