X2Go Bug report logs - #611
X2Go Client for Windows cannot create C:\Users\<username>\ssh\known_hosts file when the username has characters from languages other than the system locale

Full log

Message #60 received at 566@bugs.x2go.org (full text, mbox, reply):

Received: (at 566) by bugs.x2go.org; 9 Sep 2014 13:15:56 +0000
From mikedep333@gmail.com  Tue Sep  9 15:15:53 2014
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
	ymir.das-netzwerkteam.de
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM,
	T_DKIM_INVALID autolearn=ham version=3.3.2
Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178])
	by ymir.das-netzwerkteam.de (Postfix) with ESMTPS id 088C75DEAB
	for <566@bugs.x2go.org>; Tue,  9 Sep 2014 15:15:53 +0200 (CEST)
Received: by mail-wi0-f178.google.com with SMTP id hi2so1158151wib.11
        for <566@bugs.x2go.org>; Tue, 09 Sep 2014 06:15:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        bh=TDMD9DInwGIgjGBdOjuJ1SEHzEqScXdfZ+LlhGVL6co=;
        b=D0I/h3otqjW9n6EzFZsH6j3QdsVQ79v6zt+RH/wtkiKzTcXmozRu9eYHidk2ReGUs9
         U0MSo66lqgmJEuUWrT90p6ci4hF3U10ZOOKF/Z9uiSdd6WTjx7/hpC3F8FO2gnCftBdg
         FSB5LOIAELZ2lOLDac/5sbwVFkssmOsXiBmsFmnYoss1zNGfOdEsDx9XCmBfA0vyX6sV
         G4aDMl3TorNwmTr2+ZZKyqVUEwvs8wdzHydB2ymhsviwfQ3bH4vfQlasaWdFm9U7qhWf
         3rqu2gGpu49TJoom/YRyd0JUoYZsX37W5ahCByn+CZuxKj44Gev2E+937Gg31M/VOAzh
         vOFA==
MIME-Version: 1.0
X-Received: by 10.194.87.41 with SMTP id u9mr3070000wjz.124.1410268552644;
 Tue, 09 Sep 2014 06:15:52 -0700 (PDT)
Received: by 10.180.106.39 with HTTP; Tue, 9 Sep 2014 06:15:52 -0700 (PDT)
In-Reply-To: <20140908085833.Horde.1KVN9LjgQ3GOkz4EX-JZjA1@mail.das-netzwerkteam.de>
References: <000001cfb552$3193a8a0$94baf9e0$@gr>
	<handler.566.B.140775428816658.ack@bugs.x2go.org>
	<000001cfc60a$32831de0$978959a0$@gr>
	<20140901195103.Horde.HOAd34FDzvE7MwwNBCtaAg1@mail.das-netzwerkteam.de>
	<CAMKht8gaM7uupVMhTkPR3pg2jcO5sBRHCbYkMV8kCf0EvKK8sg@mail.gmail.com>
	<CAMKht8jxisag8A9F3yFBSg3BQ+PODLzrayu90S0G8OaQKZXtfQ@mail.gmail.com>
	<000001cfca83$2b11d830$81358890$@edu.gr>
	<CAMKht8hMDouAZr0zh0jQ8HhA_COJOfC-d0BKY4u6hTcCDEoQmQ@mail.gmail.com>
	<20140908085833.Horde.1KVN9LjgQ3GOkz4EX-JZjA1@mail.das-netzwerkteam.de>
Date: Tue, 9 Sep 2014 09:15:52 -0400
Message-ID: <CAMKht8jjsK7nK-CyU6OeDMRBgJHx9SC89YQdWLLGL_4+0yVMvA@mail.gmail.com>
Subject: Re: [X2Go-Dev] Bug#566: X2Go Client for Windows 4.0.2.1 cannot create
 C:\Users\<username>\ssh\known_hosts file when the local Windows account
 username has non-Ascii characters
From: Michael DePaulo <mikedep333@gmail.com>
To: 566@bugs.x2go.org
Cc: George Trakatelis <trakatelis@uom.edu.gr>, Mike Gabriel <mike.gabriel@das-netzwerkteam.de>
Content-Type: multipart/mixed; boundary=089e0102fb4e1e6d790502a1bce9

[Message part 1 (text/plain, inline)]

On Mon, Sep 8, 2014 at 4:58 AM, Mike Gabriel
<mike.gabriel@das-netzwerkteam.de> wrote:
> Hi Mike#2,
>
>
> On  So 07 Sep 2014 15:23:22 CEST, Michael DePaulo wrote:
>
>> On Sun, Sep 7, 2014 at 6:04 AM, George Trakatelis <trakatelis@uom.edu.gr>
>> wrote:
>> [...]
>>>
>>>
>>> a. Installed German keyboard. Hit ';' to produce an accented letter
>>> and -to my surprise- got ö. So I thought ölexandr was the right name to
>>> test for username.
>>>
>>> b. Created user ölexandr and logged in as that user.
>>> The patch did not work, as you mentioned.
>>>
>>> c. As toLocal8Bit() returns the local 8-bit representation of a string,
>>> I changed the system locale for non-unicode programs to German.
>>> Now the patch worked!
>>>
>>> I think the patch solves the problem for the Windows users who use
>>> English
>>> as a foreign language and prefer having (non-Ascii) usernames in their
>>> native language.
>>>
>>> -George
>>>
>>> PS. A better title for the bug would be
>>>
>>> X2Go Client for Windows 4.0.2.1 cannot create
>>> C:\Users\<username>\ssh\known_hosts file when the local Windows account
>>> username has non-Ascii characters
>>
>>
>> Hi Mike#1 and George,
>>
>> 1. What do you think of using toUtf8() instead? I am hoping that will
>> fix the bug regardless of the locale.
>>
>> I tried changing it to that (bug566.utf8.test.patch), but it still did
>> not fix this bug with the greek username on my system with the locale
>> set to English. It did not introduce a regression for my ASCII user
>> account at least.
>>
>> If libssh needs to be recompiled for Unicode, I can do that. I just
>> recompiled it for bug #590.
>>
>> FYI: This is the API we are calling:
>>
>> http://api.libssh.org/stable/group__libssh__session.html#ga7a801b85800baa3f4e16f5b47db0a73d
>
>
> I am getting the suspicion, that this is a libssh issue.
>
> X2Go Client and libssh should communicate via UTF-8 (because it is _the_
> encoding that applications use internally, nowadays) and libssh should
> handle whatever conversion (probably to UTF-16 wide string?) is necessary
> for creating files, being aware of usernames, etc.
>
>> 2. in sshmasterconnection.cpp, we also set "SSH_OPTIONS_USER" . I
>> think that we should also change its value from to.Ascii() to
>> toLocal8Bit() or toUtf8(). bug566.test.v2.patch has this change for
>> toLocal8Bit(), and bug566.utf8.test.patch has it for toUtf8().
>
>
> Same thing. SSH_OPTIONS_USER should be set as UTF-8 and libssh should
> convert to whatever encoding is needed.
>
> Greets,
> Mike
[...]

Hi Mike#1 and George,

1.

I believe the best approach is to try to compile libssh with unicode
support, and pass unicode values to it. There is no guarantee that a
user's username (and home dir path) is in the same language as the
language that is set for non-Unicode programs. After all, the
"Language for non-Unicode programs" is called the "system locale", it
applies to all users on the system.

If we have to for the 4.0.3.0 release of x2goclient, we should fix
this bug for the system locale only, and then fix the bug for all
languages/locales later.

2.

See bug 474 for George's patch for compiling libssh with MinGW 4.8.2.
It worked :)

3.

I made some progress compiling libssh 0.6.3 with Unicode support.

With MinGW 4.4, when I specified the following values in CMAKE, libssh
built successfully, but it did not seem to change anything.
CMAKE_EXE_LINKER_FLAGS:STRING=-municode
CMAKE_MODULE_LINKER_FLAGS:STRING=-municode
CMAKE_SHARED_LINKER_FLAGS:STRING=-municode
CMAKE_STATIC_LINKER_FLAGS:STRING=-municode

With MinGW 4.8.2, when I specify those values, libssh fails to build.
I've attached the output.

This is a positive sign because it implies that MinGW 4.8.2 supports
-municode, whereas MinGW 4.4 did not.

I plan to try to resolve this build failure later tonight.

-Mike#2

[libssh-0.6.3-fails-to-build-mingw4.8.2-and--municode.txt (text/plain, attachment)]

Send a report that this bug log contains spam.