getHostKey returns an empty string

[Hjdskes]

Hello,

I am running into the case where getHostKey always returns an empty string. This happens on two very distinct SFTP servers, one of which is an AWS Transfer Family SFTP server; the other is an old mainframe-like SFTP server.

In both cases, the host key type returned is 1, which I assume to be TYPE_PLAIN as per

libssh2-hs/libssh2/src/Network/SSH/Client/LibSSH2/Foreign.chs

Line 114 in 58f5a34

kht2int TYPE_PLAIN = 1

Below is the code I use to initialise my SFTP session:

withSftpUser :: forall m a
  .  MonadLog m
  => MonadMetrics m
  => MonadThrow m
  => MonadUnliftIO m
  => SftpConfigF Text
  -> (Sftp -> m a)
  -> m a
withSftpUser config action = withSession (host config) (port config) $ \session -> do
  checkHost session (host config) (port config) (knownHosts config) >>= \case
    MISMATCH -> logError "Host key mismatch" >> throwM ERROR_KNOWN_HOSTS
    FAILURE -> logError "Failed to verify host key" >> throwM ERROR_KNOWN_HOSTS
    NOTFOUND -> logError "Host key not found" >> throwM ERROR_KNOWN_HOSTS
    MATCH -> logInfo "Verified host key"
  case credentials config of
    MkUserPass (UserPass (Username username) password) ->
      liftIO $ usernamePasswordAuth session (Text.unpack username) (Text.unpack password)
  withSftpSession session action

withSftpSession :: forall m a
  .  MonadUnliftIO m
  => Session
  -> (Sftp -> m a)
  -> m a
withSftpSession session = bracket (liftIO $ sftpInit session) (liftIO . sftpShutdown)

withSession :: forall m a
  .  MonadUnliftIO m
  => Text
  -> Int
  -> (Session -> m a)
  -> m a
withSession hostname port = bracket
  (liftIO $ initialize True >> sessionInit (Text.unpack hostname) port)
  (liftIO . sessionClose)

As far as I can see, the FFI in libssh2-hs for getHostKey is correct. Am I missing something?

[portnov]

I would suggest the following plan of investigation:

1. Try to run the same code you're having problem with against a server you can control. For example, just run openssh-server on your localhost. Will this code return something other than empty string? If not, then probably it's something related to specific server configuration.
2. Write similar code in plain C + libssh2. As you can probably see, Haskell code which uses libssh2-hs is very similar in structure to C code (it is, in fact, imperative), so it shouldn't be a problem to rewrite it in C. And test that C code against servers you're having problems with. Will session_hostkey still return an empty string? If so, then it's either problem of server configuration or a bug in libssh2 itself. If no, it's something in haskell bindings.
3. Will C code return an empty string on your localhost?

[Hjdskes]

Thanks. I ran my code against https://hub.docker.com/r/atmoz/sftp which has both an RSA and an ED25519 host key. I also find an empty string as the host key.

Going further, I adapted the SFTP example from libss2 into a minimal example to fetch the host key in C (code below). This code also returns an empty string, but does print a finger print:

[nix-shell:~/Downloads/libssh2-1.10.0/bin]$ ./sftp 10.241.167.53 22 username password
Fingerprint: 28 75 36 2E B2 8B E8 8D 87 63 28 9F 42 0E D1 6F 2D 8B 38 EE
Host key: ""
Host key type: 1
all done

I added a function to get the finger print on Haskell and can confirm that even though the host key is an empty string, I can get a finger print. At this point, I suspect that libssh2_session_hostkey is not the function we want to use (clearly the handshake and thus key exchange succeeded, and we have a fingerprint, yet no host key), but I can't find what to use instead. Do you have an idea?

/*
 * "sftp 192.168.0.1 22 user password"
 */

#include "libssh2_config.h"
#include <libssh2.h>
#include <libssh2_sftp.h>

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <stdlib.h>

const char *username = "username";
const char *password = "password";
short port = 22;
static const char *const hostkey_method_ssh_ed25519 = "ssh-ed25519";
static const char *const hostkey_method_ssh_rsa = "ssh-rsa";
static const char *const hostkey_method_ssh_dss = "ssh-dss";
static const char *const hostkey_method_ssh_all = "ssh-ed25519,ssh-rsa,ssh-dss";

int main(int argc, char *argv[])
{
    unsigned long hostaddr;
    int sock;
    struct sockaddr_in sin;
    const char *fingerprint;
    LIBSSH2_SESSION *session;
    int rc;
    LIBSSH2_SFTP *sftp_session;

    if(argc != 5) {
        fprintf(stderr, "%s host port user pass\n", argv[0]);
        return 1;
    }

    hostaddr = inet_addr(argv[1]);
    port = atoi(argv[2]);
    username = argv[3];
    password = argv[4];

    rc = libssh2_init(0);
    if(rc != 0) {
        fprintf(stderr, "libssh2 initialization failed (%d)\n", rc);
        return 1;
    }

    /*
     * The application code is responsible for creating the socket
     * and establishing the connection
     */
    sock = socket(AF_INET, SOCK_STREAM, 0);

    sin.sin_family = AF_INET;
    sin.sin_port = htons(port);
    sin.sin_addr.s_addr = hostaddr;
    if(connect(sock, (struct sockaddr*)(&sin),
               sizeof(struct sockaddr_in)) != 0) {
        fprintf(stderr, "failed to connect!\n");
        return -1;
    }

    /* Create a session instance
     */
    session = libssh2_session_init();
    if(!session)
        return -1;

    /* Since we have set non-blocking, tell libssh2 we are blocking */
    libssh2_session_set_blocking(session, 1);

    rc = libssh2_session_method_pref(session, LIBSSH2_METHOD_HOSTKEY, hostkey_method_ssh_all); //"ssh-ed25519,ssh-rsa,ssh-dss");
    if(rc) {
        fprintf(stderr, "Failure setting method prefs: %d\n", rc);
        return -1;
    }

    /* ... start it up. This will trade welcome banners, exchange keys,
     * and setup crypto, compression, and MAC layers
     */
    rc = libssh2_session_handshake(session, sock);
    if(rc) {
        fprintf(stderr, "Failure establishing SSH session: %d\n", rc);
        return -1;
    }

    /* At this point we havn't yet authenticated.  The first thing to do
     * is check the hostkey's fingerprint against our known hosts Your app
     * may have it hard coded, may go to a file, may present it to the
     * user, that's your call
     */
    fingerprint = libssh2_hostkey_hash(session, LIBSSH2_HOSTKEY_HASH_SHA1);
    fprintf(stderr, "Fingerprint: ");
    for(int i = 0; i < 20; i++) {
        fprintf(stderr, "%02X ", (unsigned char)fingerprint[i]);
    }
    fprintf(stderr, "\n");
    int hostkey_type = 0;
    const char *hostkey = libssh2_session_hostkey(session, NULL, &hostkey_type);
    fprintf(stderr, "Host key: \"%s\"\nHost key type: %d\n", hostkey, hostkey_type);

    /* Authenticate via password */
    if(libssh2_userauth_password(session, username, password)) {
        fprintf(stderr, "Authentication by password failed.\n");
        goto shutdown;
    }

    sftp_session = libssh2_sftp_init(session);

    if(!sftp_session) {
        fprintf(stderr, "Unable to init SFTP session\n");
        goto shutdown;
    }

    libssh2_sftp_shutdown(sftp_session);

  shutdown:

    libssh2_session_disconnect(session, "Normal Shutdown");
    libssh2_session_free(session);

    close(sock);
    fprintf(stderr, "all done\n");

    libssh2_exit();

    return 0;
}

[portnov]

In libssh2-hs, getHostKey is a binding to libssh2_session_hostkey, see https://www.libssh2.org/libssh2_session_hostkey.html.

Unfortunately I am not a big libssh2 expert :) Personally I use only a couple of features of this library (keys-based auth, execute commands + scp). So if you need to know which libssh2's functions must be called and in which order (and can't find it in documentation or examples), you probably have to consult with libssh2's developers. Last time I checked the main place of communication was the mailing list, but now it seems that github issue tracker is also alive.

[Hjdskes]

Thanks; I wrote to the mailing list. I'll post the answer if/when I have it :)

[portnov]

Haha :)
I've read your thread in the mailing list. The most important part is, obviously,

Oh, I see, the key is not a string, it contains binary data and (usually) starts with a zero byte :-).

The funny thing is, it appears that libssh2_knownhost_checkp function is written in such a way, that if you pass an empty string as host key, it always returns success. Probably that's why the library/bindings seemed to work for me with such a bug.

I implemented a fix in the PR #67. Please review.

Since Haskell type signatures of exported functions are changed (String -> ByteString), this may be a breaking change, so I'll have to publish a new "major" version on hackage — 0.2.1.0, for example.

[Hjdskes]

Haha yea it was quite the "D'oh" moment :)

I'll have to publish a new "major" version on hackage — 0.2.1.0, for example.

Please do! It would make my life easier not having to carry this in my custom implementation. That said, I'll quickly open a PR to add some ciphers :)