Ganymed SSH2 for Java FAQ

This FAQ includes information regarding topics that were discussed in e-mails between developers and users of the Ganymed SSH2 for Java library.

Ganymed SSH2 for Java homepage: http://www.ganymed.ethz.ch/ssh2
Last update of FAQ: 24 aug 2005.

Please report bugs, typos and any kind of suggestions to Christian Plattner (plattner at inf.ethz.ch).

Sections:

• When I start program XYZ with putty (or openssh, ..., whatever) then everything works. However, if I use "Session.execCommand", then XYZ behaves differently or does not work at all!
• My program sometimes hangs when I only read output from stdout! Or: can you explain me the story about the shared stdout/stderr window in the SSH2 protocol? Or: what is this "StreamGobbler" thing all about?
• Why are the session's Input- and OutputStreams not buffered?
• Why can't I execute several commands in one single session?
• Do you have an example for the usage of feature XYZ?

When I start program XYZ with putty (or openssh, ..., whatever) then everything works. However, if I use "Session.execCommand", then XYZ behaves differently or does not work at all!

Short answer:

The most often source of problems when executing a command with Session.execCommand() are missing/wrong set environment variables on the remote machine. Make sure that the minimum needed environment for XYZ is the same, independentely on how the shell is being invoked.

Example quickfix for bash users:

1. Define all your settings in the file ~/.bashrc
2. Make sure that the file ~/.bash_profile only contains the line source ~/.bashrc.
3. Before executing Session.execCommand(), do NOT aquire any type of pseudo terminal in the session. Be prepared to consume stdout and stderr data.

Note: If you really want to mimic the behavior of putty, then don't use Session.execCommand(), instead aquire a pty (pseudo terminal) and then start a shell (use Session.requestPTY() and Session.startShell()). You then have to communicate with the shell process at the other end through stdin and stdout. However, you also have to implement terminal logic (e.g., escape sequence handling (unless you use a "dumb" pty), "expect-send" logic (output parsing, shell prompt detection), etc.).

Long answer:

If you login by using putty, then putty will normally request a "xterm" pty and your assigned shell (e.g., bash) will be started (a so called "interactive login shell"). In contrast, if you use Session.execCommand() to start a command then (unless you ask for it) no pty will be aquired and the command will be given to the shell as an argument (with the shell's "-c" option).

The way a shell is being invoked has an effect on the set of initialization files which will be read be the shell.

To demonstrate the difference, try the following (from the command line, e.g., with an OpenSSH client):

1. Login interactively and print the environment with the "env" command:
    
   [user@host ~] ssh 127.0.0.1
   [user@host ~] env
    
   
2. Let the ssh server execute the "env" command (equivalent to using Session.executeCommand()):
    
   [user@host ~] ssh 127.0.0.1 "env"

If you compare the two outputs, then you will (unless you have adjusted your shell's settings) observe different environments.

If you are interested in the details, then please read the INVOCATION section in man page for the bash shell. You may notice that the definitons of "interactive" and "non-interactive" (and combinations with "login") are little bit tricky.

My program sometimes hangs when I only read output from stdout! Or: can you explain me the story about the shared stdout/stderr window in the SSH2 protocol? Or: what is this "StreamGobbler" thing all about?

In the SSH2 low level protocol, each channel (e.g., session) has a receive window. When the remote SSH daemon has filled up our receive window, it must wait until we have consumed the input and are ready to accept new data.

Unfortunately, the SSH2 protocol defines a shared window for stderr and stdout. As a consequence, if, for example, the remote process produces a lot of stderr data and you never consume it, then after some time the local receive window will be full and the sender is blocked. If you then try to read() from stdout, your call will be blocked: there is no stdout data (locally) available and the SSH daemon cannot send you any, since the receive window is full (you would have to read some stderr data first to "free" up space in the receive window).

Fortunately, Ganymed SSH2 uses a 30KB window - the above described scenario should be very rare.

Many other SSH2 client implementations just blindly consume any remotely produced data into a buffer which gets automatically extended - however, this can lead to another problem: in the extreme case the remote side can overflow you with data (e.g., leading to out of memory errors).

What can you do about this?

1. Bad: Do nothing - just work with stderr and stdout Inputstreams and hope that the 30KB window is enough for your application.
2. Better, recommended for most users: use two worker threads that consume remote stdout and stderr in parallel. Since you probably are not in the mood to program such a thing, you can use the StreamGobbler class supplied with Ganymed SSH2. The Streamgobbler is a special InputStream that uses an internal worker thread to read and buffer internally all data produced by another InputStream. It is very simple to use:
   

   InputStream stdout = new StreamGobbler(mysession.getStdout());
   InputStream stderr = new StreamGobbler(mysession.getStderr());

   You then can access stdout and stderr in any order, in the background the StreamGobblers will automatically consume all data from the remote side and store in an internal buffer.
3. Advanced: you are paranoid and don't like programs that automatically extend buffers without asking you. You then have to implement a state machine. The method Session.waitUntilDataAvailable() is exactly what you need: it blocks until either stdout or stderr data has arrived and can be consumed with the two InputStreams. You can then check with InputStream.available() (for stdout and stderr) which InputStream has data available (i.e., a read() call will not block). Be careful when wrapping the InputStreams, also do not concurrently call read() on the InputStreams while calling Session.waitUntilDataAvailable(), otherwise the call may block.
4. The lazy way: you don't mind if stdout and stderr data is being mixed into the same stream. Just allocate a "dumb" pty and the server will hopefully not send you any data on the stderr stream anymore. Note: by allocating a pty, the shell used to execute the command will probably behave differently in terms of initialization (see also this question).

Why are the session's Input- and OutputStreams not buffered?

If you need it, then this libaray offers quite a raw type of access to the SSH2 protocol stack. Of course, many people don't need that kind of low level access. If you need buffered streams, then you should the do the same thing as you would probably do with the streams of a TCP socket: wrap them with instances of BufferedInputStream and BufferedOutputStream. In case you use StreamGobblers for the InputStreams, then you don't need any additional wrappers, since the StreamGobblers implement buffering already.

This code snippet will probably work well for most people:

InputStream stdout = new StreamGobbler(mysession.getStdout());
InputStream stderr = new StreamGobbler(mysession.getStderr());
OutputStream stdin = new BufferedOutputStream(mysession.getStdin(), 8192);

Why can't I execute several commands in one single session?

If you use Session.execCommand(), then you indeed can only execute only one command per session. This is not a restriction of the library, but rather an enforcement by the underlying SSH2 protocol (a Session object models the underlying SSH2 session).

There are several solutions:

• Simple: Execute several commands in one batch, e.g., something like Session.execCommand("echo Hello && echo again").
• Simple: The intended way: simply open a new session for each command - once you have opened a connection, you can ask for as many sessions as you want, they are only a "virtual" construct.
• Advanced: Don't use Session.execCommand(), but rather aquire a shell with Session.startShell(). See also this question.

Do you have an example for the usage of feature XYZ?