How to Update OpenSSH on an HPUX 11.11 PA-Risc box

We made a few tweaks to our QA Automation system to improve test case execution speed. A side effect of the changes made support for OpenSSH Versions lower than 4.4 a real trial to support. As only one machine in our test lab was running an out-dated version of OpenSSH (The HPPA 11.11 machine) it was decided that it should be updated to a more recent edition of OpenSSH.

I was expecting it to take an hour (tops), but my shortcut method didn't work. I ended up using a multi-step process (including compiling OpenSSL) to get the box up to date.

Notes:

• DepotHelper [hpux.connect.org.uk]
• OpenSSH 5.9 package [hpux.connect.org.uk]
• OpenSSL Source Code [openssl.com]
• In this case, I specifically grabbed the 1.0.0e source
• I had to compile from source since the version of OpenSSL that came with OpenSSH from DepotHelper was not the correct version. Hopefully this gets corrected in a later release (I emailed them and let them know the problem I had)
  
  
• You can get an idea of what version of OpenSSH is installed on your system by running this command ( ssh -V ):
  
  bash-3.2# ssh -V
  OpenSSH_4.1, OpenSSL 0.9.7e 25 Oct 2004
  HP-UX Secure Shell-A.04.10.002, HP-UX Secure Shell version

Part 1: Install DepotHelper, Install Latest OpenSSH version

• Download & install the DepotHelper package manager for your platform (PA-Risc 11.11 for me)
• For help on Installing DepotHelper (Mostly navigating the swinstall tool) see this previous article.
  
  
• Run this command to install OpenSSH using DepotHelper:
  
  /usr/local/bin/depothelper openssh
• This will install OpenSSH and dependencies to /usr/local/bin, /usr/local/sbin, /usr/local/etc, etc...
  
  
• Generate any host keys you need (or re-use your existing keys). You can do this with the ssh-host-config command
  
  
• Configure your sshd_config file (or copy over your old one). The sshd_config file used by the OpenSSH Server can be found in /usr/local/etc/sshd_config
  
  
• Try to run /usr/local/sbin/sshd
• Make sure this won't conflict with the existing port-binding on the built-in SSH server!

If all goes well, you will see an sshd process running when you run a ps -ea | grep ssh and you should be able to SSH in over the port configured in the sshd_config file

(Read on for my troubleshooting & resolution- the above didn't quite work out for me)

Part 2: Troubleshooting

• After performing the above steps I was greeted with this error when trying to run the sshd server:
  
       bash-3.2# /usr/local/sbin/sshd
       /usr/lib/dld.sl: Can't find path for shared library: libcrypto.sl
       /usr/lib/dld.sl: No such file or directory
       ABORT instruction (core dumped)
  
  
• I checked /usr/local/lib and found that there was a libcrypto.sl.new and tried to rename it, but I got the following error messages:
  
  Before adding execute bit:
  
       bash-3.2# ./sbin/sshd
       /usr/lib/dld.sl: Call to mmap() failed - TEXT /usr/local/lib/libcrypto.sl
       /usr/lib/dld.sl: Permission denied
       ABORT instruction (core dumped)
  
  After adding execute bit (chmod +x /usr/local/lib/libcrypto.sl) I got a better message:
  
       bash-3.2# /usr/local/sbin/sshd
       OpenSSL version mismatch. Built against 1000005f, you have 1000100f
  
       Looks like the version of OpenSSL That is bundled with OpenSSH is incorrect when retrieved with DepotHelper
  
  
• After some digging I found out that 1000005f means OpenSSL v1.0.0e (6 Sep 2011)

Part 3: Compiling OpenSSL on Archaic platforms (Like HPUX PA-RISC 11.11 systems!)

• I downloaded OpenSSL v1.0.0e from the OpenSSL.org Sources page
  
  
• Extracted the source files to the HPUX 11.11 box
  
  
• Set the config script to be executable (chmod +x config) and ran:
  
  ./config   (To configure the build)
  make      (To build it)
  
  (GCC was already installed on this box)

• During Compilation I found these errors:
  
  gcc -I.. -I../.. -I../asn1 -I../evp -I../../include  -DOPENSSL_THREADS  -DDSO_DL -D_REENTRANT -march=2.0 -O3 -DB_ENDIAN -D_REENTRANT -c b_sock.c
  b_sock.c: In function 'BIO_get_accept_socket':
  b_sock.c:629: error: field 'sa_in6' has incomplete type
  b_sock.c: In function 'BIO_accept':
  b_sock.c:861: error: field 'sa_in6' has incomplete type
  *** Error exit code 1
  
  
• I opened up b_sock.c to lines 629 and 861 and found Directives indicating the use of IPV6:
  
       #if OPENSSL_USE_IPV6
               struct sockaddr_in6 sa_in6;
       #endif
  
  
• This is problematic since this is a VERY old HPUX machine which does NOT have IPv6 support installed. HP makes it difficult to get the IPv6 module for this machine, too.
  
  
• Since this machine does not support IPv6, I grepped all the source files looking for OPENSSL_USE_IPV6. I made sure to comment out anything that was IPv6 specific. Here is a file listing of all the files I had to change:
• e_os.h
• apps\s_cb.c
• crypto\bio\bss_dgram.c
• crypto\bio\b_sock.c
  
  Looking back I think I could have saved myself some effort by just modifying e_os.h
  
  
• After saving the changes I ran make again. This time OpenSSL compiled, but the 'tests' did not:
  
  md2test.c:1: error: syntax error before '.' token
  md2test.c:1:12: warning: no newline at end of file
  *** Error exit code 1
  
  rc5test.c:1: error: syntax error before '.' token
  rc5test.c:1:12: warning: no newline at end of file
  *** Error exit code 1
  
  jpaketest.c:1: error: syntax error before '.' token
  jpaketest.c:1:12: warning: no newline at end of file
  *** Error exit code 1
  
  
• In looking at these test files, they all refer to dummytest.c. I saved myself some trouble and just sym-linked md2test.c, rc5test.c and jpaketest.c to dummytest.c (using a command like ln test/dummytest.c test/md2test.c )
  
  
• After re-running make I finally had a successful build! ... or so I thought. Turns out that OpenSSL had compiled a libcrypto.a file rather than libcrypto.sl. For reference, here is the message you see when you try to use the .a file like a .sl:
  
       bash-3.2# /usr/local/sbin/sshd
       /usr/lib/dld.sl: Bad magic number for shared library: /usr/local/lib/libcrypto.sl
        /usr/lib/dld.sl: Exec format error
       ABORT instruction (core dumped)
  
  
• Since I needed the Dynamically linked version of libcrypto, I had to make some changes and try compiling again:
  
  
• make clean          (this ensures that no old build products get in the way)
• ./config shared     (this tells the compiler to create .sl shared libraries on hpux)
• make
• make INSTALL_PREFIX=/var/some/location install     (Installs to a temp location)
  
  After running these commands I found that the build output included the shared library
  
  
• I performed final installation by recursively copying the installation to /usr/local:
  
  cp -r /var/some/location/ssl/ /usr/local/openssl-1.0.0e
  
       This created a versioned instance of OpenSSL that lives in /usr/local/openssl-1.0.0e
  
  
• Finally, I created a symlink so the new OpenSSH version would use the correct libcrypto.sl file:
  
       ln /usr/local/openssl-1.0.0e/lib/libcrypto.sl /usr/local/lib/libcrypto.sl
  
  
• At this point OpenSSH didn't complain about the libcrypto.sl file anymore!

Part 4: Add the required Elliptic Curve DSA key

• At this point I found I had another problem: Since I copied over my existing keys (to preserve ssh client/server relationships) I found that I didn't have an ecdsa key file and that OpenSSH v5.9 requires it out of the box.
  
  
• Rather than regenerating all of my user keys, I used ssh-keygen to create an elliptic curve DSA key:
  
  bash-3.2# /usr/local/bin/ssh-keygen -b 521 -t ecdsa
  Generating public/private ecdsa key pair.
  Enter file in which to save the key (//.ssh/id_ecdsa): ssh_host_ecdsa_key
  Enter passphrase (empty for no passphrase):
  Enter same passphrase again:
  Your identification has been saved in ssh_host_ecdsa_key.
  Your public key has been saved in ssh_host_ecdsa_key.pub.
  The key fingerprint is:
  53:25:c5:4b:ea:c2:5f:37:03:d9:de:8c:f1:ca:fd:df root@blasted-hpux-box
  The key's randomart image is:
  +--[ECDSA  521]---+
  |       ..        |
  |       .o        |
  |       o...o     |
  |      . .oo .    |
  |   . .  S  o +   |
  |    o ....+ o +  |
  |     o . . o . . |
  |      .      . o.|
  |              +oE|
  +-----------------+

Part 5: Set the new OpenSSH Server to run on startup

• To complete installation, I configured the new OpenSSH Server to run at startup in place of the shipping SSH Server
  
  
• To get the new version of SSH Server to start I had to alter the /sbin/init.d/secsh file:
  
  Before:
  WHAT='HP-UX Secure Shell'
  WHAT_PATH=/opt/ssh/sbin/sshd
  WHAT_PID=/var/run/sshd.pid
  WHAT_CONFIG=/etc/rc.config.d/sshd
  
  After:
  WHAT='HP-UX Secure Shell'
  WHAT_PATH=/usr/local/sbin/sshd
  WHAT_PID=/var/run/sshd.pid
  WHAT_CONFIG=/etc/rc.config.d/sshd
  
  
• It is important to note that future changes to sshd_config MUST be made to the config file located here: /usr/local/etc/sshd_config.

Finally:

• At this point the new SSH Server took over once I restarted the HPUX box. For good measure I symlinked the ssh commands from /usr/local/bin to /bin (or wherever they are by default).

• This ensures that when I do ssh or ssh-keygen that I'm using the newer command that comes with the newer OpenSSH
  
  
• I may have saved myself some grief by compiling everything from source (would have avoided the OpenSSL Version mis-match). So if you have another, better way of upgrading OpenSSH list it in the comments below.