Migrating Telnet to SSH without Migrating

Published: 2017-05-04
Last Updated: 2017-05-04 16:20:16 UTC
by Rob VandenBrink (Version: 1)
6 comment(s)

I recently had a security assessment / internal pentest project, and one of the findings was "I found an AS/400 running telnet services" (actually unencrypted tn5250, but it comes to the same thing)
The client's response was that this host was up for history purposes only, it was not longer production system.  So it was only used occassionally when they needed transaction history from before their migration to the current system.  Which doesn't really address risk around their client's information on that host.

We've all been there.  We've found a telnet service that should be migrated to SSH, but the affected device either doesn't support SSH, or the client for one reason or another can't put resources into enabling encrypted services.  In the case of the AS400 above, they'd need to do an OS update, which would require an application update to an app they had retired, on a system that isn't production anymore.

We see this in legacy systems, but in Industrial Control Systems (ICS) that control factories, water or hydro utilities we see this all the time in production - and the answer there is "the gear doesn't support ssh, and in some cases doesn't support credentials".  In ICS systems in particular, gear like this is often on the same 5,7 or 10 year depreciation cycle as might be seen on an industrial press or other manufacturing equipment, so upgrades are really a long-term thing, there are no quick fixes.  Even finding where all the vulnerable gear is (physically, not on the network) can be a challenge

So what to do?

In some cases, I've front-ended the problem child gear with a cheap SSH gateway.  A Raspberry Pi does  a decent job here for less than $100 per node. The Pi runs "real" linux, so you can secure it.  The solution looks like this:

 

Physically, it looks like this - often we'll just velcro the Pi to the host it's protecting, the "Unencrypted DMZ" ends up being a 1 ft ethernet cable:


The linux account that the user will use gets a ".profile" file, which lookslike this:

telnet x.x.x.x (the target host)
exit (which logs out the SSH session when the telnet session ends)

When the user logs in, their session immediately telnets to the back-end device.  You can expand this with NAT and port forwarding to expose other services in either direction, depending on your "appetite for complexity".  In most cases I do this for, all we're shooting for is an SSH gateway solution - backups and other services go out via another path.

Of course, there are limits to this:

  • A Pi is not the most reliable solution - they boot off of SD cards, and those do fail.  Keep some spare, imaged SD cards
  • It's not the fastest solution - the Pi only supports 100mbps ethernet, and the ethernet NICs are  USB based so you won't ever see it reach 100, but for telnet that's fine.
  • Disable the Pi's wireless if you're not using it as one of the NICs.
  • If the client doesn't have Linux skills in-house, keeping things patched and properly maintained can be a challenge.  If they've got linux chops, this'll plug right into their ansible/puppet/chef or whatever infrastructure.
  • More on the above, no Linux distro is secure out of the gate, least of all the Pi which comes out of the box with default or no credentials (or no password more like).  You'll want to secure the device with something like the CIS Hardening Guide for whatever comes close to your distro (My Pi runs Raspian, so I used the CIS Debian guide, but there are Redhat or Ubuntu distros for the Pi as well). 
  • Using NISTR 7966 to further address SSH can also be a huge help (http://nvlpubs.nist.gov/nistpubs/ir/2015/NIST.IR.7966.pdf ) - if you can simplify things so that the client uses SSH keys for the SSH session, then only has to login once to the telnet back-end, that can keep everyone a lot happier (and more secure to boot)
  • If you need more reliability or speed, you can keep the same approach with something like an Intel NUC or one step up (something with 2 NICs) - a cheap, compact SSD based computer with 2 NICs.  Using a "real" host allows you to go with an easier to manage, more full featured solution - I'll often use pfSense or even OpenWRT instead of a self-secured Linux distro for something like this.

Other approaches?
The ICS / SCADA approach I often see at client sites is to put all the problem devices in one VLAN, then segregate that VLAN.  That works well, and you likely still want to do that even with the SSH approach above.  Often we'll require a VPN session to get to that VLAN, which is pretty easy if that VLAN is off the same firewall that already has an internet VPN configured on it.  This works well in hospital situations as well.

The risk in this approach is that if an attacker does get access to that VLAN, it's just too easy to pivot or spread out, all the vulnerable hosts are in one place - it's really a "wolf amongs the chickens" situation.  So we'll try to further segregate or secure services as much as possible within that VLAN, or split it up into smaller pieces at least.  Anything you can do to reduce the "splatter zone" after the security incident is a good thing!

What have you done to secure "unsecurable" (for whatever reason) services?  Please, use our comment form to tell us the neat and "free or almost free" approach you've adopted!

===============
Rob VandenBrink
Compugen

Keywords:
6 comment(s)

Comments

Another factor to consider is "can the service (in this case telnet) be reached from outside of the organization in question?" If so, another thing would be to use an ACL to restrict telnet services from an un-trusted network/source address.

Port forwarding might work as well (i.e. - forwarding X11 through SSH) for purposes of security.

Just my two cents worth
Considering those systems are usually a bit old, as mentioned, they usually host important / valuable information or other capabilities that cannot be migrated.

Securing them further can only help; I have found from previous experience a practice of doing internal VPN's to be satisfactory. As an added benefit, you can even add your corporate MFA tokens (or assign purpose-specific ones)

In essence, you set up a firewall with VPN capability, create session configuration (say AnyConnect profiles) and the firewalled environment contains only those legacy interfaces. In those areas, you can tolerate clear text because it is very limited and under specific security policies. In case of a PCI DSS environment, you may have a hard time, but if you assess properly the risk and know your QSA, you should be fine.

More details: the VPN profile will come with split-tunnel enabled, and this is ok because you are on the corporate LAN, not the internet. Having split-tunneling allows the access to be defined as a single route added in the routing map of the user's computer.

So your telnet traffic is still telnet inside the user computer, which is acceptable, where it enters an IPSec tunnel all the way to your sensitive zone. If you have more than one such system, you can create specific VLAN's and trunk all those on the firewall. And i find using a firewall adds credibility in an audit.
Had a VAXstation 3100 a few years ago that I enabled native SSH on.

I
t

w
a
s

v
e
r
y



s

l

o

w

.
The best solution I can think of is to piggyback it off an existing system that is kept up to date on patching rather than add another component requiring maintenance. That way whatever existing access and authentication controls you have on the frontend system would protect the AS/400.

Isolating it in a vlan could provide some additional protection, but I wouldn't consider that to be any more protection than the concept of 'security through obscurity'.

There are many different environments, some more permissive than others where some solutions (such as a Pi or Beaglebone) would be acceptable. The environments where I am employed now would not allow that sort of a solution, however.

I like solutions that don't add responsibilities or workloads. Use what exists if it's properly set up.
When we've had this situation, we were in a position to be able to disconnect the affected server from the network, and attach a dedicated PC to serve as console to it. The only way to access that server was from the dedicated PC.

In another situation, we had a half-dozen or so Server 2000, NT4, and XP virtuals that were retired and used for record retrieval once or twice a year. We did the same thing for those - dedicated an isolated ESXi host to them, and provided a single PC with vSphere Client that was used to access them. The "servers" were internally connected, but had no connectivity in or out.

Obviously, disconnecting the resource and providing a dedicated entry point isn't always feasible for a variety of legitimate (and not-so-legitimate) reasons, but it's a nice option to have when it is.
[quote=comment#39480]The best solution I can think of is to piggyback it off an existing system that is kept up to date on patching rather than add another component requiring maintenance. That way whatever existing access and authentication controls you have on the frontend system would protect the AS/400.

Isolating it in a vlan could provide some additional protection, but I wouldn't consider that to be any more protection than the concept of 'security through obscurity'.

There are many different environments, some more permissive than others where some solutions (such as a Pi or Beaglebone) would be acceptable. The environments where I am employed now would not allow that sort of a solution, however.

I like solutions that don't add responsibilities or workloads.[/quote]

Thoughts like these only contribute to breaches across the world. the "Good Enough" approach is what got most developers and companies into this kind of mess in the first place, most notably being the developers of these older technologies never guessing they would be attacked in any way or subjected to mis-use. The power companies are the biggest example of this unfortunately. Equipment that uses telnet, and other rudimentary protocols to configure or be managed, and in many cases dont even require a password, or the password is something like 12345678.

I have ran into such devices numerous times in industrial environments, and even found them to be wide open with their ports forwarded for remote access with little if any source restriction from the internet because an engineer thought it would be nice to be able to adjust settings on his PLC from his easy chair at the corporate office. And usually the same engineers answer when told this is a huge risk is something like, Hey noone knows what our IP is, or noone knows the password to the telnet....

This type of approach can actually be a serious bolstering of the security of such devices, especially if the source connections are then restricted using iptables on the PI, And many pieces of this kind of equipment in engineering spaces and deployments have pad-lockable fuse/control/junction boxes on them where the PI could easily be concealed, thus even an intruder who has gained physical access to the equipment wouldnt easily notice its presence, further complicating any attempt he made to connect to the device in question.

So many times ive seen and heard of silly ideals, and corporate policies like those making it nearly impossible to properly secure equipment, and in a few cases penalizing those who go against the policy to secure something anyway.

Diary Archives