Access Controls for Network Infrastructure

Published: 2010-08-05
Last Updated: 2010-08-06 15:22:09 UTC
by Rob VandenBrink (Version: 1)
8 comment(s)

It seems that every time I do a security assessment or pentest, the findings include problems with access controls for things like routers, switches, fiber channel switches, bladecenters and the like.  And when I say "every time", I really mean EVERY TIME.  So, what kind of problems are common, and what can you do to prevent them?

Default Credentials

People love default credentials.  I've actually had a customer tell me "if we forget the password, we can just google for it" - like somehow that's a good thing.  Even for gear that comes with default credentials that force you to change them on login (for instance, fiber channel switches), I've seen other engineers then change the password back to the default - their rationalle being that it reduces their support calls later.

The number of bladecenters that I've seen that have changed the default admin user and password I think I can count without taking my shoes off (ie - it's a small number - almost nobody changes their bladecenter admin password).  This access gives the right to power off servers, decommission servers, unpatch servers, hack windows or linux passwords, almost everything is possible - the bladecenter admin access gives you the near-equivalent of physical access, and we all know that physical access trumps almost every control in the book.

People - default credentials are BAD.  If you are using default userid and password, and someone compromises you, we don't call that "hacking", we call it "logging in".  You don't need an uber-hacker to target you in this situation, ANYONE WITH GOOGLE can compromise you.  Change the admin password on your gear, and if possible, change the admin userid.  Better yet, back end admin logins with another directory (read on ...)

Prevent Access in the First Place
- Access Classes
Many products will allow you to restrict administrative access in the configuration.  For instance, Cisco gear has the "access-class" config statement.  HP, Brocade, Juniper and others all have an equivalent construct.  The commands you'd put in your switch or router config might be:

first, define the subnet or ip addresses of authorized administrative workstations ip access-list standard ACL-ACCESS-CLASS
  permit host 192.168.72.10
  permit 192.168.68.0 0.0.0.255
next, apply the ACL to your access line vty 0 15
  access-class ACL-ACCESS-CLASS in


Prevent Access in the First Place, Reloaded - Define a Management Network


Even with all the other controls we'll talk about, implementing a management network is a good move.  It zones all infrastructure admin into one place, you can control access to this netowrk using VPN controls or a "jump box".  The VPN approach is a neat one - it means that if you are an authorized admin, you can use an IPSEC or SSL VPN solution to get an IP that has access to admin your network from anywhere in the company.  This is really handy for admins that are mobile within the company or have to provide support from home (or the cottage, the beach, or anywhere else they can find you).

Encrypt all Administrative Accesses
I still see LOTS of admin access over standard HTTP and Telnet.  And there are LOTS of tools that will strip passwords out of this type of traffic - you can do this with CAIN if you want a GUI, but really even doing it with wireshark or tcpdump is pretty simple.  Be sure to force SSH (Version 2 if you can, Version 1 can be decrypted), or HTTPS for administration of critical network infrastructure.

Even what you might consider non-critical infrastructure should see the same protections.  I've done a pentest on a company that used telnet to administer their UPS gear (to monitor temperature and humidity as well as power characteristics and remote control of power etc).  Unfortunately for them, they used their AD admin password to login to their UPS, which they accessed using plaintext HTTP and Telnet.  Doubly unfortunate, as a standard procedure they logged in each morning to check logs etc.

Checking logs daily would normally be a really good thing if they had other controls in place (for instance, using HTTPS or SSH for administration), but since they were being pentested, I had their admin password within 10 minutes of starting the actual process !

On a cisco router or switch, the commands you need to set up SSH are:

first, define a hostname and domain name hostname yourdevicename
ip domain-name yourcompany.com

next, generate the key
(use some number equal or greater than 1024 for stronger key strength)


crypto key generate rsa general-keys modulus 1024 
next, force SSH version 2 and force SSH for access

ip ssh version 2


line vty 0 15
  transport input ssh


If you allow web administrative access (that's a whole different discussion), forcing https on network instrastructure is generally even easier:

disable plaintext HTTP no ip http server

enable SSL encrypted HTTPS
entering this last line will generate the (self-signed) SSL cert to
encrypt the webmin sessions.


ip http secure-server


To go one better, I'd also suggest that you replace the self-signed certs that is used by default for HTTPS admin on most gear, using certs on SSH is also a really good mechanism.  Without replacing the default certificates, tools like ettercap can still be used to mount a man-in-the-middle attack and recover passwords.

Back-end Authentication and Change Logging


After everything else is said and done, there's still way too much gear out there that has a single administrative account, or no account at all (only access passwords).  This plays hob with managing change - since every config update is done using the single admin account, if a change goes bad everyone in your team can deny making that change.  (Does your team have "Ida Know" as an honorary member?)  If you don't take a stab at non-repudiation of changes, this WILL bite you eventually.

So, what to do?  Should you define a userid for each and every user on all your infrastructure gear?  Well, only as a last resort.  Most network infrastructure has the capability of back-ending authentication and access controls using some external source.  Popular back-ends are what you'd expect - RADIUS, TACACS, Kerberos, LDAP and Active Directory.  I'd say pick one and go with it.  I often go with RADIUS back-ended with AD (IAS, now NPS) because it's simple, easy to troubleshoot, and supported by almost everything.  Mind you, it's also likely that if you go with RADIUS you are then susceptible to other attacks, but you can mitigate that by setting your RADIUS server up in a private vlan or by using other intelligent design decisions to implement security controls.

Basic definition of AAA, as well as the definition of the RADIUS server
(Note that the RADIUS server config will need to match this)

aaa new-model
radius-server host 1.2.3.4
auth-port 1645 acct-port 1646 key randomcharkey

this line forces radius authentication for login by default.  If radius is down (ie - no response is received from the radius server, or the radius keepalive is missed), then local authentication still works.  Note that on one hand this leaves you open to attacks that involve DOSing your  RADIUS server.  On the other hand, you still have access to your network gear if your RADIUS servers or domain controllers are offline. aaa authentication login default group radius local
defining the source interface is important, since the ip of the device is normally hard-coded on the radius server ip radius source-interface Vlan1
on some gear you may also need to force authentication on individual lines: line vty 0 4
login authentication radius
line console 0
login authentication radius
line aux 0
login authentication radius


Don't forget to LOG ALL ACCESSES (this is built into RADIUS and TACACS) and LOG ALL CONFIG CHANGES (lots of tools will do this for you - syslog will log that a change occurred and who made it, CATTOOLS and RANCID (thanks to our reader "Bmac" for this correction) are 2 that come to mind for more complete change logging, I've also written shell scripts to do this.  Feel free to suggest others that you use as a comment to this diary).  In some cases, you may also want to log all commands as well (most gear will let you do this in syslog). 

What does this give you?  As changes occur, you are notified that a change happened, who made it, and what it was.  You can then compare this to the CHANGE REQUEST FORM that you have in your CHANGE CONTROL SYSTEM, to be sure that:

  • the change made was both requested and approved
  • the change happened during the change window
  • the person making the change was the one authorized to do it

if it was an unauthorized change, you have the culprit identified and can have, shall we say, a discussion that's appropriate to the situation.

Basic commands for simple logging and NTP time sync are, well, pretty simple (as with most examples in this diary, it can get more complicated)

Basic Syslog logging is a one-liner logging 192.168.5.7
Similarly, setting a target host to get time from is also very simple ntp server 192.168.5.6


I've phrased this discussion in the context of network infrastructure gear, but really many of these points extend to other datacenter infrastructure components as well.  Replacing default certificates used for RDP to critical Windows servers is a good move, as is certificate updates for things like VMware vSphere (both vCenter servers and ESX Hosts).  Using a Management network is an important part of designing a virtual infrastructure from any vendor, as well as bladecenters (for the same reasons).  Management networks can also be used to protect things like RADIUS authentication, syslog, NTP and SNMP based network management, all of which are sent in cleartext.  Back-end authentication to an enterprise directory (often AD) is a common solution for authentication, is useful as we discussed for managing and auditing change for both Linux and VMware servers, as well as for lots of other gear besides just routers and switches.

I'm hoping that you find these suggestions to be a helpful starting point.  There's lots more that can be done in this area, please use the comment feature to let us know if you've found this useful, have your own stories in this area, or if I've missed (or messed up) anything in this diary. 

=============== Rob VandenBrink, Metafore ===============

 

 

8 comment(s)

Comments

Along with syslogs, RANCID is a great tool for configuration diffs. I'd also recommend the 'login on-failure log' and 'login on-success log' options. Combined with configuration change tracking it allows larger organizations to track any questionable changes back to the user that implemented them.
Even if your device doesn't support SNMP with encryption, configuring SNMPv3 with authentication but no privacy stops attackers from replaying captured packets.
Thanks for sharing these important reminders and how-to's with the community.
What should be IT-101 stuff is still IGNORED, even by those with 20 years of experience (AND who have been compromised) and should know better.
Don't forget about keeping the firmware current. Vendors do update these for reasons other than getting you to reboot your network. HP just released an update for a security bypass on their older 2650 switches. If you're still on CatOS because you don't want to convert the configs, shame on you.

The note on SNMP is a good one. I saw this used in a non-obvious way once. The pen test company used the default SNMP read-only community string to determine the uptime on Windows servers. Their theory was that a Windows server that was up for more than a month was not current on patches. They were correct and the default read-only community string allowed them full network access in a very short period of time by targetting one forgotten server. The network admins thought they were smart to disable SNMP write but didn't care about read-only because "There's no way that could hurt us."
I agree with the simple server probe. Another avenue used to gain access is the workstations that many times are never logged out of. Why would this be a vulnerability? Because they are also not updated.

The larger the company, the better the chances that they do not do regular updates on workstations. Why? Because they use old legacy applications. They are afraid that updates will break their applications, so they avoid updates like the plague. Now that XP SP2 is no longer supported for updates of any kind (including the recent LNK vulnerability) this will only get worse, and once you gain access to a logged in workstation, you have the keys to the palace. Not good!

Chances are a select number of these workstations have logmein, gotomypc or pcanywhere enabled, and getting input from a home user computer that has Zeus or similar information stealing kits installed is simple (if you run the botnet).

Now, with the username and password... guess what happens next! Full access!

Pen testers always forget just how easy it is to get the keys to the palace, whether it be a VPN key, a remote desktop, a remote access application, a remote access service or an established VPN from another company you "trust" who just became victim. The rest is just too scary to discuss.

Plain and simple... keep a honeypot or two running at all times as you WILL ALWAYS have visitors you didn't invite!

-Al
One more thing... It is a good idea to set a time-frame limiting access to that "someone" who can log onto the network from standard workstations remotely or otherwise.
I'll add another one...

Vendors like to have remote access to their servers in your company datacenter. Some actually demand it.

These vendors have VPN connections, LogMeIn connections, their own firewalls on their extended network and more. All of this is beyond your control as the administrators want the vendor, so you have multiple open doors all the time, just waiting to be abused.

With all this you have frustrated employees of your organization, the vendors, crackers and more just a click or two away.

You get the picture... you are NOT safe, and if you think you are, the correct word for that is DENIAL!

-Al
Defence in Depth is still the way to control all access - including suppliers.
Ideally, have their remote access via a VPN with switch you enable/disable, and using 2-factor tokens. If they don't like this, point out it'll be in their contract renewal. If they don't meet your requirements, you'll look elsewhere.
Also point out that breaches of your security through their poor security are their legal responsibility - so they may want to not use default passwords, as all their security procedures will become open knowledge in any legal dispute....

Only open up access for the hour it's required, and alarm when it's open, just to be sure some social eng doesn't happen quietly.

Diary Archives