Tuesday, April 13, 2021

HPE 1950 CLI - an undocumented COMWARE mess

 Stuck with a HPE 1950 that won't play nice?  lost connection because of a VLAN change? need to add a default route because you forgot to add one before removing the DHCP option?  look no further.

The HPE 1950 is a wonderful SMB switch for L2 and light L3 duties.  They're robust, relatively cheap, and have PoE options.  Everything you could want in a switch.  Then why is the CLI and GUI such a nightmare?  The web GUI seems to be trying too hard to shoehorn really basic functionality into really fancy and unintuitive menus.  Once you get used to the options being all over the place, you then face the lack of guidance from HPE about how to actually make use of this switch.

I've had two run-ins with the HPE 1950 that I'd like to write about.  First, there was some shenanigans with HPE support when the "DHCP Snooping" option was enabled, and it dropped all DHCP traffic, causing a Sev. 1 down.  The second, I removed a DHCP interface before adding a default route, causing me to lose all access to the web UI, luckily, I had another way.

First, what everyone is here for:

How do we make the HPE 1950's CLI actually useful?

Simple.  Connect to the unit, either through telnet (which appears to be enabled by default) or SSH, if it's enabled, or via a console cable.  Whatever option you have, connect, log in, and you'll be dropped to a relatively useless prompt of <%hostname%>.

You'll notice rather quickly, this prompt isn't really good for anything.  You can factory reset the unit using "initialize", which is almost never the best solution to the problem.  You can change the default interface IP configuration using the "ipsetup" command, but as far as I've seen, this only affects the default interface.  If you have VLAN 1 (default) disabled, or otherwise secured, then this option isn't particularly useful.  The "display" command (aliased as "show"), only allows you to see PoE information, which again, isn't specifically useful.

However, there's one command here that will get us to the 'next level': xtd-cli-mode, or extended CLI mode.  This will allow a lot more options, but again, isn't necessarily useful.  The bonus on this is that it's password protected.  Luckily, HPE's own forums has the password published for all to use.  It is "foes-bent-pile-atom-ship".  why this password? ask HPE.  I really don't know.

Next, we have a "extended" prompt. show is useful, there's a lot more commands ( "?" will bring up what's available), some of them may solve your problem, unless you need to add a route or something.

To add a route, or change an interface or similar, you'll want to issue the "system-view" command.  This puts you into the real administrators seat.  the question mark will be your friend, since this mode is basically undocumented.

Some helpful stuff:

vlan # - add a vlan to the switch config. (brings up a sub-menu to add description and other things)

interface (gig/xge) 1/0/# - enter switchport configuration mode

interface vlan # - enter vlan interface configuration mode

Under interface config mode, you can also shut, and no shut, and also issue vlan commands.  "port" is the keyword (similar to "switchport" on Cisco), where "port link-type (access/hybrid/trunk)" for switchport mode access/trunk, and "port (access/hybrid/trunk) vlan # (untagged/tagged)" will add vlans to an interface.

To Summarize:

Connect by available method
enter "xtd-cli-mode"
enter password "foes-bent-pile-atom-ship"
enter "system-view"

Afterwards, use "?" the same way you would on any other router/switch CLI, and you should be able to figure the rest out.


Story time:

Several years ago, I was implementing HPE 1950's for a client for the first time.  I was checking out all the features.  After deployment, I continued looking into what we could/should implement for security, performance, etc.  They were using the switching entirely in Layer 2 mode, the only L3 interface was for management.  I came across the DHCP snooping option, and enabled it to see what it would do.  I was in for a ride.  I thought "snooping"? how bad could it be?  Well, apparently HPE's idea of snooping is also DHCP enforcement, where you have to authorize DHCP servers.  At the time, HPE had a bug in the 1950 where this option, once enabled, had no GUI button to turn it off.  This has since been fixed, but at the time I was stuck, so I quickly assigned it the LAN DHCP server's IP as an authorized DHCP server, and thought that would 'fix it' at least to the point where DHCP would function.  Wrong again.  I don't know, to this day, whether the feature was just completely broken, or if I had done something horribly wrong. DHCP was down, for everyone on the LAN.  This network just went live.  So I tried everything to try and fix it, even getting to the useless CLI modes I've listed above, with no success.  At the time I didn't have the "extended cli mode" password so I was at an impasse.  I called HPE and started a support ticket, the network is down, so I indicated it was the highest severity, and began working with a technician, while a coworker zipped around the office to static IP all the workstations they could, to try to get some people at least working until we fixed the problem.

After several hours of working with HPE we were nowhere.  No resolution in sight, but being keen-eyed helped me here.  I saw the HPE tech get into extended CLI mode, and while I didn't have the password, I saw him use the "sys" command (aka "system-view"), and when he disconnected to have a higher-level team callback (who calls back on a Sev.1 issue?), I re-entered system-view and issued the "no dhcp snooping enable" command, and fixed my own issue.  Years later I found the extended CLI mode password, and was able to complete the whole thing myself if needed, but to this day, I've never been so curious about the DHCP snooping, and whether it's been fixed, to test it out and actually enable it.

Second story:

I'm prepping a small set of 1950's in Layer 3 static routed mode for a client, this is a growing network, but they haven't yet budgeted for a truly L3/routed setup, much to my dismay.  So I proposed an approximation of one, that should be relatively simple to adapt to full L3 switching and expand exponentially, when the time comes.  I drew up an IP subnetting plan, broke down subnets per building, making each 1950 a gateway/router for the network.  I finally got their switches into my lab for prep.  There were about 6 new switches, only two needed routing enabled (the third building had an L3 capable switch already - which was adjusted after the fact).  Well, while I'm prepping the switches, I setup a management vlan.  VLAN 101 was used for the purpose.  The problem I ran into is that the native/default interface (VLAN1) being DHCP, obtained a default route from the DHCP server, which I had setup before hand.  After setting up VLAN 101, I removed the IP from the VLAN1 interface, and the unit dropped off the face of the earth.  It clicked relatively quickly for me that I forgot to add a default route through VLAN 101's gateway, to route back to my workstation on VLAN1.  No problem, I'll just switch my workstation to VLAN 101, except, VLAN 101 doesn't have DHCP, and I'm not in physical proximity to my lab.  So I go to my lab router, add DHCP to VLAN 101, change my VLAN to 101, and my lab workstation drops off the face of the earth.  facepalm. I didn't add VLAN 101 to the port I'm connected to.  what now?  So I jumped onto the lab router (a Cisco ISR), and started poking around.  I found that I could telnet into the problematic switches, since the lab router had IPs on all relevant VLANs, and local connectivity works without a DG being set.  I got into the switch that didn't have a default route and after getting into "system-view" mode, I was able to issue an "ip route-static 0.0.0.0 0 <gatewayIP>" with some extra parameters for priority and comments.  So now that switch is fixed.  I backed out and connected to the switch my workstation was connected to, I checked for ports that are up/up, and found my system on gigabit port 2, once into the interface config, issued 'port link-type hybrid' and 'port hybrid vlan 101 tagged' and poof, my workstation popped up.

I find it ironic that I was able to fix the IP routing issue before I got my workstation online again.  This happened because I was trying to remember how to get into what Cisco would call "configure terminal" mode, again.


I'm not perfect, but I'm pretty proud of my ingenuity on figuring all this out about a platform where the CLI is basically not documented.  I don't want this information to go to waste, and I hope it helps someone else.  Be well.