Network Engineer – A Dying Breed, In A Devil’s Industry

I’ve often shared with work acquaintances that being a network engineer is all-too-often the most unrewarding job in IT, which many helpdesk-level coworkers are somewhat shocked to hear.

Local college graduates in the last three years for the Lincoln area tend to be trending away from a network-based focus in their IT degrees, and pushing more towards programming, server engineering, virtualization, and “the cloud” – even though these things all fundamentally involve networking, to an extent.

Local job postings seem to be following a similar trend, with few positions listed on boards such as LinkedIn or Indeed strictly asking for just a network-based skillset, but requiring the ability to branch out into other focuses such as servers, Linux, virtualization, and Satan’s spawn itself – telecom.

If you’re reading this and have been working as a network engineer, you likely know this story all too well. But, we’ll get more into what I mean by that in a bit.

For now, let’s start down a road of looking at the often-unappreciated Network Engineer, and how we seem to have been brought to this point in IT.

“We Don’t Need Network Engineers Anymore!”

In a recent job interview, I was told the above statement word-for-word. In other similar interviews, I’ve received a more-or-less transliterated copy of this phrase when asking about the company’s plans moving forward.

The most common reason for this is the buzzword that’s been dominating the industry for the last ten years or so – “the cloud”.

Why would an outsourced cloud-based infrastructure eliminate the need for network engineering in many organizations? Well, with the advent of providers such as Microsoft Azure and Amazon AWS, many network-oriented tasks that previously required dedicated personnel for the onsite equivalent (such as firewalls, load balancing, bandwidth, etc.) can be accomplished through a web interface at the click of a button and the corresponding invisible withdrawal of bank funds.

Because of this, many shifts to cloud-oriented infrastructure are often accompanied by the layoff of some or all of the organization’s network team. What’s somewhat sad about this is that for the majority of large enterprises that moved their server farm to the cloud, the move is seldom necessitated by a degradation of network quality or stability, but is more likely aggravated by shortcomings on the server administration team! It’s rarely necessitated by money (cloud is more expensive for 90% of large enterprise use cases, believe it or not). In some cases, it’s necessitated by a desire for redundancy, but cloud-based redundancy on the desired level will almost always be more expensive to a robust in-house supported solution.

Cloud is killing network engineers, folks. Sad, but true.

The Network Is The Most Easily-Blamable Thing In IT

In an audit of several workflow practices, I’ve been invited to sit in on some of the day-to-day Helpdesk calls and activities within multiple organizations over the course of my career. The end goal was to facilitate better knowledge transfer and issue troubleshooting processes for the team.

What was actually uncovered was that many Helpdesk personnel were, in fact, blaming various oddities that users reported to them on “the network” or the “network issues” that they described as being inherent to everything! In the audit cases I observed, and later ended up troubleshooting personally, less than 2% of the issues were actually caused by network-related problems, but rather stemmed from a combination of poor troubleshooting, lack of Level 1 knowledge, or simply a poor work ethic and wanting to “get people off the phone”.

In multiple previous jobs, the network has been blamed for a similar set of abnormalities, but by a different team – the programmers. The programming teams had a high tendency to not thoroughly torture-test their code enough on their dev environment, and then attributed any “after-effects” on the production environment post-port to “the network” or “network filtering”. To be fair, the blame scale was slightly more level at 20% of the problems being network-caused. Still, roughly 80% of issues were a result of bad error-handling, poor troubleshooting, improper understanding of documented security practices in place, or plain and simple shite code. (programmers don’t like being told they’re writing shite code, by the way)

​With all the often-misguided blame being thrown at the network, it should come as little surprise that many job postings for “Network Engineers” require knowledge about systems, servers, Linux, programming, scripting, and virtualization – you have to know enough about how to do everyone else’s job in the organization to be able to properly defend yourself when they blame problems on you! I say this slightly in jest, but I think every Network Engineer reading this can attest that there’s an element of truth to it!

Since an increasing number of technicians and engineers in IT don’t understand networking, its resulting ambiguity has made it an easy “dump bucket” for issues that seemingly can’t be immediately blamed on anything else. I can attest that this has been the case for every employer I’ve ever worked at, and I see little sign of that trend changing.

Networking Is Dumbed Way, WAY Down

One of my coworkers once told me with a heavy sigh, “It’s amazing how as soon as there’s a problem that people want to attribute to the network, everyone knows more than you.” It’s sad how much truth there is to this statement.

Many folks who don’t know any better believe that networking is the easiest thing in the world. I mean, sure – anyone can go out, buy a little SOHO router, a 24-port Netgear switch, and a box of cables. Plug everything in, and voila! It all works straight out of the box. Congrats, Johnny! You’re a network engineer!

​This same frame of thought is reflected in many issues reported to the network team that incorporate something along the lines of:

  • “Well, everything on the network is just slow!”
  • “I PINGed one of the other routers in the organization, and the times were high, so yeah – there’s a network issue.”
  • “The problem can’t be my computer, so that only leaves the network.”

Note that the common mindset here is that as soon as there’s an issue, the end user (or in some cases, your helpdesk staff!) knows more than the Network Engineers do in that they’ve already “diagnosed” the issue themselves and have brought it you in a more-or-less upset mode of thinking. At this point, they’ve pretty much already made up their mind that the problem is you or on your end, and won’t take “no” for an answer if you try to suggest otherwise.

While I strive to appreciate people genuinely trying to be helpful, there’s a point where anyone in a clear frame of mind has to realize that there’s a reason why I’m a Network Engineer, and they aren’t. In a well-managed, monitored network, the situations will be few and far between where a user experiences a genuine problem that I don’t already know about – and if I know about it, I’ve likely already placed a phone call to the affected department/group to confirm, and have a mass email for notification either on the way or in the works.

As an example, going back to item #2 in the above bulleted list, many people don’t know that Cisco routers will, by default, prioritize traffic to themselves on a lower level than they will for passthrough traffic. What this means is that using PING to them in order to test actual line latency is often a distorted test because they will give you an artificial sense of latency. The same is true of traceroute. 

People Fear What They Don’t Understand

While I’m not going to be so bold as to blanket-statement stereotype and say that this the case for an entire demographic, I’ve encountered a number of people over the course of my career that were somewhat scared of networking because they didn’t understand it and didn’t want to.

Subnetting isn’t an easy concept. The intricate details of routing protocols and how to troubleshoot them aren’t a simple matter, either. A lot of port rules, network policy, and comparable concepts are fairly difficult to wrap your head around – which may explain part of why Helpdesk-level personnel attribute blame to “the network” without knowing any more of what’s going on than that.

More curious still is the inherent fear of network concepts that seems to be present among a variety of IT staff. My theory is that many programmers, database engineers, web analysts, and Helpdesk technicians view the inner workings of the network as ‘the void’, and strive to stay away from it because it’s too complicated or confusing to be worth their time to learn.

There’s also the mentally that “it isn’t my job to know networking, so I won’t”. While I’ll admit that there’s an element of truth to this statement, also take into consideration what subsystem in IT today isn’t dependent on a network in some way/shape/form? Pretty much none – almost all systems these days are more or less reliant on a network for some aspect of their functionality. With that in mind, it’s always beneficial to know networking to the extent that it supports your own job specialization. (in the case of a certain Network Engineer who writes articles on this website, you may find that you actually like networking better than the coding work you were doing before, and change career gears to it!)

Where Will Network Engineers Be 20 Years From Now?

I honestly don’t know.

With the rise to prominence of things such as Software-Defined Networking which use a simplified GUI to automate vast sections of networking, it’s becoming progressively easier to get by without knowing networking in an organization simply because this software (which isn’t cheap, mind you) does everything for you.

Cloud technologies, as mentioned prior, are another piece of the equation that can’t be discredited. I know a handful of former peers and acquaintances that lost their jobs in networking because their companies outsourced to the Cloud and paid the extra premium to downsize their staff as a result.

If you see a genuine network engineer, they’re a mistreated, yet mythical creature. They’ve been forced into a position where they wear a dozen different hats on a daily basis, bear the accusation of every other department in the organization, and likely know enough about how to do several other folks’ jobs to arguably get them fired.

Only time will tell if this endangered species has a place in the IT world of tomorrow.

Caleb Huggenberger is a 31 year-old systems engineer, old-school guitar and amplifier builder, and Eastern culture enthusiast. Outside of long work days, he enjoys electronics engineering, cast iron campfire cooking, and homesteading on his acreage in the Indiana countryside.

Leave A Comment (please keep things clean & civil)

Your email address will not be published. Required fields are marked *