A Day in the Life of Our NOC Team: A Q&A with Marcus Prieto

 

Next up in our Q&A blog series, we’re checking in with a team member who has one of the most important responsibilities at UPN: keeping our network up and running. Please meet Marcus Prieto, UPN’s Director Network Operations Center (NOC).

How long have you been with UPN?

I have been with UPN since June of 2017. Prior to UPN, I worked for Ericsson as their Network Operations Manager. When I came aboard, UPN was looking to evolve from a reactive monitoring NOC to a proactive full-service customer NOC. In addition to network support, UPN wanted to have more control of the customer experience and post-installation support. I was lucky enough to be chosen for that position and we got to work putting processes in place and building this thing from the ground up.

 

What makes UPN’s NOC team different?

First, we’re not just a network monitoring center. We handle all aspects of the network, from partnering with field operations to provision and turn-up new customers, to managing preventative maintenance routines. On the monitoring side, our goal is to see a network issue before the customer and proactively contact them, if we do receive an incoming call, our customers are not going to be greeted with a traditional “ticket taker.” When they call our support line, they are reaching Tier 1 support automatically. In typical NOCs, you’ll have someone answering the phone, opening a ticket, and gathering information from a very high level. Those people then put that ticket into a dispatch queue for someone on the Tier 1 team to pick up. But at UPN, the technicians we employ are what we consider Tier 1. When the customer calls in, they are going to be working with someone who has experience, ultimately leading to a faster and more thorough resolution. We also employ a lead tech on each of our three shifts. Cumulatively, we have around 60 years of experience between those leads, and that leads to faster resolution in our Tier 1 team as well.

 

That falls in line with everything we hear about UPN – old-school customer service.

That’s a great way to explain it – old-school customer service. When a customer calls, they are not asked to press a number before speaking to someone. Our call analytics show, on average, we’re answering the phone in less than 15 seconds. Imagine that you’re disputing a credit card charge with your bank and being able to speak to the exact person who can solve your problem within 15 seconds. That’s unheard of.

 

Can you give me an overview of the structure of the NOC team at UPN?

We employ multiple technicians across 3 shifts to provide coverage 24 hours a day, 7 days per week. There is also a technical lead and manager for escalations and support. It could be 3 in the morning on Christmas Day, and somebody is there watching the network, answering the phones, and working trouble tickets and alarms as they come in. The team members are located at the UPN headquarters in Kansas City. They are all full-time employees working every day to support the network and the partnerships that we have with our customers.

 

What is a typical day for you and your team members?

My typical day begins by looking at what took place the night before and reviewing the new customer installation tasks for the day. We have hand-offs between each shift, but the night shift or the mid-shift is responsible for a lot of the maintenance that’s performed between 12 a.m. and 6 a.m. Because we are working remotely right now, I’ll usually hop onto Microsoft Teams to ensure I am up to speed on anything that happened during the previous shifts.

And then it depends on what kind of meetings are lined up for me that day. I’ll usually meet with the sales team to talk about potential customers that might be coming on board and review contracts or RFPs. My job is not only making sure that the day-to-day operations are running smoothly, but also handling any escalations with customers. If there’s a customer who is having trouble or feels like their ticket is not progressing as it should, they have access to my cell phone, which I answer 24 hours a day. There is a lot of collaboration on my part between sales teams, customers, field teams, and other team members.

For the NOC technicians, their typical day involves scheduling new installation tasks and pre-provisioning switches to be ready for field operations to call in for testing, monitoring the network, and watching for alarms, and certain thresholds that we have set across devices in each market. They will work with the field on any trouble tickets that are pending, making sure we have good communication on tickets internally and externally. They also manage the day-to-day partnerships and relationships that we have with customers.

We really want customers to feel that doing business with us is a partnership. We have a 98% retention rate and an average contract length of nearly ten years. We feel that this has a lot to do with treating our customers as partners.

I consistently remind the team in the NOC that customers are reaching out to us when something’s wrong. When they need assistance with something, we can essentially take a bad situation and turn it into a positive experience by providing good communication, good support, and being as knowledgeable as possible about our network.

For customers that reach out to us through our web portal or by email, we are committed to a 30-minute response time. The team does a fantastic job of getting back to customers, we feel being responsive regardless of the issue is good customer service. They do a great job of helping customers, managing the network, and tackling all the communication from the field to the customers.

 

For people who aren’t as familiar with a NOC, what is a typical number of alerts that one might get during a single shift?

We own and manage our fiber, which provides a stable network environment. But even though our network is extremely stable, we can’t prevent people from cutting fiber accidentally.

During the ice storm in February in Texas, there were a number of power outages across the state due to the severity of the storm. And then Iowa experienced derecho storms, which are essentially landlocked hurricanes with gale-force winds. That event knocked out some power to our customers, so there can be two extremes. It can be very calm one day and then an act of God can hit, and everything lights up red temporarily. I’d say there’s not really a typical day or typical amount of alerts. Alerts are always going to happen at any data center NOC (though I can proudly say we experience fewer than most thanks to UPN’s superior network). The true test is how the team handles them. The key for UPN is having accurate data to help correlate alerts and then prioritize restoration efforts.

 

What qualities do NOC team members need in order to be good at their job?

You need to be hardworking, hungry for knowledge and good with people. This is not a behind-the-scenes role. We’re fielding hundreds of calls a day and working directly with customers. You have to be smart in the sense that you understand how networking works, but you must also be “people smart” – understanding how to empathize and recognize that the person calling you is probably not having a great day. You must also be a good team player – understanding the importance of collaboration with not only our internal customers, the sales team and the field engineer team, but also the external customers who keep the lights on here.

 

You’ve been in the industry for 20 years. Has this role changed significantly in any way?Are there any trends you can talk about?

Technology is really what has changed. I have talked with colleagues who have been around since the days when everything was on coax cable and there was no fiber. You can get a lot more information from the devices that are out there now remotely than you could 10-15 years ago. Troubleshooting on a TDM copper network versus an ethernet fiber or glass network is very different. The technology is what has facilitated the most change in how people troubleshoot, support, and monitor networks. Clearly, the way UPN interacts with and treats our customers is a constant, though; year after year, that remains the same.

 

Is there anything else that has surprised you over the years besides snowstorms in Texas or landlocked hurricanes in Iowa?

Surprisingly enough, you have outages that people wouldn’t think of where fiber is damaged by bullets from guns and squirrels chewing the line. Those are probably the two most interesting for me. We get many tickets where a squirrel or some sort of rodent got inside of a case and started chewing on things.

 

Is there anything else you want to say about UPN’s network in relation to others in the U.S. market?

We own our fiber and our network is private to our customers. We also don’t offer any residential services. The customer’s traffic is the only thing going across our fiber, which is unique to this industry. When we do see an alert and need to reach out to the field, our team knows immediately how it runs because they’re the ones who have had their hands on it and put it into the ground. But inevitably, the unexpected does happen with car accidents, squirrels, weather or fiber cuts. But the ability to get directly on the phone with Tier 1 support is a true partnership. It’s not just our customer’s issue, it’s our issue.

 

Is there anything else you’d like to say?

UPN is a great company to work for. We’ve been listed in Kansas City Business Journal’s “Best Place to Work” for the past two years. So not only is our network very stable but our NOC is filled with top-notch people.

Thank you, Marcus! Connect with Marcus on LinkedIn and feel free to leave a comment if you have any questions for him. Stay tuned for more Q&As with our expert team, and keep an eye on our Careers page for Network Operations Center Engineer and Network Operations Center Technician Jobs.

Learn more about our dark fiber map and read all about our fiber networks.