Research, Wi-Fi, mobile and Internet of Things (IoT) devices, collaborative projects and media-rich digital learning are prevalent across U.S. campuses. Voice over Internet Protocol (VoIP) and unified communications platforms are edging out wired phones. Everything seems to be moving to Wi-Fi.
Higher education institutions are under fire to support the plethora of technologies that students, staff, faculty and researchers expect, regardless of where users are on campus and which device they use.
To meet demand, educational institutions need a flexible network infrastructure that expands and adjusts to meet ever changing demands and fully embraces modern technologies.
The University, an Ivy League institution serving more than 8,000 students, incubates some of the world’s most cutting-edge research in the fields of computer science, data science and machine learning, engineering and neuroscience, among others.
In 2017, the University’s Board of Trustees, VP/CIO and network staff contemplated a new architecture and approach that would result in a faster, scalable and highly secure network that reached the edge of what technology offers, enabling continued institutional success. The University’s vision for a modernized, next-generation network called for a new architecture that made security integral to the design and greatly expanded wireless coverage and bandwidth (“Wireless First”).
In addition, the University saw the redesigned network as a lab, with sensors dotted throughout campus, micro-segmentation and high performance, allowing researchers to use specialty instruments and run data-intensive tests without affecting general users. The University envisions allowing researchers to experiment with new networking and compute ideas on the network as they further the dual research and teaching missions.
Other significant elements included a “network as code” approach with substantial automation and orchestration, and support for Internet Protocol version 6 (IPv6).
A Board of Trustees directive also required that implementation be completed in phases on an accelerated schedule, with considerable value achieved within the first phase.
The University retained an independent technology consulting firm to drive the vision and design. Planning involved extensive collaboration between University staff, the consulting firm, peer universities and several leading and innovative vendors.
Based on input from all parties, the resulting roadmap recommended major network design elements, and laid out a plan for incorporating security into the design, Wi-Fi upgrades, limiting vendor lock-in and other improvements that fit into the University’s strategies and operations while minimizing disruption.
Fabric, RBAC and Borders
The team concluded that a mesh network fabric with identity and access management controls, and borders for routing and internet access best fit the needs.
The foundation of this project is a network fabric, which uses several interconnecting switches that form a mesh (see Figure 1) for data exchange and delivery of services. Because it’s not tied to a specific network physical topology, a fabric provides flexibility and agility, as well as resiliency, ease of segmentation and the ability to automate services more readily. The University plans to implement two core locations initially, with fiber connections between each aggregation switch and the cores, and a third core in the future. The fabric will also incorporate software-defined networking (SDN) techniques and technologies, which will help the University split large data flows over paths to minimize hops and latency while maximizing capacity.
Figure 1: Simplified view of mesh fabric with two cores
An identity-aware network allows for access control through user and device identity and authentication, and policy enforcement to control traffic. The University network will include Role-Based Access Control (RBAC), which grants access to network resources based on user and device roles created by an administrator. Figure 2 provides an overview of RBAC in a higher education setting. The University is also evaluating onboarding services, which are needed for devices the first time they access the network, and is leaning toward the use of 802.1X/ Extensible Authentication Protocol-Transport Layer Security (EAP-TLS) certificates for authentication. Initial roles will be simple and will evolve as experience and technology allow. The vision is to work towards identity-aware zero-trust networking with compliance zones/enclaves as needed, and to use the certificates to enable borderless and VPN-less connectivity on and off campus.
Figure 2: Role-based access control
The redesigned network will also have two campus border locations, each with routers and firewalls, internet and research network connectivity providing automated failover and the ability to route dynamically (see Figure 3). Because research data transfers can consume a lot of bandwidth (elephant flows), the solution includes the capability to bypass the firewall for specific types of traffic flows (Science DMZ) if and when needed through automation and orchestration.
Figure 3: Campus border
The University’s network security initiative will be met by tightly integrating security concepts and technologies into every facet of the network. Areas of focus include distributed sensors for analytics, Distributed Denial of Service (DDoS) mitigation, RBAC, micro-segmentation, automation and orchestration, a security information and event management (SIEM) system, DNS security, 802.1X EAP-TLS (user and device certificates) and efforts to integrate security and risk into the culture.
The University’s Wireless First strategy recognizes that devices tend to be wireless, unless there’s a specific need to use a wired connection. To prepare for Wireless First and a potentially dramatic increase in the number of IoT devices (many of which use Wi-Fi), the University has been replacing older wireless access points (APs) and increasing density. The cycle will continue with upgrades to 802.11ax APs once the technology is ready for widespread deployment. By that point, the University expects to have more than 13,000 APs installed across campus.
Additionally, the University is considering Bluetooth Low Energy (BLE) beacons to further enhance connectivity and location-aware services.
Highlights of Additional Technologies
Today’s network fabrics and other major network elements typically come with automation and orchestration functionality. An administrator automates processes and workflows, and then orchestrates them to run automatically, saving staff time and effort. Automation and orchestration can be used for deploying and optimizing both wired and wireless network components, provisioning network devices, providing a consistent RBAC role user experience and much more. Many vendors’ automation and orchestration products are focused on staying within their product line. Because the University wishes to avoid vendor lock-in and be ready for emerging technologies from all vendors, the automation and orchestration solution must be highly flexible and able to function well across vendors.
The University was an early investigator of the NG911 service, which provides enhanced location data for 911 calls made from smart devices. However, locating Wi-Fi softphone callers is still a challenge. The forward-looking network vision includes Real-Time Location System (RTLS) technology as a means to provide the current location of wireless devices to aid in 911 service location, as well as non-emergency uses such as wayfinding, asset tracking and space awareness for parking.
Finally, the network team is building an infrastructure lab for testing and learning about new technologies, and to run proofs of concept (PoCs), prior to implementation on the production network. The infrastructure lab will also provide an ideal sandbox for innovation where researchers develop technologies that could eventually be used at the University and beyond.
Phased Approach to Rollout
The University chose to implement the network redesign project in phases, focusing on the highest impact wireless and security improvements occurring in the first phase. Figure 4 shows major goals for both Phase 1 and Phase 2. Key to the plan is to avoid the traditional big-bang updates and architectural changes every 5 to 15 years.
Figure 4: Overview of phased approach
Transitioning from a flat network topology with a single core location (a potential point of failure) to a flexible network fabric designed for growth will meet the University’s educational and research needs well into the future. Services across a fabric are more capable and efficient, and more easily managed from a central console, replacing many manual administrative tasks with automation and orchestration to reduce errors, improve service delivery and free staff to focus improving capabilities.
Integrated RBAC for Wi-Fi users and devices, the ability to micro-segment the network, N+1 (resilient) borders with firewalls and a DDoS mitigation service will substantially increase the network’s security posture. Improvements to the logging and monitoring infrastructure will offer deep insights into network traffic, enabling University staff to identify performance issues and bottlenecks more quickly, as well as protect and optimize applications.
The addition of thousands of wireless APs and eventual upgrade to 802.11ax will provide dense coverage at higher speeds, supporting users on any Wi-Fi‒enabled device anywhere on campus.
Automation and orchestration boost operational efficiencies, and provide better quality assurance and quality control. Transforming manual operations and business rules into automated IT services minimizes human error, allows IT staff to focus on higher priority tasks, and standardizes the overall network design.
Finally, an essential reset (fresh start) of the entire network infrastructure allows the University to adopt a proactive continuous improvement model (CIP), in which regular reviews are conducted to ensure implementation phases are on track, that bandwidth metrics align with needs, and to identify new technology that should be incorporated to maintain an infrastructure enabling the institutional mission.
Table 1 summarizes major network design elements and their expected benefits.
Table 1: Major Design Elements and Expected Benefits
|Network fabric and RBAC solution||Micro-segmentation of network|
|Elimination of single points of failure and large fault domains|
|More capable and efficient services, more easily managed|
|Elephant flow support (research computing)|
|Designed for growth|
|Support for legacy back-end systems, such as building automation systems (BAS)|
|Role-based access (identity management); controlled access to network resources through authentication, based on user and device roles|
|Border firewalls and routers||Automatic failover, resiliency|
|Enhanced border protection|
|Wi-Fi upgrades||Dense Wi-Fi coverage at higher speeds|
|Meet increased usage demands from mobile and Internet of Things (IoT) devices|
|Network data monitoring, packet broker, distributed sensors||Capability for deep network traffic analysis|
|Automation and orchestration||Improved service delivery; increased operational efficiencies|
|Replace many manual administrative tasks; error reduction|
|Log aggregation and SIEM||Capability for deep network traffic analysis|
|Faster identification of performance issues|
|DDoS mitigation||Enhanced border protection|
|Substantially reduce effects of DDoS attacks|
|DNS intelligence-based security||Prevent users from accessing compromised websites|
|Wi-Fi analytics||Improved customer experience|
|Ability to quickly pinpoint issues for proactive resolution|
|Security analytics||Enhanced network security|
|Ability to quickly pinpoint issues for proactive resolution|
|Application analytics||Visibility into application use across the network|
|Applications protected and optimized|
A network redesign is complex. It involves every part of a campus and requires input from key personnel from many different departments. The project has to be carefully planned and implemented to minimize disruption to users. The process often takes longer than initially anticipated, and requires a commitment of time from staff who are already dedicated to other priority projects. The complexities of the University’s network redesign were amplified, in part, due to the accelerated schedule for completion. To alleviate some of the pressure, the University expects to rely heavily on vendors’ professional services to handle a significant portion of the network implementation tasks.
A critical decision to be made is the selection of the network fabric and RBAC vendor. What is the best solution that will meet requirements and future-proof their investment? University network staff and stakeholders carefully weighed the pros and cons of multiple vendors versus a single vendor, gaps in desired functionality, maturity of each solution and interoperability issues with the existing Wi-Fi infrastructure. The team also hosted several multi-vendor design meetings to confirm and further detail relevant design areas, and then revised the overall network design based on those meetings. Even so, a clear solution wasn’t apparent. The University continues to research the solutions and will perform PoCs before selecting which network fabric and RBAC solution to implement.
The University’s Ivy League status and world-class reputation was a major benefit during the planning phase. Leading vendors flew in for multiple on-site design sessions and provided detailed proposals that helped the team narrow their choices and form the network design. Smaller or less well-known institutions can struggle in this respect, and may need to rely more on peers as sources of information for similar projects.
The University recognized the value of an outside driver (the consulting firm), without which the planning phase would not have been as efficient or even successful. The consulting firm assessed the existing network and processes, coordinated vendor meetings and proposals, refined the results of design meetings, and provided an assessment a choices for the new network.
Private university in the U.S. Northeast
Implement a next-generation campus network
Integrate security throughout design
Employ Wireless First strategy
Improve network speed, capacity and resiliency
Research on and using the network
Considerable growth, including a new campus under construction
Significant deferred maintenance on a large enterprise scale
Achieve considerable value within a short time frame
Network fabric and RBAC
Resilient campus borders with routers and firewalls
Intrinsic security throughout the network
Expanded and upgraded Wi-Fi coverage
Automation and orchestration
Flexible, resilient network designed for growth
Dense Wi-Fi coverage at higher speeds
Improved operational efficiencies