TNMA Case Study
Problem
U. S. Central Command (CENTCOM) was mandated in August 2006 by Joint Task Force - Global Network Order (JTF-GNO) to have an enterprise network monitoring and management capability providing a network common operating picture (NETCOP) and fault management, configuration, accounting, performance and security (FCAPS). Theater engineers traditionally performed cursory network management through rudimentary tools with no hierarchical design. Configuration and access limitations often made access to remote servers and appliances unsuccessful. Coordination with engineers throughout CENTCOM's area of responsibility (AOR) often was conducted via phone, email or fax to simply enable logins and read/troubleshoot appliances. Administrators and engineers on CENTCOM's NIPRNet and SIPRNet, the non-secret and secret networks, were unable to properly monitor those networks. Results were time-consuming, difficult and insufficient. CENTCOM lacked real-time operational and situational awareness.
Vision
Establish a centralized system providing situational awareness for Tier 1 network layer and mission critical services across the theater, supporting net-centric operations and a common, joint enterprise architecture. The result should be centrally managed monitoring capability, resulting in improved network health, optimization, and reliability. It should provide network managers the ability to proactively monitor and/or manage network resources in both a regionally focused and an enterprise-wide manner.
Analysis
Operational awareness must go beyond ordinary situational awareness. Both measure the current situation and model what is going on, but operational awareness determines how the incident or problem affects the warfighter. CENTCOM had a collection of unrelated vendor tools, business processes, organizational structures and knowledge elements. Inaccurate, untimely, and/or incomplete operational awareness was compromising command and control functions. Slow or failing components and intrusions were being detected in a makeshift fashion that was inconsistent, misleading, tedious and susceptible to human error. A tool set was needed to provide an enterprise-wide solution ensuring synchronization among all levels while providing accurate data to all levels of network management from the field to CENTCOM HQ. That tool set is Theater Network Management Architecture (TNMA).
The TNMA Solution
Project managers and engineers researched available commercial off-the-shelf (COTS) and government off-the-shelf (GOTS) products. Products chosen for the TNMA suite were: Netcool (manager of managers; event management; executive reporting); TACACS (authentication, accounting and authorization of Tier 1 network devices); ArcSight (collect syslog and security events from IDS/Firewalls; correlate security events; interface with TACACS for authentication; forward high-interest and correlated events to Netcool via UDP Ports 161/162); Cisco Network Compliance Manager (change/configuration management of Tier 1 network devices; send configuration/change events to Netcool via UDP ports 161/162); NetQoS (collect network statistics via netflow; provide performance reporting via web interface to Netcool's dashboard (TCP port 80)); Packet Design — provides routing analysis and monitoring of Tier 1 network devices to include routing modeling/simulation and path analysis; forwards routing events to Netcool via UDP ports 161/162. The following milestones were established and completed to bring the system from Initial Operating Capability (IOC) to Final Operating Capability (FOC):
- Multi-Platform Open Systems Interconnect (MTOSI) Alarm Service
- Integration of Security Event Alarms
- Discovery of Tier 0 (Class Files)
- Creation of Video Teleconferencing Node View
- Enrichment of Events
- Portal Stabilization
- Improvement and Monitoring of Performance Tool
- User Role Creation
- TNMA Compliance Process
- TNMA Development Lab Standup
- TNMA Accreditation
- TNMA Curriculum
- Dashboard Transformation
- Single Sign-on
- Knowledge Database
- VoIP Monitoring and Management
- Continued Operations Plan/Disaster Recovery
- MTOSI Configuration Service
- MTOSI w/Secure Socket Layer (SSL)
TNMA was implemented into CENTCOM's NIPR and SIPR enclaves, providing complete and ubiquitous situational awareness of network health and meeting the objectives of FCAPS. Management of each of the five levels of FCAPS (F-fault management; C-configuration; A-accounting or allocation; P-performance; S-security) is imperative to have an effective enterprise network management capability.
At the F level, network problems are found and corrected. Potential future problems are identified and steps taken to prevent the from occurring or recurring.
At the C level, network operation is monitored and controlled. Hardware and programming changes are coordinated, including the addition of new equipment and programs, modification of existing systems, and removal of obsolete systems and programs.
At the A level, resources are distributed optimally and fairly among network subscribers to make the most effective use of the systems available. The goal is to minimize the cost of operation while ensuring service levels are met.
At the P level, the overall performance of the network is managed. Throughput is maximized, bottlenecks avoided, and potential problems identified. A major part of the effort is to identify which improvements yield the greatest overall performance enhancement.
At the S level, the network is protected against hackers, unauthorized users, and physical or electronic sabotage. Confidentiality of user information is maintained where necessary or warranted. The security systems also are designed to allow network administrators to control what each individual authorized visitor can (and cannot) do with the system.
The TNMA components provided an enterprise-wide solution for the FCAPS model. In addition, the TNMA tools provide a NETCOP for visibility of Tier-1 network information and Tier 0 network throughout the AOR. Reports resulting from collection of Tier-1 network configuration and utilization provide the ability to proactively conduct performance management and network analysis, facilitating the expansion of emerging network technologies such as Voice over Internet Protocol (VoIP) and Video Telephony.
End-to-end network performance, traffic analysis, and device performance data is presented in a customizable, web-based interface. Event management tasks are automated rather than requiring processing of each alert by hand, allowing network operators to spend time on higher priority tasks. Network support staff can identify and correct trends that could lead to problems such as degraded security posture, network instability and service interruptions. A single appliance provides visibility into the dynamic routing operation of the entire network, allowing engineers to view the real-time routing structure of the entire network as a seamless topology map, even when the network is running multiple protocols and spans multiple domains.
Conclusion
TNMA was developed and implemented in less than a year, beginning with conceptualization in May 2007 and becoming fully operational in March 2008. It provides a turnkey solution with a greater level of fidelity to Defense Information Systems Agency (DISA) Net-Centric objectives than previously available tools. TNMA is a hierarchical architecture with customized implementation at multiple levels. Its design increases scalability of the solution without adversely affecting its supportability. Through use of commercial off-the-shelf components and open standard software, a substantial cost-savings was achieved, reducing the cost of each suite from approximately $6.5 million to $2.6 million. The design is modular and works with standard rack unit configurations, requiring less space than previous equipment configurations. Additional cost reductions are possible, while the design allows for future functional capabilities.