AI training, inference, and high-performance computing workloads are placing new demands on data center networks. Large clusters of accelerators must exchange enormous amounts of data while maintaining high utilization, low latency, predictable completion times, and reliable communication across increasingly complex fabrics.

Traditional Ethernet provides the open ecosystem and scalability needed for these environments, but AI and HPC traffic patterns create challenges that go beyond simply increasing link speed. Bursty collective communication, congestion, multipath traffic, packet reordering, and communication among very large numbers of endpoints can all affect application performance.

Ultra Ethernet addresses these requirements with an Ethernet-based communications architecture optimized specifically for AI and HPC. At the center of the architecture is Ultra Ethernet Transport (UET), which introduces new approaches to RDMA transport, congestion management, multipath communication, delivery semantics, and network scalability.

For verification teams, these capabilities create an important challenge: a design must be validated not only for correct packet transmission, but also for how transport behavior, congestion control, ordering, errors, and network conditions interact across a large-scale fabric.

Why Ultra Ethernet Matters for AI and HPC

Distributed AI workloads depend on communication between large numbers of GPUs, accelerators, CPUs, and memory resources. The network connecting those components can directly influence how quickly an AI training job or HPC workload completes.

The Ultra Ethernet Consortium is developing an open, interoperable Ethernet architecture designed specifically for high-performance AI and HPC networking.

The consortium’s current published specification is Ultra Ethernet Specification 1.0.3. The architecture extends beyond a single protocol and addresses multiple layers of the communications stack, including software interfaces, transport, congestion control, link-layer behavior, security, management, and performance.

One of its central components is Ultra Ethernet Transport (UET), which modernizes RDMA communication for large-scale Ethernet environments.

This is particularly important because AI and HPC traffic can produce synchronized bursts, incast conditions, rapidly changing congestion, and large collective operations that behave differently from conventional enterprise network traffic.

Key Technical Challenges in Ultra Ethernet Verification

Multipath Packet Delivery Changes the Traffic Model

Traditional Ethernet networks commonly use Equal-Cost Multi-Path (ECMP) techniques to assign flows to available network paths. Ultra Ethernet introduces finer-grained multipath behavior in which packets associated with traffic can be distributed across multiple available paths.

This can improve network utilization and reduce the effect of individual congested paths, but it also means that packets may not arrive in the same order in which they were transmitted.

Verification teams therefore need to exercise more than simple send-and-receive traffic. The environment should test:

• Traffic distributed across multiple paths
• Packet reordering
• Different path latency
• Congested and uncongested path combinations
• Loss and retransmission conditions
• Correct data placement despite out-of-order arrival

The DUT must maintain correct transport semantics even while the network dynamically changes how packets reach their destination.

Congestion Control Must Respond to AI Traffic Patterns

Congestion is particularly important in accelerator networks because many endpoints may communicate with the same destination at nearly the same time.

Ultra Ethernet includes congestion-control mechanisms designed to react quickly to these conditions and make better use of available paths. UET includes sender-based congestion management and also defines optional receiver-driven mechanisms that can control how much traffic senders are permitted to inject toward a receiver.

From a verification perspective, congestion control should not be treated as a single steady-state feature.

Teams need to create changing conditions such as:

• Sudden incast traffic
• Rapid changes in available bandwidth
• Congestion on only one of several paths
• Packet loss or trimming
• Multiple senders competing for receiver resources
• Recovery after congestion clears

The environment must then confirm that the DUT reacts correctly without introducing deadlock, uncontrolled retransmission, excessive packet loss, or incorrect flow behavior.

Different Transport Services Create Different Ordering Requirements

Not every AI or HPC workload has the same ordering requirements. Some applications require strict message ordering, while others can tolerate packet reordering as long as the complete data operation is eventually delivered correctly.

UET supports multiple delivery models so implementations can better match transport behavior to workload requirements.

Verification teams therefore need to ensure that the DUT applies the correct semantics for the transport service being exercised. A sequence that is acceptable under an unordered service may represent a protocol error in a service that requires ordered delivery.

Coverage should include transport service, packet order, message size, network path, loss condition, and completion behavior rather than verifying each feature independently.

Scalability Changes Endpoint-State Verification

Large AI clusters may contain thousands or substantially more communicating endpoints. Maintaining large amounts of persistent per-peer state can become expensive in hardware and difficult to scale.

UET is designed to reduce this dependency by allowing connection resources and state to be associated more closely with active transactions rather than requiring extensive permanent state for every potential communication peer.

This creates additional verification requirements around transaction initialization, active-state management, completion, resource reuse, and communication between changing endpoint combinations.

Verification should include cases where connections or transaction resources are created and released rapidly, many peers become active, and resources approach implementation limits.

Loss and Error Conditions Must Be Tested Deliberately

Ultra Ethernet is designed to operate efficiently over Ethernet environments where packets may be dropped because of congestion or link errors. This makes error recovery and reliability behavior important parts of the verification plan.

Teams should deliberately introduce conditions such as packet loss, corruption, duplicate traffic, out-of-order packets, link errors, and congestion-related events.

The objective is not simply to confirm that communication eventually recovers. Verification should determine whether loss is detected correctly, whether retransmission occurs according to the selected transport behavior, whether data is placed correctly at the receiver, and whether application-visible completion remains valid.

How Verification Teams Should Approach Ultra Ethernet

Build Protocol-Aware Endpoint Models

A verification environment should include configurable endpoint behavior capable of generating realistic UET traffic rather than relying solely on generic Ethernet frames.

Endpoints should be able to exercise different transport operations, message sizes, ordering requirements, traffic rates, and network conditions.

This allows the verification environment to model the workloads and communication patterns that the protocol was created to support.

Combine Transport and Ethernet Verification

Ultra Ethernet operates on an Ethernet foundation, so verification must maintain visibility across multiple layers.

An error observed at the transport layer may originate from link behavior, while a link-level condition may affect congestion management or transport recovery.

SmartDV’s Ethernet – Up to 800G VIP provides verification support across high-speed Ethernet MAC and PCS environments, including error injection, protocol checking, and functional coverage.

This layered approach helps teams distinguish between Ethernet-level issues and UET-specific transport behavior during debug.

Exercise Congestion Dynamically

Static bandwidth limits are not enough to validate congestion control designed for AI networks.

Verification should vary traffic patterns over time and create situations where one path becomes congested while another remains available. The DUT can then be checked for appropriate path selection, rate adjustment, recovery, and overall transport behavior.

This type of scenario is particularly valuable because many difficult congestion problems emerge from interactions among several senders and receivers rather than one isolated flow.

Measure Cross-Feature Functional Coverage

Individual feature coverage does not necessarily demonstrate that realistic combinations have been tested.

Useful Ultra Ethernet cross coverage can combine:

• Transport service type
• Message size
• Number of active endpoints
• Path count
• Packet ordering
• Congestion condition
• Error type
• Recovery behavior

This helps expose untested combinations where multiple independently correct features interact.

The same principle applies broadly to complex SoC verification. SmartDV discusses this in How Reusable Verification IP Supports Scalable SoC Verification.

Ultra Ethernet, RoCE, and the Evolving AI Network

RDMA is already widely used to reduce software overhead and move data efficiently between systems. RDMA over Converged Ethernet (RoCE) provides RDMA communication over Ethernet and remains an important technology in high-performance networks.

Ultra Ethernet takes a broader approach by defining a transport architecture designed specifically around the scalability, congestion, multipath, and workload requirements of next-generation AI and HPC systems.

For verification teams, this means existing expertise in Ethernet and RDMA remains useful, but UET-specific transport behavior requires additional modeling, stimulus, checking, and coverage.

Related SmartDV Products and Internal Resources

SmartDV’s Ultra Ethernet VIP supports verification of Ultra Ethernet endpoints implementing UET, modern RDMA behavior, congestion-control mechanisms, multipath packet distribution, error injection, and functional coverage.

SmartDV’s current Ultra Ethernet VIP product information identifies support for UEC Specification 1.0.3. Because the Ultra Ethernet Consortium continues to publish specification revisions, engineering teams should confirm the precise specification revision required for their target implementation when planning a verification program.

AI and HPC systems also rely on multiple complementary high-bandwidth technologies. SmartDV’s article HBM and LPDDR6 Verification Challenges for AI, HPC, and Edge Computing examines the memory side of these architectures, while Why UCIe Verification Is Critical for Chiplet-Based SoC Design covers die-to-die connectivity within advanced multi-die systems.

Article Summary

Ultra Ethernet is designed to extend Ethernet for the communication patterns created by large-scale AI and HPC systems. UET introduces transport behavior optimized for RDMA, multipath packet delivery, congestion management, flexible ordering, scalable endpoint communication, and reliable operation over demanding network conditions.

Those capabilities also expand the verification problem. Teams need to validate how transport services behave across changing paths, congestion states, endpoint combinations, ordering conditions, packet loss, and recovery scenarios.

Effective Ultra Ethernet verification therefore requires protocol-aware endpoint models, layered Ethernet and UET checking, dynamic congestion testing, deliberate fault injection, and functional coverage that measures interactions across transport and network conditions rather than isolated features.

Frequently Asked Questions

What is Ultra Ethernet?
Ultra Ethernet is an open Ethernet-based communications architecture developed by the Ultra Ethernet Consortium for high-performance AI and HPC networks. It includes Ultra Ethernet Transport and enhancements across transport, software, congestion management, link behavior, security, and related networking functions.

What is UET?
UET stands for Ultra Ethernet Transport. It is the transport protocol at the center of the Ultra Ethernet architecture and is designed to support high-performance RDMA communication, scalable endpoint interaction, multipath networking, congestion management, and transport services suited to AI and HPC workloads.

Why is multipath verification important for Ultra Ethernet?
Ultra Ethernet can distribute packets across multiple network paths to improve utilization and reduce congestion. Verification must confirm that the implementation behaves correctly when packets arrive out of order, paths have different latency, congestion moves between paths, or packets are lost.

Why does congestion control matter for AI networks?
AI workloads can generate synchronized, high-bandwidth traffic from many accelerators at once. These patterns can create rapid congestion and incast conditions. Verification needs to confirm that congestion-control mechanisms respond correctly as network conditions change.

Can Ultra Ethernet operate on existing Ethernet infrastructure?
Ultra Ethernet builds on standard Ethernet and is designed to maintain Ethernet interoperability while introducing endpoint, transport, and optional network and link enhancements optimized for AI and HPC. Existing Ethernet infrastructure can therefore remain part of an Ultra Ethernet architecture, although the features and performance available depend on the capabilities implemented by the endpoints and network.

Verify Ultra Ethernet Designs with SmartDV

As AI and HPC networks scale, verification must extend beyond basic Ethernet connectivity to address transport behavior, multipath traffic, congestion, ordering, reliability, and interaction among large numbers of communicating endpoints.

SmartDV provides Verification IP for Ultra Ethernet, high-speed Ethernet, RoCE, and other advanced interfaces used throughout AI, HPC, networking, and data center systems.

» Contact SmartDV to discuss Ultra Ethernet, UET, Ethernet, RDMA, or Verification IP requirements for your next SoC, ASIC, or FPGA project.