Apollo x16D: Troubleshooting Dante Connections with Dante Controller

Table of Contents

Connectivity

Check the UAD meter & Control Panel application to verify the Apollo x16D is connected and detected. Verify the x16D is physically connected to the other Dante devices in the network by Ethernet. The network cable needs to be CAT5e or better for a 1Gbps network connection.

Open the Dante Controller application, and check to see if the expected devices are all present.
Can Dante Controller see other devices and can Dante devices discover (see) each other? Are all devices set to the same sample rate?  Note that only some Dante devices are capable of sample rates above 48 kHz.

Devices that can't communicate with each other can't elect a clock leader or establish subscriptions. The network status and clock status panes in Dante Controller are especially helpful here:

1_x16D_DC_NetworkView1.png

2_x16D_DC_NetworkView2.png

Once you have established that the devices are detected, verify that they are routed correctly. For x16D to utilize low latency operation through UAD Console, you will need to use Flex Routing to assign Dante output channels from UAD Console:

Commonly, a 1:1 patching is desirable, and such diagonal multi-subscriptions can be achieved by holding down the Ctrl key and clicking the [-] symbol at the top left corner of the intersection between two devices (note that the - symbol might be vertical or horizontal):

If you are unable to see the connected devices or make successful subscriptions, you will need to work through the remaining troubleshooting steps.

Physical Issues

In the event of a physical issue, things to look out for are:

  • Are network activity lights flashing on both the device and switch?
  • Is the Dante Device powered?
  • Does the unit require PoE (Power Over Ethernet) and is it getting enough supply from the switch?
  • Is the network cable category (CAT 5e or better) or fiber cable bad? (Use a tester, or simply try another known good cable.) If the suspected faulty cable run goes through conduit or walls, run your testing cable across the floor.
  • Is the unit plugged "all the way in" to the port to make the necessary connection?
  • Are the copper contacts or optical ends dirty or oxidized, potentially disrupting the signal?
  • For software like Dante Controller or Dante Virtual Soundcard, is the application assigned to the correct network port on your machine?

In Dante Controller, you can change the selected Ethernet port to manage devices connected to that port:

5_x16D_DC_UnmanagedMode.png

Logical Issues

  • Ensure that basic connectivity to the device is established. Once the network port is active, the next step is to get Dante discovery happening.

  • Connect your computer to the same network, if VLANs are in use on a switch. Make sure that all devices are connected to the same VLAN, and ensure Dante discovery is possible.

  • Initially, start with the basics:
    1. Close and re-open Dante Controller.
    2. Reboot the computer.
      Bear in mind network switches and Dante devices will take a while to power on and discover each other.
  • If you suspect a problem with the network switch, try connecting directly to the device or through a known good unmanaged switch. Check that the device is configured on the right subnet or the right VLAN.

  • Perform a ping test. To do so, follow the steps below:

macOS

  1. On macOS, open Applications > Utilities > Terminal.
  2. Type ping -c <number of times to ping> <IP address>

Windows

  1. To bring up the Run dialog, press the Windows key + R.
  2. Type cmd and press Enter.
  3. Type ping <IP address> and press Enter.
  • Check if there is a duplicate MAC or IP address or Dante device name causing a functional conflict on the network.

  • Check if the Dante device is secured or hidden with Device Lock, Dante Director or Dante Domain Manager.

Recovering Secured/Locked Devices Dante devices can be achieved in two basic ways:

Device Lock

Device Lock prevents people from making changes to a device by requiring operators to enter a 4-digit code. Unfortunately, this doesn't limit access to the person who entered the code, you can only unlock the device (at which point anyone can make changes), then lock the device again when you're done. But this is a simple, lightweight security measure, especially helpful for things that should not change often. 

Devices can also be enrolled in Dante Domain Manager (DDM) or Dante Director. These systems will hide the Dante Devices from the network, only allowing an operator to see the devices when they log in to the server and enter credentials.

This will keep the device secured while authorized people are able to work on it. In both cases, there are easy ways to release devices from Device Lock or DDM/Dante Director. However, let's suppose a device was secured with one of the methods above and you don't remember the Device Lock PIN, or you don't have access to the DDM or Dante Director server anymore. 

Now you're in a recovery situation. Regardless of which system was used, the first step is to prove you have physical access to the unit and wish to override the security settings.

Lonely Reset – Isolate the Device

Lonely Reset is a process by which you demonstrate you have control of the device. Essentially, you create a network where only the Dante device and your computer with Dante Controller are present. There should be no other Dante devices present

If your laptop is running an interface like Dante Studio or DVS, these will need to be turned off. Then you will be able to bypass the security on the device.

Note: If the device doesn't require PoE to run, this can be done simply by plugging your computer directly to the Dante device. If it does require PoE, you could use a PoE injector or an inexpensive, known-to-work unmanaged switch. (A small, unmanaged switch is a great tool for your troubleshooting toolkit.)

Once the device and Dante Controller have been the only things on a network for at least two minutes, go to (Device View) Devices > Lock/Unlock, and Use the "Forgot PIN" option in the panel.

Once the device has been reset, you should re-connect it to the Dante network and assess if it is working as expected.

If the device was previously connected to Dante Domain Manager, you may also need to clear domain credentials for deep troubleshooting.

Clear domain credentials.png

  • Additional steps to consider:
  • Check if devices or streams are blocked on the network with ACLs (Access Control Lists on the switch) or from IGMP snooping without a querier.
    Note: When using IGMP Snooping on a network, it is essential that there is only one device set as querier, which can be configured from your network switch interface.
  • For Dante Controller on a computer, check if there is any security software blocking traffic. Try (temporarily) disabling firewalls on the computer to see if that frees things up.
  • Try reinstalling Dante Controller to see if an OS or software update caused an issue. 
  • For systems controlled by a Dante Domain Manager (DDM) or Dante Director server:
    1. Place your computer in the same VLAN as the Dante device. Try to duplicate the network path as much as possible, at least on the same switch, if not through the same port, by inserting a switch.
    2. See if the device shows up in Dante Controller without being logged in to the server. If it does, it may have been accidentally removed from the server.
    3. Try to log in to DDM or Dante Director with Dante Controller. If you can't, then you need to trace the path from your computer to the server.

Clocking

Once devices can see each other, the next step is to ensure clocking is solid. Devices that are not synchronized will not play audio or video – their outputs are muted.

Does the system resolve to one clock leader? For devices to properly sync, everything must resolve to one clock leader. If different devices follow from different, unresolved clocks, they will drift from each other. Be sure to follow the chain – find the "boundary clocks" which received time from a clock leader, then retransmit. You just need to ensure the time ultimately cascades down from a single time reference. Do you see one Primary Leader Clock at the top of the screen? Go to Dante Controller Network View > Clock Status.

7_x16D_DC_NetworkView3_ClockStatus.png

In the four columns for Primary/Secondary v1/v2 Multicast, is there a sensible chain? (If Dante Domain Manager is involved, you may need to check clocking status there, as well.) If you have boundary clocks, then you're using DDM.  Note that Dante Domain manager is currently unsupported with Apollo x16D.

Clocking Stability

  1. Go to Dante Controller Network View > Clock Status
  2. Check if the sync indicators for the devices show green.
    Note: Subscriptions are not required for clocking.
  3. In the bottom right, click on Clock Status Monitor, and pull up the History for any device in question. Ideally, you should see a single column or at least a tightly grouped set of readings. Let the system "settle" – clocks will adjust in the first 30 seconds to 5 minutes. 

The Dante PTP clocking system gradually and dynamically adjusts clock sync to prevent audio interruption. If a clock is completely out of bounds, the device will be muted until it re-clocks successfully.

A few spikes with "blades of grass" between them likely means a clock leader changed… clear the chart and watch again for stability.

A gap in the diagram might indicate a device chasing an external clock (or GPS clock) took over, and all devices had to mute to resync. For problems, consider topology, optimization and hardware.

In Dante Controller Network View > Clock Status, Ensure the device has negotiated the expected port speed. QoS (Quality Of Service settings in a managed switch) can improve clocking stability on saturated links by prioritizing PTP packets. See if there is any commonality on shaky groups. Are they on the same switch, or located down the same trunk line? If so, check bandwidth utilization on these links. Consider more bandwidth (LAG or higher bandwidth ports) and QoS optimization.

If the shaky follower is running a software interface (like Dante Virtual Soundcard), ensure the NIC (Network Interface Card) is hardware-based. If in doubt, try a USB-to-Ethernet adapter. Check the computer's performance monitor to see if the computer is being overtaxed.

How many network hops are between the leader and problem device? If this is a gigabit connection and the path has enough bandwidth, 20 hops might be a "best practice" limit.

How many devices are following the clock leader? An Ultimo chipset may only support 20-40 devices, while most Gigabit solutions can support 250-350.  The Apollo x16D utilizes a Brooklyn Dante chipset, so it makes an excellent clock leader.

For more information, read this article.

Subscriptions

Now that we know devices can discover each other and are properly clocked, the next step is to check that devices are attempting to make their subscriptions. The subscription doesn't have to be working properly for this to pass, we are just looking to see if Dante Controller reports any problem:

View Subscriptions in Dante Controller in screens like Network View > Routing to view multiple devices. Look in Device View > Receive to view one device at a time. Focus on troubleshooting subscriptions that don't have a green checkmark.

Note: The following icons aren't errors, they are informational: 

In Progress – Shown when a subscription is requested for up to several seconds…  

Pending – Shown if the subscription is taking a longer period of time to establish. This is not unusual if many subscriptions are created at once, if the network is very large, if networks are managed (or some combination thereof).

The warning icon would be shown on the receiver if a problem has occurred, such as the following:

  • Source Device Cannot be Found: Commonly, this would mean the device is not online or the subscription was made and then the transmitter's Dante Device Name was changed after the fact. The source device and channel will be revealed by rolling over the icon in the following format: @  Problem with Subscription. Roll over the icon, and you will see a message.
  • Incorrect channel format: Source and destination channels do not match. This will vary based on channel format. For instance, on an audio channel the devices are likely different sample rates. For video, it could be incompatible framerate, resolution, aspect or video codec.
  • Mismatched clock domains: The transmitter and receiver are not part of the same clock domain. This message is possible when using Dante Domain Manager for separate domains. Domains must be set for a common clock to share audio or video between them – this likely indicates a break in the chain to a common clock. Check your unicast clocking or GPS clock zones to ensure a common clock.
  • Tx Scheduler Failure: This likely means you are trying to use sub-millisecond latency over a network less than 1Gbps. Ensure network links are gigabit or better, or select a longer latency setting.
  • No receive flows:
    No more flows (TX): transmitter cannot support any more flows. There is a limit to how many flows a device can send or receive. Dante devices always have enough flows to transmit and receive all channels, and it automatically manages these flows for you. However, after many changes, there are paths the data can take that might inefficiently pack the flows. Restarting the transmitter and/or receiver forces these flows to be restructured. As a follow-up, it may be time to consider multicast within a subnet. If that isn't possible, try sending the signal to a device with more flows, and have that device act as a splitter.
  • Cannot change: Locked receiver:
    This message will appear if you attempt to make or break a subscription between transmitter and receiver when the receiver is locked.
  • Warning: Locked transmitter:
    This message will appear if you attempt to make or break a subscription between transmitter and receiver when the transmitter is locked.
  • Access control failure:
    Transmitter is locked. This indicates the receiver believes the subscription has been successful in the past, but the transmitter does not believe it was authorized. This mismatch could be caused by changing the name of the receiver after locking the transmitter

Articles in this section