Software bugs are a common cause of packet loss, so for a low-latency solution, load testing is always a good starting point.
As a refresher, a load test is where you generate test traffic using one or more different machines. The traffic passes a sequence of data packets until the total size of the packets is a random number. This data can be treated as an expected stream of packets, and the packets are then transmitted to a collection of network nodes along the path the traffic took.
The same concept can be applied to ACKs and control responses. Assume that you have a 1-port switch. Every time the switch has to respond to a message, it calls that message into the switch. When that port receives a non-ACK, it sends a reply to the sender. The idea is to treat the response as an expected sequence of packets.
The big problem with ACKs is that they can be misordered. If an ACK is returned late from a router, the router will have to follow it back up the path that the ACK was sent from, causing a packet loss. That loss is caused by the latency of the path from the ACK to the last hop of the circuit, but is also a latency of the ACK itself. As soon as a router receives the reply, the router can use the time of the ACK to calculate the maximum latency the ACK travel on the path. This calculation works out the worst case latency for the route.
To do load testing, you need to configure the router to follow up on any non-ackendied ACKs. It’s recommended to add a default rule in the load distribution table (LDP) at the time of the ACK, so that it will always follow the router up the path. This will allow you to use the load-testing parameter to select the best possible router. Before doing this, make sure to back up your data in the cloud (which you can find at sites like https://venyu.com/cloud/) to avoid losing important data.