NTP Server and DXi Time Synchronization Troubleshooting

Overview

Quantum recommends that users configure DXi systems to synchronize their system date and time with Network Time Protocol (NTP) servers. When troubleshooting time issues in

non-standard configurations, an understanding of some of the NTP principles and troubleshooting tools can be useful. This article uses one customer scenario as a framework to

explain those principles and show an example of using those tools.

The Scenario

In this example, a customer has blocked most of the NTP pool servers on their local internal network. They decided to set up their own NTP servers, which set the time base for

all their internal systems. If customers choose to do this, they must be careful creating their own NTP servers, and must check the server accuracy and the subdomain delays.

Service has found time base stability problems when using Virtual Servers for NTP servers.

In the cases examined in this article, one system drifted and another received bad time offsets via the ntpd process.

NTP Overview

See the overview of NTP at https://en.wikipedia.org/wiki/Network_Time_Protocol
View the diagram at https://upload.wikimedia.org/wikipedia/commons/8/8d/NTP-Algorithm.svg
A typical NTP client will regularly poll three or more servers on diverse networks. To synchronize its clock with a remote server, the client must compute the round-trip delay time and the offset.
The round-trip delay δ is computed as

Where:

t₀ is the client's timestamp of the request packet transmission
t₁ is the server's timestamp of the request packet reception
t₂is the server's timestamp of the response packet transmission
t₃ is the client's timestamp of the response packet reception

Refer to http://doc.ntp.org/4.1.2/debug.htm.
Verify correct operation using the ntpq and ntpdc utility programs.
Look at the offset and jitter column. Note: read the Debugging Checklist at the bottom of the web page.
A concept that affects the time base is that the estimated time error of the peer clock is measured as an exponential average of RMS time differences.
In summary, the ntpd daemon polls the NTP time server several times and averages the results.

Case #1

The first system at this customer site was showing: time is off, should be 13:20:20+ DDT a delta of 35 min.

Why? These IP NTP servers cannot be reached from the customer's internal network.
The webgui has a class of NTP pool servers configured.
This shows in the file below.
None of these NTP servers can be reached from the customer's network; they are all blocked by the customer.

[~]# cat /etc/ntp/step-tickers
192.43.244.18
208.66.174.71
66.187.233.4
75.13.24.211
127.127.1.0

Here is another example DXi system, also internal, but with a reference to an internal NTP time server at 198.182.130.79.

[~]# ntpdc -p
     remote           local      st poll reach delay   offset    disp
=======================================================================
=LOCAL(0)        127.0.0.1       10   64 377 0.00000 0.000000 0.03076
*198.182.130.79 172.18.65.121    2 1024 377 0.00023 0.002008 0.12180

[~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset jitter
==============================================================================
*198.182.130.79 192.5.41.41      2 u 505 1024 377    0.238    2.008   1.519
LOCAL(0)        .LOCL.          10 l    8   64 377    0.000    0.000   0.001

Determine the number of hops:

[~]# traceroute 198.182.130.79
traceroute to 198.182.130.79 (198.182.130.79), 30 hops max, 40 byte packets
1 172.18.64.2 (172.18.64.2) 0.576 ms 0.540 ms 0.531 ms
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *

Look for a closer NTP time server with fewer hops:

[~]# traceroute 172.20.103.5
traceroute to 172.20.103.5 (172.20.103.5), 30 hops max, 40 byte packets
1 172.18.64.2 (172.18.64.2) 0.596 ms 0.555 ms 0.546 ms
2 * * *
3 * * *
4 10.227.9.18 (10.227.9.18) 0.941 ms 1.442 ms 1.431 ms
5 10.156.32.15 (10.156.32.15) 26.191 ms 26.203 ms 26.719 ms
6 172.20.103.5 (172.20.103.5) 27.359 ms 26.091 ms 27.180 ms
7 172.20.103.5 (172.20.103.5) 27.683 ms 27.674 ms 27.666 ms

Case #2

From the messages system with +/- 0.5 sec time adjustments:

May 14 00:07:56 blyshrQD6702-03 ntpd[10493]: time reset -0.498794 s
May 14 02:44:58 blyshrQD6702-03 ntpd[10493]: time reset +0.499945 s
May 14 03:26:02 blyshrQD6702-03 ntpd[10493]: time reset -0.497612 s
May 14 05:18:19 blyshrQD6702-03 ntpd[10493]: time reset +0.498591 s
May 14 07:31:42 blyshrQD6702-03 ntpd[10493]: time reset -0.497412 s
May 14 08:44:04 blyshrQD6702-03 ntpd[10493]: time reset +0.498127 s
May 14 10:23:45 blyshrQD6702-03 ntpd[10493]: time reset -0.499278 s
May 14 11:03:32 blyshrQD6702-03 ntpd[10493]: time reset +0.498536 s

Product Model          = DXi6702
Hardware Configuration = 4RxJ-10G4FC
System Version         = 2.2.1.2
Base OS Version        = 2.2.1.2-9659
Application Version    = 2.2.1.2-50480
SCM Build Version      = Build9
Kernel Version         = 2.6.18-164.15.1.qtm.4

[~]# date
Thu May 14 12:23:15 EDT 2015

What is configured in the webgui?

[~]# cat /etc/ntp/step-tickers
172.20.103.5
127.127.1.0

What do the utilities say could happen for a time shift?

[~]# ntpdc -p
     remote           local      st poll reach delay   offset    disp
=======================================================================
*172.20.103.5    172.20.180.18    2 1024   77 0.00055 -0.499074 0.45096
=LOCAL(0)        127.0.0.1       10   64 377 0.00000 0.000000 0.03085

[ ]# ntpq -p
     remote           refid      st t when poll reach   delay   offset jitter
==============================================================================
*172.20.103.5    129.6.15.28      2 u   46   64 177    0.184 501.592 268.060
LOCAL(0)        .LOCL.          10 l   16   64 377    0.000    0.000   0.001

[~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset jitter
==============================================================================
*172.20.103.5    129.6.15.28      2 u   33   64 377    0.184 501.592 268.068
LOCAL(0)        .LOCL.          10 l    4   64 377    0.000    0.000   0.001

[~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset jitter
==============================================================================
*172.20.103.5    129.6.15.28      2 u 294 1024   77    0.554   -499.07 499.681
LOCAL(0)        .LOCL.          10 l   62   64 377    0.000    0.000   0.001

Definition of the fields:

172.20.103.5 is the ntpserver
refid is 129.6.15.28; current source of synchronization
u=unicast
when = 294 peer was last heard in seconds
reach = 77 – status of the reachability register (see RFC-1305) in octal
jitter is plus minus 0.449 seconds.

Traceroute shows that the jitter is not caused by the network routing to the ntpd server, but that it does take two hops.

# traceroute 172.20.103.5
traceroute to 172.20.103.5 (172.20.103.5), 30 hops max, 40 byte packets
1 (172.20.180.3) 0.586 ms 0.568 ms 0.568 ms
2 (172.20.103.5) 0.551 ms 0.544 ms 0.537 ms

# traceroute -p 123 172.20.103.5
traceroute to 172.20.103.5 (172.20.103.5), 30 hops max, 40 byte packets
1 (172.20.180.3) 0.579 ms 0.578 ms 0.569 ms
2 (172.20.103.5) 0.540 ms 0.533 ms 0.526 ms

See what ping shows. Note: this is only an indicator, it's not the same as the NTP “peer clock measured as an exponential average of RMS time differences”

[~]# ping 172.20.103.5
PING 172.20.103.5 (172.20.103.5) 56(84) bytes of data.
64 bytes from 172.20.103.5: icmp_seq=1 ttl=63 time=0.180 ms
64 bytes from 172.20.103.5: icmp_seq=2 ttl=63 time=0.547 ms
64 bytes from 172.20.103.5: icmp_seq=3 ttl=63 time=0.178 ms
64 bytes from 172.20.103.5: icmp_seq=4 ttl=63 time=0.174 ms
64 bytes from 172.20.103.5: icmp_seq=5 ttl=63 time=0.168 ms
64 bytes from 172.20.103.5: icmp_seq=6 ttl=63 time=0.540 ms
64 bytes from 172.20.103.5: icmp_seq=7 ttl=63 time=0.178 ms
64 bytes from 172.20.103.5: icmp_seq=8 ttl=63 time=0.175 ms
64 bytes from 172.20.103.5: icmp_seq=9 ttl=63 time=0.164 ms
64 bytes from 172.20.103.5: icmp_seq=10 ttl=63 time=0.170 ms
64 bytes from 172.20.103.5: icmp_seq=11 ttl=63 time=0.542 ms
64 bytes from 172.20.103.5: icmp_seq=12 ttl=63 time=0.166 ms
64 bytes from 172.20.103.5: icmp_seq=13 ttl=63 time=0.178 ms
64 bytes from 172.20.103.5: icmp_seq=14 ttl=63 time=0.173 ms
64 bytes from 172.20.103.5: icmp_seq=15 ttl=63 time=0.180 ms
64 bytes from 172.20.103.5: icmp_seq=16 ttl=63 time=0.284 ms
64 bytes from 172.20.103.5: icmp_seq=17 ttl=63 time=0.543 ms
64 bytes from 172.20.103.5: icmp_seq=18 ttl=63 time=0.546 ms
64 bytes from 172.20.103.5: icmp_seq=19 ttl=63 time=0.178 ms
64 bytes from 172.20.103.5: icmp_seq=20 ttl=63 time=0.223 ms
64 bytes from 172.20.103.5: icmp_seq=21 ttl=63 time=0.546 ms
64 bytes from 172.20.103.5: icmp_seq=22 ttl=63 time=0.542 ms
64 bytes from 172.20.103.5: icmp_seq=23 ttl=63 time=0.178 ms
64 bytes from 172.20.103.5: icmp_seq=24 ttl=63 time=0.170 ms
64 bytes from 172.20.103.5: icmp_seq=25 ttl=63 time=0.170 ms

Messages shows the problem has existed for a long time. The end result is:

[~]# grep ntpd /var/log/messages | grep reset
Mar 30 22:51:28 blyshrQD6702-03 ntpd[10481]: time reset +0.499186 s
Mar 31 00:14:43 blyshrQD6702-03 ntpd[10481]: time reset -0.499775 s
Mar 31 01:28:54 blyshrQD6702-03 ntpd[10481]: time reset +0.499013 s
Mar 31 02:07:14 blyshrQD6702-03 ntpd[10481]: time reset -0.500907 s
Mar 31 04:06:26 blyshrQD6702-03 ntpd[10481]: time reset +0.498912 s
Mar 31 04:44:48 blyshrQD6702-03 ntpd[10481]: time reset -0.500551 s
Mar 31 05:35:29 blyshrQD6702-03 ntpd[10481]: time reset +0.502949 s
Mar 31 06:09:54 blyshrQD6702-03 ntpd[10481]: time reset -0.502146 s
Mar 31 07:02:42 blyshrQD6702-03 ntpd[10481]: time reset +0.497280 s
Mar 31 11:05:11 blyshrQD6702-03 ntpd[10481]: time reset -0.506118 s
Mar 31 12:06:24 blyshrQD6702-03 ntpd[10481]: time reset +0.499052 s
Mar 31 12:26:17 blyshrQD6702-03 ntpd[10481]: time reset -0.502444 s
Mar 31 13:11:09 blyshrQD6702-03 ntpd[10481]: time reset +0.501533 s
Mar 31 13:31:53 blyshrQD6702-03 ntpd[10481]: time reset -0.502178 s
Mar 31 17:27:48 blyshrQD6702-03 ntpd[10481]: time reset +0.498319 s
Mar 31 18:06:23 blyshrQD6702-03 ntpd[10481]: time reset -0.498221 s
Mar 31 19:19:40 blyshrQD6702-03 ntpd[10481]: time reset +0.502604 s
Mar 31 21:15:03 blyshrQD6702-03 ntpd[10481]: time reset -0.501686 s

Apr 1 05:37:58 blyshrQD6702-03 ntpd[10481]: time reset +0.501649 s
Apr 1 06:33:38 blyshrQD6702-03 ntpd[10481]: time reset -0.497194 s
Apr 1 09:19:52 blyshrQD6702-03 ntpd[10481]: time reset +0.514199 s
Apr 1 10:58:51 blyshrQD6702-03 ntpd[10481]: time reset -0.498164 s
Apr 1 13:45:54 blyshrQD6702-03 ntpd[10481]: time reset +0.498788 s
Apr 1 14:06:10 blyshrQD6702-03 ntpd[10481]: time reset -0.498109 s
Apr 1 17:52:41 blyshrQD6702-03 ntpd[10481]: time reset +0.500829 s
Apr 1 18:27:57 blyshrQD6702-03 ntpd[10481]: time reset -0.501427 s
Apr 1 18:59:52 blyshrQD6702-03 ntpd[10481]: time reset +0.497665 s
Apr 1 21:34:36 blyshrQD6702-03 ntpd[10481]: time reset -0.500144 s
Apr 1 23:02:37 blyshrQD6702-03 ntpd[10481]: time reset +0.502324 s
Apr 1 23:23:17 blyshrQD6702-03 ntpd[10481]: time reset -0.500831 s

Apr 2 00:40:57 blyshrQD6702-03 ntpd[10481]: time reset +0.495074 s
Apr 2 01:15:37 blyshrQD6702-03 ntpd[10481]: time reset -0.498159 s
Apr 2 03:44:50 blyshrQD6702-03 ntpd[10481]: time reset +0.497566 s
Apr 2 05:20:30 blyshrQD6702-03 ntpd[10481]: time reset -0.500361 s
Apr 2 09:27:52 blyshrQD6702-03 ntpd[10481]: time reset +0.493580 s
Apr 2 09:59:29 blyshrQD6702-03 ntpd[10481]: time reset -0.503104 s
Apr 2 11:06:36 blyshrQD6702-03 ntpd[10481]: time reset +0.493025 s
Apr 2 11:42:06 blyshrQD6702-03 ntpd[10481]: time reset -0.501283 s
Apr 2 14:07:57 blyshrQD6702-03 ntpd[10481]: time reset +0.499732 s
Apr 2 14:19:27 blyshrQD6702-03 ntpd[10481]: time reset -0.502409 s
Apr 2 16:33:02 blyshrQD6702-03 ntpd[10481]: time reset +0.499153 s
Apr 2 18:55:53 blyshrQD6702-03 ntpd[10481]: time reset -0.498446 s
Apr 2 21:23:18 blyshrQD6702-03 ntpd[10481]: time reset +0.499974 s
Apr 2 21:38:01 blyshrQD6702-03 ntpd[10481]: time reset -0.499051 s
Apr 2 23:40:09 blyshrQD6702-03 ntpd[10481]: time reset +0.500641 s
Apr 3 01:10:40 blyshrQD6702-03 ntpd[10481]: time reset -0.500318 s
Apr 3 03:11:55 blyshrQD6702-03 ntpd[10481]: time reset +0.496692 s
Apr 3 03:58:55 blyshrQD6702-03 ntpd[10481]: time reset -0.495924 s
Apr 3 06:21:27 blyshrQD6702-03 ntpd[10481]: time reset +0.504057 s
Apr 3 08:13:43 blyshrQD6702-03 ntpd[10481]: time reset -0.500685 s
Apr 3 09:59:12 blyshrQD6702-03 ntpd[10481]: time reset +0.500090 s
Apr 3 10:50:21 blyshrQD6702-03 ntpd[10481]: time reset -0.498227 s
Apr 3 14:15:20 blyshrQD6702-03 ntpd[10481]: time reset +0.497158 s
Apr 3 15:20:58 blyshrQD6702-03 ntpd[10481]: time reset -0.496092 s
Apr 3 17:40:23 blyshrQD6702-03 ntpd[10481]: time reset +0.502700 s
Apr 3 18:59:39 blyshrQD6702-03 ntpd[10481]: time reset -0.504786 s
Apr 3 22:38:30 blyshrQD6702-03 ntpd[10481]: time reset +0.498737 s
Apr 4 01:08:02 blyshrQD6702-03 ntpd[10481]: time reset -0.496318 s
Apr 4 03:31:03 blyshrQD6702-03 ntpd[10481]: time reset +0.498443 s
Apr 4 05:19:34 blyshrQD6702-03 ntpd[10481]: time reset -0.498638 s
Apr 4 07:15:11 blyshrQD6702-03 ntpd[10481]: time reset +0.498494 s
Apr 4 07:53:38 blyshrQD6702-03 ntpd[10481]: time reset -0.501513 s
Apr 4 11:59:15 blyshrQD6702-03 ntpd[10481]: time reset +0.499997 s
Apr 4 12:44:13 blyshrQD6702-03 ntpd[10481]: time reset -0.500103 s
Apr 4 14:02:43 blyshrQD6702-03 ntpd[10481]: time reset +0.504838 s
Apr 4 14:09:24 blyshrQD6702-03 ntpd[10481]: time reset -0.498143 s
Apr 4 15:40:12 blyshrQD6702-03 ntpd[10481]: time reset +0.497609 s
Apr 4 16:20:09 blyshrQD6702-03 ntpd[10481]: time reset -0.498938 s
Apr 4 19:18:18 blyshrQD6702-03 ntpd[10481]: time reset +0.501863 s
Apr 4 20:27:50 blyshrQD6702-03 ntpd[10481]: time reset -0.498268 s
Apr 5 00:09:17 blyshrQD6702-03 ntpd[10481]: time reset +0.500757 s
Apr 5 01:24:24 blyshrQD6702-03 ntpd[10481]: time reset -0.496102 s
Apr 5 02:21:16 blyshrQD6702-03 ntpd[10481]: time reset +0.496916 s
Apr 5 03:54:53 blyshrQD6702-03 ntpd[10481]: time reset -0.502162 s
Apr 5 04:06:55 blyshrQD6702-03 ntpd[10481]: time reset +0.502590 s
Apr 5 04:32:18 blyshrQD6702-03 ntpd[10481]: time reset -0.495744 s
Apr 5 06:53:31 blyshrQD6702-03 ntpd[10481]: time reset +0.504738 s
Apr 5 07:35:14 blyshrQD6702-03 ntpd[10481]: time reset -0.499908 s
Apr 5 09:46:38 blyshrQD6702-03 ntpd[10481]: time reset +0.500431 s
Apr 5 10:28:18 blyshrQD6702-03 ntpd[10481]: time reset -0.506038 s
Apr 5 12:55:42 blyshrQD6702-03 ntpd[10481]: time reset +0.499902 s
Apr 5 13:22:45 blyshrQD6702-03 ntpd[10481]: time reset -0.498897 s
Apr 5 15:49:59 blyshrQD6702-03 ntpd[10481]: time reset +0.500928 s
Apr 5 16:31:41 blyshrQD6702-03 ntpd[10481]: time reset -0.499298 s
Apr 5 18:38:48 blyshrQD6702-03 ntpd[10481]: time reset +0.498016 s
Apr 5 19:19:43 blyshrQD6702-03 ntpd[10481]: time reset -0.500370 s
Apr 5 21:57:24 blyshrQD6702-03 ntpd[10481]: time reset +0.497451 s
Apr 5 22:52:13 blyshrQD6702-03 ntpd[10481]: time reset -0.507046 s
Apr 6 03:57:53 blyshrQD6702-03 ntpd[10481]: time reset +0.501693 s
Apr 6 04:17:18 blyshrQD6702-03 ntpd[10481]: time reset -0.499456 s
Apr 6 05:51:41 blyshrQD6702-03 ntpd[10481]: time reset +0.497232 s
Apr 6 06:29:21 blyshrQD6702-03 ntpd[10481]: time reset -0.500882 s
Apr 6 09:16:03 blyshrQD6702-03 ntpd[10481]: time reset +0.500044 s
Apr 6 10:19:39 blyshrQD6702-03 ntpd[10481]: time reset -0.502275 s
Apr 6 14:01:32 blyshrQD6702-03 ntpd[10481]: time reset +0.502619 s
Apr 6 15:17:48 blyshrQD6702-03 ntpd[10481]: time reset -0.502408 s
Apr 6 16:18:29 blyshrQD6702-03 ntpd[10481]: time reset +0.505553 s
Apr 6 16:55:39 blyshrQD6702-03 ntpd[10481]: time reset -0.502190 s
Apr 6 20:18:30 blyshrQD6702-03 ntpd[10481]: time reset +0.500401 s
Apr 6 21:39:10 blyshrQD6702-03 ntpd[10481]: time reset -0.498847 s
Apr 7 00:08:35 blyshrQD6702-03 ntpd[10481]: time reset +0.539926 s
Apr 7 01:11:31 blyshrQD6702-03 ntpd[10481]: time reset -0.451668 s
Apr 7 03:03:52 blyshrQD6702-03 ntpd[10481]: time reset +0.504096 s
Apr 7 04:38:57 blyshrQD6702-03 ntpd[10481]: time reset -0.499302 s
Apr 7 05:30:33 blyshrQD6702-03 ntpd[10481]: time reset +0.499664 s
Apr 7 06:12:19 blyshrQD6702-03 ntpd[10481]: time reset -0.500973 s
Apr 7 06:58:28 blyshrQD6702-03 ntpd[10481]: time reset +0.498723 s
Apr 7 07:54:14 blyshrQD6702-03 ntpd[10481]: time reset -0.499377 s
Apr 7 09:11:39 blyshrQD6702-03 ntpd[10481]: time reset +0.506459 s
Apr 7 10:52:11 blyshrQD6702-03 ntpd[10481]: time reset -0.500958 s
Apr 7 12:26:46 blyshrQD6702-03 ntpd[10481]: time reset +0.502460 s
Apr 7 13:25:49 blyshrQD6702-03 ntpd[10481]: time reset -0.500329 s
Apr 7 15:03:08 blyshrQD6702-03 ntpd[10481]: time reset +0.502883 s
Apr 7 16:19:46 blyshrQD6702-03 ntpd[10481]: time reset -0.502467 s
Apr 7 17:14:26 blyshrQD6702-03 ntpd[10481]: time reset +0.496326 s
Apr 7 18:12:35 blyshrQD6702-03 ntpd[10481]: time reset -0.498959 s
Apr 7 21:32:14 blyshrQD6702-03 ntpd[10481]: time reset +0.502081 s
Apr 7 21:50:54 blyshrQD6702-03 ntpd[10481]: time reset -0.502138 s
Apr 8 01:51:01 blyshrQD6702-03 ntpd[10481]: time reset +0.486278 s
Apr 8 02:49:01 blyshrQD6702-03 ntpd[10481]: time reset -0.492495 s
Apr 8 07:13:44 blyshrQD6702-03 ntpd[10481]: time reset +0.520524 s
Apr 8 07:34:52 blyshrQD6702-03 ntpd[10481]: time reset -0.499471 s
Apr 8 09:09:53 blyshrQD6702-03 ntpd[10481]: time reset +0.502421 s
Apr 8 09:46:44 blyshrQD6702-03 ntpd[10481]: time reset -0.499725 s
Apr 8 15:42:44 blyshrQD6702-03 ntpd[10481]: time reset +0.502217 s
Apr 8 17:20:16 blyshrQD6702-03 ntpd[10481]: time reset -0.496293 s
Apr 8 20:34:03 blyshrQD6702-03 ntpd[10481]: time reset +0.491622 s
Apr 8 21:11:53 blyshrQD6702-03 ntpd[10481]: time reset -0.502154 s
Apr 8 22:00:40 blyshrQD6702-03 ntpd[10481]: time reset +0.495639 s
Apr 9 00:49:35 blyshrQD6702-03 ntpd[10481]: time reset -0.496496 s
Apr 9 03:48:41 blyshrQD6702-03 ntpd[10481]: time reset +0.498727 s
Apr 9 04:27:11 blyshrQD6702-03 ntpd[10481]: time reset -0.496758 s
Apr 9 06:05:56 blyshrQD6702-03 ntpd[10481]: time reset +0.506512 s
Apr 9 07:25:56 blyshrQD6702-03 ntpd[10481]: time reset -0.500090 s
Apr 9 09:00:03 blyshrQD6702-03 ntpd[10481]: time reset +0.498792 s
Apr 9 10:29:38 blyshrQD6702-03 ntpd[10481]: time reset -0.502494 s
Apr 9 12:19:17 blyshrQD6702-03 ntpd[10481]: time reset +0.499732 s
Apr 9 13:29:14 blyshrQD6702-03 ntpd[10481]: time reset -0.501187 s
Apr 9 15:59:01 blyshrQD6702-03 ntpd[10481]: time reset +0.499387 s
Apr 9 16:31:43 blyshrQD6702-03 ntpd[10481]: time reset -0.494679 s
Apr 9 18:20:29 blyshrQD6702-03 ntpd[10481]: time reset +0.500277 s
Apr 9 21:18:25 blyshrQD6702-03 ntpd[10481]: time reset -0.494761 s
Apr 9 22:00:05 blyshrQD6702-03 ntpd[10481]: time reset +0.499915 s
Apr 9 22:41:20 blyshrQD6702-03 ntpd[10481]: time reset -0.497839 s
Apr 10 01:13:17 blyshrQD6702-03 ntpd[10481]: time reset +0.507112 s
Apr 10 04:03:26 blyshrQD6702-03 ntpd[10481]: time reset -0.499348 s
Apr 10 06:30:47 blyshrQD6702-03 ntpd[10481]: time reset +0.501108 s
Apr 10 07:02:37 blyshrQD6702-03 ntpd[10481]: time reset -0.501291 s
Apr 10 09:00:09 blyshrQD6702-03 ntpd[10481]: time reset +0.496130 s
Apr 10 11:25:39 blyshrQD6702-03 ntpd[10481]: time reset -0.505797 s
Apr 10 14:15:06 blyshrQD6702-03 ntpd[10481]: time reset +0.499018 s
Apr 10 15:40:47 blyshrQD6702-03 ntpd[10481]: time reset -0.502297 s
Apr 10 18:56:53 blyshrQD6702-03 ntpd[10481]: time reset +0.495614 s
Apr 10 19:47:16 blyshrQD6702-03 ntpd[10481]: time reset -0.502485 s
Apr 10 20:22:56 blyshrQD6702-03 ntpd[10481]: time reset +0.499534 s
Apr 10 20:58:40 blyshrQD6702-03 ntpd[10481]: time reset -0.500453 s
Apr 10 22:15:25 blyshrQD6702-03 ntpd[10481]: time reset +0.498550 s
Apr 10 23:20:02 blyshrQD6702-03 ntpd[10481]: time reset -0.500677 s
Apr 11 00:45:50 blyshrQD6702-03 ntpd[10481]: time reset +0.499478 s
Apr 11 03:53:33 blyshrQD6702-03 ntpd[10481]: time reset -0.499099 s
Apr 11 05:11:10 blyshrQD6702-03 ntpd[10481]: time reset +0.501278 s
Apr 11 05:47:13 blyshrQD6702-03 ntpd[10481]: time reset -0.499498 s
Apr 11 06:20:48 blyshrQD6702-03 ntpd[10481]: time reset +0.500395 s
Apr 11 06:52:20 blyshrQD6702-03 ntpd[10481]: time reset -0.498371 s
Apr 11 07:43:59 blyshrQD6702-03 ntpd[10481]: time reset +0.499604 s
Apr 11 08:01:08 blyshrQD6702-03 ntpd[10481]: time reset -0.498927 s
Apr 11 11:13:55 blyshrQD6702-03 ntpd[10481]: time reset +0.499881 s
Apr 11 11:53:00 blyshrQD6702-03 ntpd[10481]: time reset -0.499493 s
Apr 11 14:15:19 blyshrQD6702-03 ntpd[10481]: time reset +0.494561 s
Apr 11 15:00:05 blyshrQD6702-03 ntpd[10481]: time reset -0.500015 s
Apr 11 18:47:53 blyshrQD6702-03 ntpd[10481]: time reset +0.499117 s
Apr 11 19:40:28 blyshrQD6702-03 ntpd[10481]: time reset -0.499421 s
Apr 11 22:04:35 blyshrQD6702-03 ntpd[10481]: time reset +0.496922 s
Apr 11 23:14:17 blyshrQD6702-03 ntpd[10481]: time reset -0.498758 s
Apr 12 01:26:50 blyshrQD6702-03 ntpd[10481]: time reset +0.499214 s
Apr 12 01:36:41 blyshrQD6702-03 ntpd[10481]: time reset -0.497080 s
Apr 12 05:41:24 blyshrQD6702-03 ntpd[10481]: time reset +0.494073 s
Apr 12 06:00:40 blyshrQD6702-03 ntpd[10481]: time reset -0.499605 s
Apr 12 07:54:57 blyshrQD6702-03 ntpd[10481]: time reset +0.501433 s
Apr 12 09:40:35 blyshrQD6702-03 ntpd[10481]: time reset -0.502577 s
Apr 12 11:13:22 blyshrQD6702-03 ntpd[10481]: time reset +0.489278 s
Apr 12 12:14:26 blyshrQD6702-03 ntpd[10481]: time reset -0.501248 s
Apr 12 17:07:57 blyshrQD6702-03 ntpd[10481]: time reset +0.502331 s
Apr 12 18:23:53 blyshrQD6702-03 ntpd[10481]: time reset -0.498298 s
Apr 12 21:50:23 blyshrQD6702-03 ntpd[10481]: time reset +0.499541 s
Apr 12 22:27:33 blyshrQD6702-03 ntpd[10481]: time reset -0.500688 s
Apr 13 06:03:49 blyshrQD6702-03 ntpd[10481]: time reset +0.496579 s
Apr 13 07:30:54 blyshrQD6702-03 ntpd[10481]: time reset -0.498203 s
Apr 13 09:49:19 blyshrQD6702-03 ntpd[10481]: time reset +0.500226 s
Apr 13 11:11:38 blyshrQD6702-03 ntpd[10481]: time reset -0.500889 s
Apr 13 14:58:04 blyshrQD6702-03 ntpd[10481]: time reset +0.517715 s
Apr 13 15:38:08 blyshrQD6702-03 ntpd[10481]: time reset -0.497274 s
Apr 13 21:08:56 blyshrQD6702-03 ntpd[10481]: time reset +0.502516 s
Apr 13 22:29:13 blyshrQD6702-03 ntpd[10481]: time reset -0.501228 s
Apr 14 01:25:29 blyshrQD6702-03 ntpd[10481]: time reset +0.503364 s
Apr 14 02:06:06 blyshrQD6702-03 ntpd[10481]: time reset -0.499839 s
Apr 14 04:49:01 blyshrQD6702-03 ntpd[10481]: time reset +0.497951 s
Apr 14 05:41:26 blyshrQD6702-03 ntpd[10481]: time reset -0.500950 s
Apr 14 08:55:50 blyshrQD6702-03 ntpd[10481]: time reset +0.500785 s
Apr 14 09:51:26 blyshrQD6702-03 ntpd[10481]: time reset -0.499108 s
Apr 14 21:10:16 blyshrQD6702-03 ntpd[10615]: time reset +0.504407 s
Apr 14 22:35:18 blyshrQD6702-03 ntpd[10615]: time reset -0.497773 s
Apr 15 00:29:12 blyshrQD6702-03 ntpd[10615]: time reset +0.516013 s
Apr 15 01:39:49 blyshrQD6702-03 ntpd[10615]: time reset -0.483362 s
Apr 15 04:55:39 blyshrQD6702-03 ntpd[10615]: time reset +0.499030 s
Apr 15 07:10:21 blyshrQD6702-03 ntpd[10615]: time reset -0.501335 s
Apr 15 11:06:45 blyshrQD6702-03 ntpd[10615]: time reset +0.498011 s
Apr 15 12:06:17 blyshrQD6702-03 ntpd[10615]: time reset -0.504157 s
Apr 15 14:06:44 blyshrQD6702-03 ntpd[10615]: time reset +0.495742 s
Apr 15 15:01:25 blyshrQD6702-03 ntpd[10615]: time reset -0.499841 s
Apr 15 18:27:53 blyshrQD6702-03 ntpd[10615]: time reset +0.498494 s
Apr 15 19:04:00 blyshrQD6702-03 ntpd[10615]: time reset -0.498573 s
Apr 15 23:32:30 blyshrQD6702-03 ntpd[10615]: time reset +0.501938 s
Apr 16 00:48:03 blyshrQD6702-03 ntpd[10615]: time reset -0.513779 s
Apr 16 02:39:15 blyshrQD6702-03 ntpd[10615]: time reset +0.500854 s
Apr 16 04:37:51 blyshrQD6702-03 ntpd[10615]: time reset -0.499522 s
Apr 16 05:59:12 blyshrQD6702-03 ntpd[10615]: time reset +0.504113 s
Apr 16 08:06:15 blyshrQD6702-03 ntpd[10615]: time reset -0.497532 s
Apr 16 10:11:06 blyshrQD6702-03 ntpd[10615]: time reset +0.501325 s
Apr 16 10:51:30 blyshrQD6702-03 ntpd[10615]: time reset -0.502219 s
Apr 16 12:47:15 blyshrQD6702-03 ntpd[10615]: time reset +0.496901 s
Apr 16 15:57:26 blyshrQD6702-03 ntpd[10615]: time reset -0.504175 s
Apr 16 16:51:14 blyshrQD6702-03 ntpd[10615]: time reset +0.498741 s
Apr 16 17:14:16 blyshrQD6702-03 ntpd[10615]: time reset -0.498957 s
Apr 16 19:07:09 blyshrQD6702-03 ntpd[10615]: time reset +0.505745 s
Apr 16 19:34:50 blyshrQD6702-03 ntpd[10615]: time reset -0.495264 s
Apr 16 22:21:14 blyshrQD6702-03 ntpd[10615]: time reset +0.499323 s
Apr 16 23:17:45 blyshrQD6702-03 ntpd[10615]: time reset -0.497606 s
Apr 17 01:01:31 blyshrQD6702-03 ntpd[10615]: time reset +0.494669 s
Apr 17 01:41:41 blyshrQD6702-03 ntpd[10615]: time reset -0.493848 s
Apr 17 08:54:38 blyshrQD6702-03 ntpd[10615]: time reset +0.503123 s
Apr 17 10:40:43 blyshrQD6702-03 ntpd[10615]: time reset -0.499773 s
Apr 17 14:28:10 blyshrQD6702-03 ntpd[10615]: time reset +0.502579 s
Apr 17 16:51:35 blyshrQD6702-03 ntpd[10615]: time reset -0.498667 s
Apr 17 18:28:48 blyshrQD6702-03 ntpd[10615]: time reset +0.500570 s
Apr 17 19:32:18 blyshrQD6702-03 ntpd[10615]: time reset -0.503299 s
Apr 17 21:59:31 blyshrQD6702-03 ntpd[10615]: time reset +0.495660 s
Apr 17 23:56:28 blyshrQD6702-03 ntpd[10615]: time reset -0.514116 s
Apr 18 02:30:36 blyshrQD6702-03 ntpd[10615]: time reset +0.499918 s
Apr 18 02:39:29 blyshrQD6702-03 ntpd[10615]: time reset -0.497489 s
Apr 18 03:53:05 blyshrQD6702-03 ntpd[10615]: time reset +0.503568 s
Apr 18 05:24:02 blyshrQD6702-03 ntpd[10615]: time reset -0.496736 s
Apr 18 08:03:10 blyshrQD6702-03 ntpd[10615]: time reset +0.506386 s
Apr 18 08:10:22 blyshrQD6702-03 ntpd[10615]: time reset -0.502107 s
Apr 18 09:40:02 blyshrQD6702-03 ntpd[10615]: time reset +0.497488 s
Apr 18 12:04:41 blyshrQD6702-03 ntpd[10615]: time reset -0.501535 s
Apr 18 12:55:16 blyshrQD6702-03 ntpd[10615]: time reset +0.508402 s
Apr 18 13:14:00 blyshrQD6702-03 ntpd[10615]: time reset -0.497074 s
Apr 18 15:55:52 blyshrQD6702-03 ntpd[10615]: time reset +0.499466 s
Apr 18 16:35:11 blyshrQD6702-03 ntpd[10615]: time reset -0.498906 s
Apr 18 21:27:03 blyshrQD6702-03 ntpd[10615]: time reset +0.498559 s
Apr 18 22:29:04 blyshrQD6702-03 ntpd[10615]: time reset -0.503733 s
Apr 18 23:37:46 blyshrQD6702-03 ntpd[10615]: time reset +0.506080 s
Apr 19 00:17:28 blyshrQD6702-03 ntpd[10615]: time reset -0.504414 s
Apr 19 02:20:28 blyshrQD6702-03 ntpd[10615]: time reset +0.497270 s
Apr 19 03:09:59 blyshrQD6702-03 ntpd[10615]: time reset -0.499931 s
Apr 19 04:54:23 blyshrQD6702-03 ntpd[10615]: time reset +0.500919 s
Apr 19 05:51:56 blyshrQD6702-03 ntpd[10615]: time reset -0.498046 s
Apr 19 06:54:05 blyshrQD6702-03 ntpd[10615]: time reset +0.498512 s
Apr 19 07:14:52 blyshrQD6702-03 ntpd[10615]: time reset -0.498083 s
Apr 19 09:32:24 blyshrQD6702-03 ntpd[10615]: time reset +0.505489 s
Apr 19 10:30:24 blyshrQD6702-03 ntpd[10615]: time reset -0.496446 s
Apr 19 12:26:50 blyshrQD6702-03 ntpd[10615]: time reset +0.499326 s
Apr 19 13:48:10 blyshrQD6702-03 ntpd[10615]: time reset -0.500480 s
Apr 19 17:26:52 blyshrQD6702-03 ntpd[10615]: time reset +0.496519 s
Apr 19 20:37:51 blyshrQD6702-03 ntpd[10615]: time reset -0.499674 s
Apr 19 22:35:08 blyshrQD6702-03 ntpd[10615]: time reset +0.499651 s
Apr 19 23:10:59 blyshrQD6702-03 ntpd[10615]: time reset -0.499420 s
Apr 20 00:48:29 blyshrQD6702-03 ntpd[10615]: time reset +0.501957 s
Apr 20 02:05:31 blyshrQD6702-03 ntpd[10615]: time reset -0.497822 s
Apr 20 03:50:50 blyshrQD6702-03 ntpd[10615]: time reset +0.495774 s
Apr 20 06:54:42 blyshrQD6702-03 ntpd[10615]: time reset -0.503750 s
Apr 20 13:45:05 blyshrQD6702-03 ntpd[10615]: time reset +0.502456 s
Apr 20 14:56:41 blyshrQD6702-03 ntpd[10615]: time reset -0.494633 s
Apr 20 20:50:34 blyshrQD6702-03 ntpd[10615]: time reset +0.499054 s
Apr 20 21:30:16 blyshrQD6702-03 ntpd[10615]: time reset -0.499270 s
Apr 21 01:27:22 blyshrQD6702-03 ntpd[10615]: time reset +0.501046 s
Apr 21 03:07:22 blyshrQD6702-03 ntpd[10615]: time reset -0.501369 s
Apr 21 04:47:08 blyshrQD6702-03 ntpd[10615]: time reset +0.495323 s
Apr 21 05:26:36 blyshrQD6702-03 ntpd[10615]: time reset -0.506779 s
Apr 21 10:26:45 blyshrQD6702-03 ntpd[10615]: time reset +0.496984 s
Apr 21 11:43:50 blyshrQD6702-03 ntpd[10615]: time reset -0.507100 s
Apr 21 14:05:43 blyshrQD6702-03 ntpd[10615]: time reset +0.511731 s
Apr 21 15:44:45 blyshrQD6702-03 ntpd[10615]: time reset -0.498192 s
Apr 21 20:49:25 blyshrQD6702-03 ntpd[10615]: time reset +0.504519 s
Apr 21 21:13:38 blyshrQD6702-03 ntpd[10615]: time reset -0.497689 s
Apr 22 00:15:00 blyshrQD6702-03 ntpd[10615]: time reset +0.501476 s
Apr 22 00:54:57 blyshrQD6702-03 ntpd[10615]: time reset -0.504294 s
Apr 22 02:28:20 blyshrQD6702-03 ntpd[10615]: time reset +0.501361 s
Apr 22 03:25:54 blyshrQD6702-03 ntpd[10615]: time reset -0.499522 s
Apr 22 11:27:35 blyshrQD6702-03 ntpd[10615]: time reset +0.496629 s
Apr 22 13:52:25 blyshrQD6702-03 ntpd[10615]: time reset -0.503551 s
Apr 22 16:42:22 blyshrQD6702-03 ntpd[10615]: time reset +0.499455 s
Apr 22 17:19:23 blyshrQD6702-03 ntpd[10615]: time reset -0.499615 s
Apr 22 18:54:42 blyshrQD6702-03 ntpd[10615]: time reset +0.500228 s
Apr 22 19:13:06 blyshrQD6702-03 ntpd[10615]: time reset -0.498326 s
Apr 22 21:16:29 blyshrQD6702-03 ntpd[10615]: time reset +0.496460 s
Apr 23 00:53:17 blyshrQD6702-03 ntpd[10615]: time reset -0.498760 s
Apr 23 06:27:29 blyshrQD6702-03 ntpd[10615]: time reset +0.500293 s
Apr 23 07:05:08 blyshrQD6702-03 ntpd[10615]: time reset -0.498642 s
Apr 23 08:21:30 blyshrQD6702-03 ntpd[10615]: time reset +0.501369 s
Apr 23 08:56:39 blyshrQD6702-03 ntpd[10615]: time reset -0.501771 s
Apr 23 12:51:00 blyshrQD6702-03 ntpd[10615]: time reset +0.505288 s
Apr 23 14:11:19 blyshrQD6702-03 ntpd[10615]: time reset -0.499485 s
Apr 23 15:07:46 blyshrQD6702-03 ntpd[10615]: time reset +0.499007 s
Apr 23 16:49:04 blyshrQD6702-03 ntpd[10615]: time reset -0.501131 s
Apr 23 19:54:18 blyshrQD6702-03 ntpd[10615]: time reset +0.509926 s
Apr 23 21:38:23 blyshrQD6702-03 ntpd[10615]: time reset -0.497575 s
Apr 24 01:06:43 blyshrQD6702-03 ntpd[10615]: time reset +0.503322 s
Apr 24 02:00:03 blyshrQD6702-03 ntpd[10615]: time reset -0.506276 s
Apr 24 09:24:55 blyshrQD6702-03 ntpd[10615]: time reset +0.491507 s
Apr 24 10:57:35 blyshrQD6702-03 ntpd[10615]: time reset -0.496853 s
Apr 24 11:58:02 blyshrQD6702-03 ntpd[10615]: time reset +0.501868 s
Apr 24 12:18:21 blyshrQD6702-03 ntpd[10615]: time reset -0.501840 s
Apr 24 14:36:45 blyshrQD6702-03 ntpd[10615]: time reset +0.497971 s
Apr 24 16:58:24 blyshrQD6702-03 ntpd[10615]: time reset -0.495029 s
Apr 24 20:47:43 blyshrQD6702-03 ntpd[10615]: time reset +0.500014 s
Apr 24 21:25:46 blyshrQD6702-03 ntpd[10615]: time reset -0.498300 s
Apr 24 22:45:35 blyshrQD6702-03 ntpd[10615]: time reset +0.500139 s
Apr 25 01:04:09 blyshrQD6702-03 ntpd[10615]: time reset -0.493526 s
Apr 25 02:29:06 blyshrQD6702-03 ntpd[10615]: time reset +0.494496 s
Apr 25 03:04:51 blyshrQD6702-03 ntpd[10615]: time reset -0.494111 s
Apr 25 05:05:15 blyshrQD6702-03 ntpd[10615]: time reset +0.499615 s
Apr 25 05:41:38 blyshrQD6702-03 ntpd[10615]: time reset -0.498849 s
Apr 25 08:53:04 blyshrQD6702-03 ntpd[10615]: time reset +0.499407 s
Apr 25 09:56:27 blyshrQD6702-03 ntpd[10615]: time reset -0.498757 s
Apr 25 12:34:05 blyshrQD6702-03 ntpd[10615]: time reset +0.500081 s
Apr 25 13:54:08 blyshrQD6702-03 ntpd[10615]: time reset -0.500825 s
Apr 25 18:17:25 blyshrQD6702-03 ntpd[10615]: time reset +0.498882 s
Apr 25 18:21:41 blyshrQD6702-03 ntpd[10615]: time reset -0.498409 s
Apr 25 20:14:26 blyshrQD6702-03 ntpd[10615]: time reset +0.499400 s
Apr 25 21:42:37 blyshrQD6702-03 ntpd[10615]: time reset -0.500486 s
Apr 26 01:27:24 blyshrQD6702-03 ntpd[10615]: time reset +0.506981 s
Apr 26 03:01:27 blyshrQD6702-03 ntpd[10615]: time reset -0.498327 s
Apr 26 04:47:07 blyshrQD6702-03 ntpd[10615]: time reset +0.504192 s
Apr 26 05:24:50 blyshrQD6702-03 ntpd[10615]: time reset -0.499283 s
Apr 26 06:44:50 blyshrQD6702-03 ntpd[10615]: time reset +0.505747 s
Apr 26 07:59:07 blyshrQD6702-03 ntpd[10615]: time reset -0.498373 s
Apr 26 09:39:37 blyshrQD6702-03 ntpd[10615]: time reset +0.499790 s
Apr 26 11:14:14 blyshrQD6702-03 ntpd[10615]: time reset -0.507138 s
Apr 26 14:30:45 blyshrQD6702-03 ntpd[10615]: time reset +0.501432 s
Apr 26 14:51:21 blyshrQD6702-03 ntpd[10615]: time reset -0.502941 s
Apr 26 20:02:36 blyshrQD6702-03 ntpd[10615]: time reset +0.493997 s
Apr 26 22:31:52 blyshrQD6702-03 ntpd[10615]: time reset -0.504583 s
Apr 27 03:19:26 blyshrQD6702-03 ntpd[10615]: time reset +0.496859 s
Apr 27 04:48:26 blyshrQD6702-03 ntpd[10615]: time reset -0.498187 s
Apr 27 06:54:12 blyshrQD6702-03 ntpd[10615]: time reset +0.504317 s
Apr 27 06:59:59 blyshrQD6702-03 ntpd[10615]: time reset -0.502671 s
Apr 27 08:39:38 blyshrQD6702-03 ntpd[10615]: time reset +0.500173 s
Apr 27 10:49:38 blyshrQD6702-03 ntpd[10615]: time reset -0.499723 s
Apr 27 12:56:54 blyshrQD6702-03 ntpd[10615]: time reset +0.498649 s
Apr 27 14:36:48 blyshrQD6702-03 ntpd[10615]: time reset -0.500188 s
Apr 27 17:02:05 blyshrQD6702-03 ntpd[10615]: time reset +0.499584 s
Apr 27 19:02:20 blyshrQD6702-03 ntpd[10615]: time reset -0.498521 s
Apr 27 20:57:19 blyshrQD6702-03 ntpd[10615]: time reset +0.499498 s
Apr 27 22:17:24 blyshrQD6702-03 ntpd[10615]: time reset -0.500790 s
Apr 28 00:57:11 blyshrQD6702-03 ntpd[10615]: time reset +0.499618 s
Apr 28 01:17:32 blyshrQD6702-03 ntpd[10615]: time reset -0.501851 s
Apr 28 05:08:48 blyshrQD6702-03 ntpd[10615]: time reset +0.496912 s
Apr 28 08:21:13 blyshrQD6702-03 ntpd[10615]: time reset -0.503572 s
Apr 28 13:41:27 blyshrQD6702-03 ntpd[10615]: time reset +0.501774 s
Apr 28 14:34:09 blyshrQD6702-03 ntpd[10615]: time reset -0.497472 s
Apr 28 17:10:14 blyshrQD6702-03 ntpd[10615]: time reset +0.500430 s
Apr 28 20:06:29 blyshrQD6702-03 ntpd[10615]: time reset -0.504991 s
Apr 29 01:35:25 blyshrQD6702-03 ntpd[10615]: time reset +0.498642 s
Apr 29 02:53:44 blyshrQD6702-03 ntpd[10615]: time reset -0.501674 s
Apr 29 05:29:31 blyshrQD6702-03 ntpd[10615]: time reset +0.500943 s
Apr 29 06:02:08 blyshrQD6702-03 ntpd[10615]: time reset -0.499092 s
Apr 29 06:25:05 blyshrQD6702-03 ntpd[10615]: time reset +0.499774 s
Apr 29 07:51:24 blyshrQD6702-03 ntpd[10615]: time reset -0.498989 s
Apr 29 09:31:12 blyshrQD6702-03 ntpd[10615]: time reset +0.496399 s
Apr 29 09:58:08 blyshrQD6702-03 ntpd[10615]: time reset -0.498419 s
Apr 29 13:47:13 blyshrQD6702-03 ntpd[10615]: time reset +0.508096 s
Apr 29 15:14:08 blyshrQD6702-03 ntpd[10615]: time reset -0.500425 s
Apr 29 17:43:50 blyshrQD6702-03 ntpd[10615]: time reset +0.503209 s
Apr 29 18:43:57 blyshrQD6702-03 ntpd[10615]: time reset -0.504406 s
Apr 29 21:27:18 blyshrQD6702-03 ntpd[10615]: time reset +0.485472 s
Apr 29 23:14:27 blyshrQD6702-03 ntpd[10615]: time reset -0.494403 s
Apr 30 00:44:56 blyshrQD6702-03 ntpd[10615]: time reset +0.510372 s
Apr 30 04:05:18 blyshrQD6702-03 ntpd[10615]: time reset -0.470096 s
Apr 30 06:53:46 blyshrQD6702-03 ntpd[10615]: time reset +0.503907 s
Apr 30 07:56:13 blyshrQD6702-03 ntpd[10615]: time reset -0.498109 s
Apr 30 10:19:00 blyshrQD6702-03 ntpd[10615]: time reset +0.500410 s
Apr 30 11:17:06 blyshrQD6702-03 ntpd[10615]: time reset -0.500164 s
Apr 30 17:49:14 blyshrQD6702-03 ntpd[10615]: time reset +0.499215 s
Apr 30 19:50:26 blyshrQD6702-03 ntpd[10615]: time reset -0.499968 s
May 1 00:23:21 blyshrQD6702-03 ntpd[10615]: time reset +0.497547 s
May 1 01:44:49 blyshrQD6702-03 ntpd[10615]: time reset -0.495781 s
May 1 05:46:06 blyshrQD6702-03 ntpd[10615]: time reset +0.501441 s
May 1 06:10:16 blyshrQD6702-03 ntpd[10615]: time reset -0.499188 s
May 1 08:28:47 blyshrQD6702-03 ntpd[10615]: time reset +0.500440 s
May 1 09:03:30 blyshrQD6702-03 ntpd[10615]: time reset -0.498627 s
May 1 10:24:32 blyshrQD6702-03 ntpd[10615]: time reset +0.501337 s
May 1 11:05:12 blyshrQD6702-03 ntpd[10615]: time reset -0.501177 s
May 1 12:57:35 blyshrQD6702-03 ntpd[10615]: time reset +0.500229 s
May 1 13:17:47 blyshrQD6702-03 ntpd[10615]: time reset -0.500650 s
May 1 15:24:38 blyshrQD6702-03 ntpd[10615]: time reset +0.503992 s
May 1 17:46:08 blyshrQD6702-03 ntpd[10615]: time reset -0.498105 s
May 1 20:24:53 blyshrQD6702-03 ntpd[10615]: time reset +0.498489 s
May 1 21:36:49 blyshrQD6702-03 ntpd[10615]: time reset -0.498197 s
May 1 23:49:47 blyshrQD6702-03 ntpd[10615]: time reset +0.504637 s
May 2 00:32:03 blyshrQD6702-03 ntpd[10615]: time reset -0.493314 s
May 2 03:22:53 blyshrQD6702-03 ntpd[10615]: time reset +0.492070 s
May 2 06:03:29 blyshrQD6702-03 ntpd[10615]: time reset -0.501340 s
May 2 09:33:21 blyshrQD6702-03 ntpd[10615]: time reset +0.498162 s
May 2 10:10:24 blyshrQD6702-03 ntpd[10615]: time reset -0.504343 s
May 2 13:16:20 blyshrQD6702-03 ntpd[10615]: time reset +0.492147 s
May 2 13:31:11 blyshrQD6702-03 ntpd[10615]: time reset -0.502149 s
May 2 15:22:49 blyshrQD6702-03 ntpd[10615]: time reset +0.497039 s
May 2 16:59:00 blyshrQD6702-03 ntpd[10615]: time reset -0.499666 s
May 2 23:26:46 blyshrQD6702-03 ntpd[10615]: time reset +0.498798 s
May 2 23:44:26 blyshrQD6702-03 ntpd[10615]: time reset -0.503459 s
May 3 01:16:39 blyshrQD6702-03 ntpd[10615]: time reset +0.499231 s
May 3 01:58:42 blyshrQD6702-03 ntpd[10615]: time reset -0.496345 s
May 3 04:21:37 blyshrQD6702-03 ntpd[10615]: time reset +0.502314 s
May 3 05:43:30 blyshrQD6702-03 ntpd[10615]: time reset -0.499504 s
May 3 07:34:54 blyshrQD6702-03 ntpd[10615]: time reset +0.501919 s
May 3 08:09:03 blyshrQD6702-03 ntpd[10615]: time reset -0.501565 s
May 3 09:54:38 blyshrQD6702-03 ntpd[10615]: time reset +0.489589 s
May 3 11:26:26 blyshrQD6702-03 ntpd[10615]: time reset -0.505335 s
May 3 12:40:33 blyshrQD6702-03 ntpd[10615]: time reset +0.499098 s
May 3 14:05:53 blyshrQD6702-03 ntpd[10615]: time reset -0.498937 s
May 3 16:53:37 blyshrQD6702-03 ntpd[10615]: time reset +0.499552 s
May 3 17:43:44 blyshrQD6702-03 ntpd[10615]: time reset -0.501431 s
May 3 20:00:26 blyshrQD6702-03 ntpd[10615]: time reset +0.504178 s
May 3 21:08:34 blyshrQD6702-03 ntpd[10615]: time reset -0.500334 s
May 3 22:59:42 blyshrQD6702-03 ntpd[10615]: time reset +0.503864 s
May 3 23:35:41 blyshrQD6702-03 ntpd[10615]: time reset -0.495255 s
May 4 01:09:52 blyshrQD6702-03 ntpd[10615]: time reset +0.499428 s
May 4 02:19:11 blyshrQD6702-03 ntpd[10615]: time reset -0.491447 s
May 4 03:37:03 blyshrQD6702-03 ntpd[10615]: time reset +0.496486 s
May 4 04:13:03 blyshrQD6702-03 ntpd[10615]: time reset -0.498492 s
May 4 08:21:28 blyshrQD6702-03 ntpd[10615]: time reset +0.500502 s
May 4 11:19:41 blyshrQD6702-03 ntpd[10615]: time reset -0.498428 s
May 4 12:32:53 blyshrQD6702-03 ntpd[10615]: time reset +0.499469 s
May 4 14:53:37 blyshrQD6702-03 ntpd[10615]: time reset -0.495805 s
May 4 18:02:19 blyshrQD6702-03 ntpd[10615]: time reset +0.500394 s
May 4 19:56:21 blyshrQD6702-03 ntpd[10615]: time reset -0.499987 s
May 5 01:55:32 blyshrQD6702-03 ntpd[10615]: time reset +0.490770 s
May 5 02:55:05 blyshrQD6702-03 ntpd[10615]: time reset -0.500989 s
May 5 06:47:34 blyshrQD6702-03 ntpd[10615]: time reset +0.501506 s
May 5 07:09:18 blyshrQD6702-03 ntpd[10615]: time reset -0.499155 s
May 5 09:35:27 blyshrQD6702-03 ntpd[10615]: time reset +0.497706 s
May 5 10:37:02 blyshrQD6702-03 ntpd[10615]: time reset -0.499872 s
May 5 12:55:33 blyshrQD6702-03 ntpd[10615]: time reset +0.502053 s
May 5 14:30:42 blyshrQD6702-03 ntpd[10615]: time reset -0.498415 s
May 5 17:20:04 blyshrQD6702-03 ntpd[10615]: time reset +0.497993 s
May 5 17:54:03 blyshrQD6702-03 ntpd[10615]: time reset -0.498506 s
May 5 20:16:43 blyshrQD6702-03 ntpd[10615]: time reset +0.498698 s
May 5 20:53:32 blyshrQD6702-03 ntpd[10615]: time reset -0.500811 s
May 5 23:11:32 blyshrQD6702-03 ntpd[10615]: time reset +0.501912 s
May 5 23:49:00 blyshrQD6702-03 ntpd[10615]: time reset -0.499674 s
May 6 02:14:51 blyshrQD6702-03 ntpd[10615]: time reset +0.495125 s
May 6 02:42:28 blyshrQD6702-03 ntpd[10615]: time reset -0.499147 s
May 6 04:09:33 blyshrQD6702-03 ntpd[10615]: time reset +0.498138 s
May 6 04:45:56 blyshrQD6702-03 ntpd[10615]: time reset -0.499041 s
May 6 07:06:08 blyshrQD6702-03 ntpd[10615]: time reset +0.504798 s
May 6 08:47:13 blyshrQD6702-03 ntpd[10615]: time reset -0.498440 s
May 6 10:23:21 blyshrQD6702-03 ntpd[10615]: time reset +0.499378 s
May 6 11:40:53 blyshrQD6702-03 ntpd[10615]: time reset -0.497662 s
May 6 14:45:04 blyshrQD6702-03 ntpd[10615]: time reset +0.502518 s
May 6 15:46:50 blyshrQD6702-03 ntpd[10615]: time reset -0.497935 s
May 6 18:51:22 blyshrQD6702-03 ntpd[10615]: time reset +0.500128 s
May 6 19:57:54 blyshrQD6702-03 ntpd[10615]: time reset -0.499190 s
May 7 01:27:36 blyshrQD6702-03 ntpd[10615]: time reset +0.500436 s
May 7 01:31:52 blyshrQD6702-03 ntpd[10615]: time reset -0.501280 s
May 7 03:20:20 blyshrQD6702-03 ntpd[10615]: time reset +0.502223 s
May 7 04:00:43 blyshrQD6702-03 ntpd[10615]: time reset -0.496326 s
May 7 07:37:54 blyshrQD6702-03 ntpd[10615]: time reset +0.500594 s
May 7 08:05:58 blyshrQD6702-03 ntpd[10615]: time reset -0.497301 s
May 7 09:35:39 blyshrQD6702-03 ntpd[10615]: time reset +0.497789 s
May 7 10:13:13 blyshrQD6702-03 ntpd[10615]: time reset -0.498005 s
May 7 14:22:31 blyshrQD6702-03 ntpd[10615]: time reset +0.498092 s
May 7 15:03:57 blyshrQD6702-03 ntpd[10615]: time reset -0.500993 s
May 7 17:31:36 blyshrQD6702-03 ntpd[10615]: time reset +0.493705 s
May 7 17:56:16 blyshrQD6702-03 ntpd[10615]: time reset -0.498239 s
May 7 20:15:46 blyshrQD6702-03 ntpd[10615]: time reset +0.499613 s
May 7 22:44:52 blyshrQD6702-03 ntpd[10615]: time reset -0.498014 s
May 8 02:56:17 blyshrQD6702-03 ntpd[10615]: time reset +0.499422 s
May 8 03:29:41 blyshrQD6702-03 ntpd[10615]: time reset -0.501391 s
May 8 05:35:13 blyshrQD6702-03 ntpd[10615]: time reset +0.501124 s
May 8 06:05:00 blyshrQD6702-03 ntpd[10615]: time reset -0.501547 s
May 8 08:01:26 blyshrQD6702-03 ntpd[10615]: time reset +0.498243 s
May 8 08:36:29 blyshrQD6702-03 ntpd[10615]: time reset -0.499368 s
May 8 13:49:22 blyshrQD6702-03 ntpd[10615]: time reset +0.504955 s
May 8 14:36:39 blyshrQD6702-03 ntpd[10615]: time reset -0.498864 s
May 8 17:23:34 blyshrQD6702-03 ntpd[10615]: time reset +0.499227 s
May 8 18:31:56 blyshrQD6702-03 ntpd[10615]: time reset -0.501608 s
May 8 20:33:56 blyshrQD6702-03 ntpd[10615]: time reset +0.503855 s
May 8 20:55:34 blyshrQD6702-03 ntpd[10615]: time reset -0.501226 s
May 8 22:03:32 blyshrQD6702-03 ntpd[10615]: time reset +0.491951 s
May 8 23:06:58 blyshrQD6702-03 ntpd[10615]: time reset -0.500173 s
May 9 00:28:39 blyshrQD6702-03 ntpd[10615]: time reset +0.499499 s
May 9 01:44:35 blyshrQD6702-03 ntpd[10615]: time reset -0.499227 s
May 9 04:33:33 blyshrQD6702-03 ntpd[10615]: time reset +0.505044 s
May 9 05:34:31 blyshrQD6702-03 ntpd[10615]: time reset -0.497777 s
May 9 09:59:59 blyshrQD6702-03 ntpd[10615]: time reset +0.504214 s
May 9 12:00:48 blyshrQD6702-03 ntpd[10615]: time reset -0.498043 s
May 9 14:39:59 blyshrQD6702-03 ntpd[10615]: time reset +0.503576 s
May 9 15:36:45 blyshrQD6702-03 ntpd[10615]: time reset -0.502617 s
May 9 16:17:42 blyshrQD6702-03 ntpd[10615]: time reset +0.501971 s
May 9 18:41:55 blyshrQD6702-03 ntpd[10615]: time reset -0.499537 s
May 9 23:24:20 blyshrQD6702-03 ntpd[10615]: time reset +0.500757 s
May 10 00:04:11 blyshrQD6702-03 ntpd[10615]: time reset -0.498917 s
May 10 01:36:26 blyshrQD6702-03 ntpd[10615]: time reset +0.498337 s
May 10 03:03:20 blyshrQD6702-03 ntpd[10615]: time reset -0.499151 s
May 10 06:25:31 blyshrQD6702-03 ntpd[10615]: time reset +0.488652 s
May 10 07:32:14 blyshrQD6702-03 ntpd[10615]: time reset -0.497883 s
May 10 08:36:52 blyshrQD6702-03 ntpd[10615]: time reset +0.497758 s
May 10 09:01:54 blyshrQD6702-03 ntpd[10615]: time reset -0.497659 s
May 10 11:06:50 blyshrQD6702-03 ntpd[10615]: time reset +0.508501 s
May 10 13:19:02 blyshrQD6702-03 ntpd[10615]: time reset -0.503418 s
May 10 15:56:42 blyshrQD6702-03 ntpd[10615]: time reset +0.501392 s
May 10 16:33:27 blyshrQD6702-03 ntpd[10615]: time reset -0.500081 s
May 10 19:44:39 blyshrQD6702-03 ntpd[10615]: time reset +0.506874 s
May 10 20:03:25 blyshrQD6702-03 ntpd[10615]: time reset -0.500660 s
May 10 21:50:32 blyshrQD6702-03 ntpd[10615]: time reset +0.497117 s
May 10 22:23:22 blyshrQD6702-03 ntpd[10615]: time reset -0.498882 s
May 11 00:24:07 blyshrQD6702-03 ntpd[10615]: time reset +0.498878 s
May 11 01:34:50 blyshrQD6702-03 ntpd[10615]: time reset -0.499754 s
May 11 03:26:15 blyshrQD6702-03 ntpd[10615]: time reset +0.496915 s
May 11 03:52:41 blyshrQD6702-03 ntpd[10615]: time reset -0.499085 s
May 11 06:19:51 blyshrQD6702-03 ntpd[10615]: time reset +0.500363 s
May 11 06:41:02 blyshrQD6702-03 ntpd[10615]: time reset -0.497139 s
May 11 07:27:04 blyshrQD6702-03 ntpd[10615]: time reset +0.500734 s
May 11 09:29:05 blyshrQD6702-03 ntpd[10615]: time reset -0.501882 s
May 11 11:07:21 blyshrQD6702-03 ntpd[10615]: time reset +0.500786 s
May 11 11:27:53 blyshrQD6702-03 ntpd[10615]: time reset -0.500638 s
May 11 13:24:46 blyshrQD6702-03 ntpd[10615]: time reset +0.499988 s
May 11 13:56:48 blyshrQD6702-03 ntpd[10615]: time reset -0.500487 s
May 11 15:16:33 blyshrQD6702-03 ntpd[10615]: time reset +0.501159 s
May 11 15:48:59 blyshrQD6702-03 ntpd[10615]: time reset -0.498059 s
May 11 19:57:07 blyshrQD6702-03 ntpd[10615]: time reset +0.493805 s
May 11 20:37:28 blyshrQD6702-03 ntpd[10615]: time reset -0.499700 s
May 12 04:49:48 blyshrQD6702-03 ntpd[10615]: time reset +0.498131 s
May 12 06:05:53 blyshrQD6702-03 ntpd[10615]: time reset -0.498974 s
May 12 07:19:31 blyshrQD6702-03 ntpd[10615]: time reset +0.505644 s
May 12 09:24:08 blyshrQD6702-03 ntpd[10615]: time reset -0.493882 s
May 12 10:17:11 blyshrQD6702-03 ntpd[10615]: time reset +0.499613 s
May 12 11:33:22 blyshrQD6702-03 ntpd[10615]: time reset -0.501636 s
May 12 13:13:04 blyshrQD6702-03 ntpd[10615]: time reset +0.502545 s
May 12 13:55:38 blyshrQD6702-03 ntpd[10615]: time reset -0.499543 s
May 12 18:00:43 blyshrQD6702-03 ntpd[10493]: time reset +0.497910 s
May 12 19:56:23 blyshrQD6702-03 ntpd[10493]: time reset -0.498851 s
May 12 20:32:07 blyshrQD6702-03 ntpd[10493]: time reset +0.499918 s
May 12 20:53:16 blyshrQD6702-03 ntpd[10493]: time reset -0.503420 s
May 12 21:30:49 blyshrQD6702-03 ntpd[10493]: time reset +0.503824 s
May 12 22:03:27 blyshrQD6702-03 ntpd[10493]: time reset -0.503579 s
May 13 00:50:17 blyshrQD6702-03 ntpd[10493]: time reset +0.486401 s
May 13 01:39:18 blyshrQD6702-03 ntpd[10493]: time reset -0.501373 s
May 13 05:37:34 blyshrQD6702-03 ntpd[10493]: time reset +0.498619 s
May 13 06:53:00 blyshrQD6702-03 ntpd[10493]: time reset -0.497565 s
May 13 08:37:56 blyshrQD6702-03 ntpd[10493]: time reset +0.501582 s
May 13 11:04:10 blyshrQD6702-03 ntpd[10493]: time reset -0.496104 s
May 13 12:48:04 blyshrQD6702-03 ntpd[10493]: time reset +0.499957 s
May 13 13:07:04 blyshrQD6702-03 ntpd[10493]: time reset -0.499503 s
May 13 16:12:28 blyshrQD6702-03 ntpd[10493]: time reset +0.487547 s
May 13 17:27:31 blyshrQD6702-03 ntpd[10493]: time reset -0.499394 s
May 13 19:49:09 blyshrQD6702-03 ntpd[10493]: time reset +0.500664 s
May 13 20:27:51 blyshrQD6702-03 ntpd[10493]: time reset -0.497584 s
May 13 23:35:37 blyshrQD6702-03 ntpd[10493]: time reset +0.500260 s
May 14 00:07:56 blyshrQD6702-03 ntpd[10493]: time reset -0.498794 s
May 14 02:44:58 blyshrQD6702-03 ntpd[10493]: time reset +0.499945 s
May 14 03:26:02 blyshrQD6702-03 ntpd[10493]: time reset -0.497612 s
May 14 05:18:19 blyshrQD6702-03 ntpd[10493]: time reset +0.498591 s
May 14 07:31:42 blyshrQD6702-03 ntpd[10493]: time reset -0.497412 s
May 14 08:44:04 blyshrQD6702-03 ntpd[10493]: time reset +0.498127 s
May 14 10:23:45 blyshrQD6702-03 ntpd[10493]: time reset -0.499278 s
May 14 11:03:32 blyshrQD6702-03 ntpd[10493]: time reset +0.498536 s

Here is another example system that uses the same NTP internal server 172.20.103.5

[~]# cat /opt/DXi/system-serialnumber
CX1251BVE01114

[~]# cat /etc/ntp/step-tickers
172.20.103.5
127.127.1.0

[~]# ntpdc -p
     remote           local      st poll reach delay   offset    disp
=======================================================================
*172.20.103.5    172.20.103.38    2 1024 377 0.00002 -0.000261 0.15022
=LOCAL(0)        127.0.0.1       10   64 377 0.00000 0.000000 0.03044

[~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset jitter
==============================================================================
*172.20.103.5    129.6.15.28      2 u 853 1024 377    0.025   -0.261   2.646
LOCAL(0)        .LOCL.          10 l   38   64 377    0.000    0.000   0.001

This shows only one hop. This raises concerns about the two hops shown above.

# traceroute 172.20.103.5
traceroute to 172.20.103.5 (172.20.103.5), 30 hops max, 40 byte packets
1 (172.20.103.5) 1.014 ms 1.015 ms 1.009 ms

Summary

The ntpd daemon polls the NTP time server several times and averages the results, calculating the exponential average of RMS time differences.
Time shifts can produce unexplained errors in backup programs that check on file status (stat of file) after file creation or the writing of data.
DXi systems need to be time synced to the same clock as the Clients. Check the client time base and compare it to the DXi time base.
DXi systems could be in different time zones. For replication, check the DXi time sync between the Source DXi and Target DXi.
Check for jitter multiple times at random intervals in the 24 day cycle. The jitter in this example was fine most of the time, but random checks showed the root cause of +/- 1/2 second time shift reported in the /var/log/messages file.