December 31, 2006
Revised January 7, 2007
This is just a quick report to document the OD cable swap found by Hans and Jeff, and its fix.
Hans' Eagle Eyes Spot an Anomaly
In early December 2006, Hans and Jeff noticed some strange OD hit statistics. Below are emails:
Date: Sun, 3 Dec 2006 23:34:40 -0800 (PST) From: Hans-Gerd Berns <firstname.lastname@example.org> To: Bill Kropp <email@example.com>, Kate Scholberg <firstname.lastname@example.org>, Hank Sobel <email@example.com>, Ed Kearns <firstname.lastname@example.org> Cc: Eric Thrane <email@example.com>, Jeff Wilkes <firstname.lastname@example.org>, Aaron Herfurth <email@example.com>, Jeff Griskevich <firstname.lastname@example.org> Subject: swapped OD paddle card-to-QTC cables found Hi OD folks. Today we (first shift, Jeff and I) examined a strange pattern on the OD hitmap we noticed earlier. The hitmap showed a cluster of 6 OD channels near the top of the quad 3 wall section with higher hit statistics than their neighbors, while another cluster of 6 channels near the bottom of the wall showed lower statistics than their neigbors. From the tube IDs we suspected the possibility of a swapped cable, and indeed found black ribbon cables 07 and 08 in crate 1 of hut 3 swapped at the QTC inputs. We swapped them back and started a new run (31671), and now the hitmap looks smoother again, the two clusters gone. So, paddle card channels 3.1.07.XX and 3.1.08.XX were swapped for a while until today!! We don't know when the swap happened by mistake, probably earlier in the summer at the beginning of SK-III when we were fighting multiple electronics (QTC,HV) problems. <...> Cheers. - Hans Date: Thu, 7 Dec 2006 15:57:27 -0800 (PST) From: Hans-Gerd Berns <email@example.com> To: Kate Scholberg <firstname.lastname@example.org> Cc: Bill Kropp <email@example.com>, Hank Sobel <firstname.lastname@example.org>, Ed Kearns <email@example.com>, Eric Thrane <firstname.lastname@example.org>, Jeff Wilkes <email@example.com>, Aaron Herfurth <firstname.lastname@example.org>, Jeff Griskevich <email@example.com>, Fannie Dufour <firstname.lastname@example.org>, Ishihara <email@example.com> Subject: Re: swapped OD paddle card-to-QTC cables found Hi Kate. Ok, I'm glad to hear the effect of a swapped QTC input cable isn't that bad after all. I don't have a tool on hand to visually check the hitmaps of past runs, but just from the rough tube hit summaries (logged on sukant for every run) I can see that the cable swap goes back to July. Up to run 30668 (July 17/18) the statistics for tube#'s 1032...1055 (paddle card#'s 3.1.07.01 ... 3.1.08.12) seem to be in the right order. Then statistics are swapped for cards 3.1.07 (1032...1043) and 3.1.08 (1044...1055) starting with run 30701 (July 18/19) until run 31670 (Dec 4) when we discovered the swap. According to the run summaries, the OD was turned off for almost the entore first shift on July 18 in order to run LED test runs. I guess some OD work (QTC repair?) was done while the OD was off, since we had a lot of electronics problems going on until things ran more smoothly after mid July. Cheers. - Hans
Two cables, with 12 tube signals each, were swapped at the QTC inputs for several months. One can see this manifest in Figure 1 showing OD hits (normalized by number of events with OD data) vs channel number.
Clearly the hit statistics are swapped around channels 1032 to 1055, for the green and blue lines (runs 30701 and 31670). The effect is also visible for the hitmap by tube numbers (note that the mapping between tube number and channel is not completely regular).
The first software fix I tried was to swap the cable numbers at skread time, before filling the commons with OD data. However this turned out to be problematic for remaking OD calibration constant files, because pedestals (which are electronics-dependent) and gains (which are tube-dependent) are entangled in the OD calibration constants (pedestals and gains are not stored as independent numbers). So, I made a more fundamental fix: this was to create an additional OD PMT map file, pmtmap2.anti, containing the new channel-to-cable number mapping for the cable-swapped period. This software fix allows simple regeneration of OD calibration constants for periods with and without the cable swap.
This change however entails an additional file that must be kept in the calibration constant area, odmap_version, which keeps track of OD map versions. Currently there are just two OD map versions, pmtmap1.anti (the standard one) and pmtmap2.anti, but in principle more could be added. Code to read the odmap_version file and handle the OD map versions was modeled on the tqa_version file-reading code and added to the skrd routine anticalsk.c. All required routines and files were committed to the repository on January 6, 2007.
Figure 2 shows hitmaps made with the new skrd library with the fix. The channel hitmap looks the same as it did before the software fix, as expected (since electronics channels did not change, and the channels did indeed have different tubes attached to them for different periods). However the tube hit map now correctly shows the same pattern for all runs.
OD calibration constant (tqa) files will need to be updated for these channels, for the periods corresponding to each configuration. OD calibration constant generating code was also updated to reflect the new skrd library.
Two OD cables were swapped for several months at the beginning of SK III. The swap was compensated for in software by means of new code added to skrd to handle different OD channel-to-tube maps for different periods.
Overall this problem affected 24 out of 1880 tubes, so it should not have a giant effect on anything - but best to fix it.