They Stole the Authentication

Some of the statements that I made in my recent appearance on Mike Creavey’s Gracious Guest podcast have attracted some criticism from Hugh Farey, most of which I believe is completely unfounded and unjustified, so please forgive me but I feel it is necessary for me to respond in detail to some of his observations.

After reading Hugh’s blog earlier this week, I wrote to him refuting many of his comments and I’m pleased that he has graciously accepted that some of these were incorrect and has since apologised and corrected them. For example, he now accepts that he was wrong when he stated that neither Ernesto Brunati nor Remi Van Haelst had identified errors and anomalies in the Nature report shortly after its publication. He had claimed that neither of them had made any such comments until 1997 but as I pointed out to Hugh, there are multiple sources that make it clear that both these men had raised the same errors and anomalies that I outlined in the podcast during a 1989 Paris Shroud conference and that they continued to repeat their claims in the years before 1997.

However, Hugh’s blog makes other criticisms of my podcast which he does not wish to retract and so I’ll take the opportunity here to explain why I think the statements that I made were justified.

During the podcast I questioned the significance level value for the Shroud dating result that was published in the report. This was recorded as 5% but anyone repeating the calculation will find it gives a result of 4.18%, which rounds to 4%, not 5%, an error which was discovered by Brunati shortly after the publication of the test report. Hugh’s objection is as follows:

Sure, anybody can use an online calculator and find the precise value, and it is 4.24%, but it seems that in 1988 the easiest thing to do was to look it up. In the table above, for instance, we find that successive chi-squared values of 4.605, 5.991 and 7.378 correspond to probabilities of 10%, 5% and 2.5% respectively. As 6.35 is closer to 5.991 than it is to 7.378, it indicated a probability of 5%, although in her own notes, Leese was careful to put “<5%.” Either way, the weak correlation was both noticed and carefully discussed, and appropriate consideration given to the final conclusion.

This response, and earlier comments in his blog, state that the British Museum staff working in 1988 would have used tables, such as those from a 1970 publication ‘Statistical Tables for Science, Engineering and Management’ by J. Murdoch and J.A. Barnes, to calculate the significance level. These tables, as Hugh points out, have a low level of accuracy and prior to the advent of cheap electronic calculators in the early 1970s, they were no doubt widely used. However, there is a simple formula that can be used to calculate the significance level using the measurements provided in table 1 of Nature’s report [for those interested it is SL = 100 x exp(-chi-square/2)] and there can be little doubt when analysing the C-14 test results in 1989, the statisticians would have used a calculator or even a spreadsheet application on a PC to calculate these values.

I understand that Hugh now accepts that the calculation above is correct and indeed he has used it to check and correct the value of 4.24% given by the online resource that he had previously been using instead. However he still maintains that the British Museum statisticians, working in 1989, would have used a set of printed tables rather than a calculator to determine the significance level, even though this method was far less accurate. This does not seem at all plausible.

Hugh also disputed my statement that table 2 of Nature gave incorrect values for the weighted mean and margin of error for both Arizona and Oxford’s Shroud results. For Arizona, he argued that they made eight measurements which could be treated as four pairs and then attempted to justify why their margin of error was stated as 31 instead of 17 by stating that, “One possibility, which at least results in the error finally published, is that the second of each of the four pairs was simply ignored for error calculation purposes.”

He didn’t however mention that in 1988/89, when the peer reviewers were checking the report, no-one other than staff from the three laboratories and the British Museum knew that Arizona had made eight measurements. This was only revealed several years later by Morven Leese. As far as the peer reviewers were concerned, the only measurements made by Arizona were the four listed in table 1 of Nature’s report.

As anyone who has listed to the podcast will know, the key point that I was making was that there were clear errors and anomalies in the Nature report, this being just one, that were apparently overlooked in the peer review process but were identified by Van Haelst, Brunati and others shortly after the report was published. In 1989, The only way to calculate the weighted mean and margin of error result for Arizona from the information contained in the test report was to use those four measurements in table 1 of the report, which gives a margin of error of 17, not 31 as documented in table 2. If the reviewers had performed their own calculations to check that the mean dates and scatter given in table 2 were correctly derived from those in table 1, they would have spotted the error. They did not make any comments in their reviews about this clear and obvious calculation error and I can only conclude from this that they failed to adequately check the report.

Hugh also objected to my comment that there were errors in the published result for Oxford, stating that “It was apparently Oxford’s practice to round all its figures to the nearest five (see their raw results at the top of this post). That being so, it is quite incorrect to average those figures to a precision greater than five. To adjust ‘750±30’ to ‘749±31’ is ignorant, not clever.”.

However, it wasn’t until 28 years later that the raw data became available and so no-one back in 1989 would have had access to the raw results that Hugh referred to above and which he used to justify his claim that this is not an error. In any case, I would also question whether it would have been good practice to use a different calculation for the weighted mean and margin of error of Oxford’s measurements (to be rounded to the nearest 5) than the one used for Arizona and Zurich (not to be rounded).

Hugh’s final comments refer to what he described as ‘…an unjustified assault on some of our most celebrated scientists’. Clearly Hugh, and presumably Dan Porter, have a high regard for the scientists who worked on the radiocarbon dating project despite several reasons to doubt their professional integrity and competence based on their performance in this project. Here are just a few:

• The 1986 planning meeting in Turin agreed that it was important to take samples from multiple areas of the Shroud in order to ensure the integrity of the test. As Bill Meacham, who was present at that meeting, and other experts pointed out (Marian Scott, Paul Maloney, Stewart Fleming, Dr Baxter of the International Radiocarbon Calibration Programme, etc) if the sample was taken from only one area, there was a risk that the date measurement would not be representative of the cloth as a whole. Despite this, at a subsequent planning meeting held in London in January 1988, the directors of the three laboratories asked that their samples be taken from a single site in order to better guarantee the homogeneity of the results. It’s clear from this that these three laboratories were prepared to sacrifice the likelihood of obtaining an accurate date for the Shroud in favour of increasing the chances of getting mutually compatible results. Their motives and that decision are therefore highly suspect and reflect poorly on their scientific integrity.
• Michael Tite failed to inform relevant parties of his decision to include a third control, with the result that the samples of this material had to be placed in envelopes rather than metal canisters that had been prepared for all the other samples. This last minute, poorly communicated change to the plan is bad management that falls below the standards expected of a competent head of scientific project.
• When handing over the samples to the laboratories, Tite provided them with receipts which gave the known dates of each of the control samples, effectively undermining their value as controls. He later acknowledged that “It would have been better if the labs did not know the dates. If I did it again, in hindsight, I would do it differently.” Again this is poor management and well below the standards expected of a competent scientist.
• Even though the blind test protocol had already been abandoned due to the Shroud fabric being easy to identify, after the samples had been distributed to the laboratories both Michael Tite (in a BBC documentary) and Edward Hall (in an interview with the Independent Newspaper) claimed that there was no way that they could distinguish the Shroud from the controls. These claims were clearly dishonest and reflect poorly on the character and integrity of these two scientists.
• Despite having signed non-disclosure agreements (NDAs) in an attempt to ensure that the results of the test would be kept secret until publicly announced, there were multiple breaches and leaks. Hall, who had signed this agreement, later claimed that “Frankly, I think it was a hopeless prospect to keep the result secret. You couldn’t. With the best will in the world”. The absence of personal integrity revealed by this comment appals me. Throughout my career I had to sign many NDAs when working with High Street banking clients whenever they had to give me and my teams access to sensitive internal information. It was very clear to us all that there would be extremely severe consequences for anyone who breached those NDAs and had the CEO of the company ever made a statement similar to that of Hall, contracts and orders would be immediately cancelled and the company share price would hit the floor.
• As I noted in the podcast, there were multiple errors and anomalies in the published test report that should have been spotted. Michael Tite didn’t even check records of the sampling operation when incorrectly documenting the dimensions of Shroud cutting as 70mm x 10mm. He apparently felt it was acceptable to ‘wing it’ instead by entering “… a very rough figure based on my memory.” This is a sloppy lack of attention to detail and accuracy when producing such a high profile scientific paper.
• As stated earlier, the table of measurements published in the official test report listed only four measurements for Arizona’s Shroud sample when in fact eight measurements had been made. The report gave no indication that these eight measurements had been consolidated into four. This is clearly inaccurate and misleading reporting which surely cannot be considered to be acceptable in scientific circles.
• Brunati, Van Haelst and others wrote to the British Museum and the three laboratories with questions about apparent errors in the test report but didn’t receive satisfactory responses. According to Van Haelst “Of all the scientists involved in the radiocarbon dating of the Shroud, only Dr. Tite, Dr. Hedges, Dr. Morven Leese and Dr. Wölfli replied to some of my many questions. Except for Dr. Hedges, all answers were evasive. Prof. Hall replied by saying that he could not spent [sic] his precious time to answer the questions posed by scientists of little standard, blinded by faith.” This lack of respect for fellow scientists and refusal to answer questions is hardly acceptable behaviour from “some of our most celebrated scientists”.
• Cardinal Ballestrero’s scientific advisor, Luigi Gonella, gave his views about these scientists after having worked with them over many months: “They behaved like dogs. I protest for their absolute non- professionalism in the field of ethics. I protest about the infamous way they did it. I told them to their faces that they are mafiosi”

Unlike Hugh, I don’t believe that the professional and personal integrity of scientists that behave in this way is something to be celebrated.