For those who don’t know what this post is about. Well it turns out that a membrane protein crystallographer made a mistake in a computer program that resulted in certain structures “having the wrong hand” . The mistake was discovered when another crystallographer solved a structure of a related protein and ended up with an almost identical structure but in a different hand .
The issue has set off an interesting debate among crystallographic and non crystallographic circles , most opinions center around the fact that “these were hot structures and so things were overlooked” and this is a result of “sloppy science”. While I agree that things do get ugly when the race to publish overtakes the desire to do good science .I don’t think Chang or anyone else involved in this has been “sloppy” in as much as the actual error committed by the group has been committed by several prominent crystallographers in the past.Each of those errors led to new ways of checking structures for correctness and an overall awareness of the importance of using “published” software protocols instead of homegrown hacks.
To give the Chang group some credit , they were , and are working on some very difficult problems in crystallography . If the group is guilty of anything it is that they did not double check their structures when they flew against the face of several biochemical results. But considering the low resolution of the structures and the amount of data analysis involved in getting at these results. It is not surprising that the error remained undetected till Kaspar Locher solved the structure of a similar protein which “exposed” the error. To use”exposed” in this context makes it smell like the mistake was in some way intended with a desire to hide some flaw in the work. I am convinced that the error made by the Chang group was an honest mistake. The quick retraction is a testament to the groups desire to set the record straight.
Several bloggers have commented on this issue by asking that computational biologists check the results of their program with “positive controls” and “test cases”. Lets just say that this is easier said than done in crystallography. The error crept in at a very early stage in the analysis and would have been caught only if the group used their homegrown program to re-solve another similar low resolution structure starting from the raw images or early stage un-reduced data. It is still very difficult to obtain “raw” data for any published structure , without which re-solving structures “ab initio” is still not possible.
Several opinions also use the issue to question the peer review process. It is entirely un-reasonable to expect the referees or the protein data bank which stores all such crystallography data to check every step in structure solution for correctness given the constraints of time ,funding and man-power.
If anything all of these make a case for open access publishing , where we are not constrained by the requirement to keep methods brief and being concise can hide potential sources of error. In this context , the “home-baked” program at fault here was not mentioned in the methods section. Again, I would again not blame the authors for this.
This also makes the case for everyone to seriously fund and encourage “repeat science” ( as my adviser used to say , that’s the “re” in research) without which such errors would probably go undetected in the future.
To conclude, similar errors have been made in the past by crystallographers. Checking the correctness of structure solution has improved significantly after such problems were discovered in the past. I am very confident that the retraction and the awareness it has raised in the crystallography community , will make us all check and double check our data especially when it flies in the face of several published results.
Refs: Science (Dec 22, 2006),
CCP4 bulletin board : comments from Crystallographers ( see posts in the Dec 22 2006 to Dec 31 2006 range with title Retraction of ABC transporter structures – were there warning signs) or these links 1 , 2 , 3 , 4 , 5 , 6