I wish there was an arXiv for failed projects

Posted in Uncategorized by Pierre Jacob on 6 October 2010

This XKCD idea of a “terrible-ex tracking system” is a good one. It reminds me of hatebook, the anti-social network that helps you keep track of your enemies.

Anyway, in the same spirit we could have a database of failed projects. It might be interesting to see why some models or methods don’t work; but when it doesn’t work it’s usually not published. And that doesn’t seem right because we can learn a lot of things by studying failures. In particular it would allow not to waste time on discovering things that have previously been discovered, considered as bad ideas and therefore not published.

It could be something like arXiv, but of course there wouldn’t be full articles. A few pages explaining what was tested and why it didn’t work should be enough. I realize that from an individual point of view, if a project doesn’t give good results it’s more strategic not to talk about it, to keep it in a drawer until a possible right moment when the project is restarted. Thus in a competitive atmosphere writing about your failures is like shooting oneself in the foot.

However from a global perspective it would definitely be efficient. It is a place on which people would first search when starting a new research project.


3 Responses

Subscribe to comments with RSS.

  1. Alex P said, on 7 October 2010 at 15:35

    I know something like that exists for machine learning, it’s called ”Journal of Interesting Negative Results in Natural Language Processing and Machine Learning”.

    Best, Alex

  2. pierrejacob said, on 7 October 2010 at 20:21

    Thanks for the link, this is exactly what I had in mind. Apparently they have only one article already!

  3. pierrejacob said, on 7 October 2010 at 20:24

    Plus their links page:
    is very interesting and points to other similar journals like the “Journal of Articles in Support of the Null Hypothesis” (what a title!)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: