ITT: Your replication crisis experiences.
I'll start:
>Work with machine learning.
>See some paper with over 90 citations about a model.
>Github repo of the model being heavily used.
>Apply the model on the data provided by the same authors, and it fails.
>Apply it on other dataset, it fails.
>See in the repo issues that many people failed to reproduce the results in the paper and had the same problem as me.
>Paper was still published in a high impact journal and cited 90 times.
Why are things like this?
I'll start:
>Work with machine learning.
>See some paper with over 90 citations about a model.
>Github repo of the model being heavily used.
>Apply the model on the data provided by the same authors, and it fails.
>Apply it on other dataset, it fails.
>See in the repo issues that many people failed to reproduce the results in the paper and had the same problem as me.
>Paper was still published in a high impact journal and cited 90 times.
Why are things like this?
