Carrots contain vitamin A, or retinol, and this is required for eyes to operate in low-light conditions. If you have a vitamin A deficiency, eating carrots would correct this and improve your night vision! They won't make you superhuman though!
The rumour begun in World War II. To prevent the Germans finding out that Britain was using radar to intercept bombers on night raids, they issued press releases stating that British pilots were eating lots of carrots to give them exceptional night vision. This fooled the British public, as well as German military and an old wive’s tale was born!