Actually their eyesight is pretty good, it's just different. I recently learned the reason why those "dusk & dawn" attacks happen so often is because of how different they are. There are a few interesting differences:

• Human's stretch their eyes into the shape necessary to adjust focus close or far; basically we manipulate our eye shape to see depending on what we're focusing on (distance wise) with our muscles. Our iris also plays a factor in this for light (just like aperture of a camera).
• Sharks eyes don't change shape, instead they move forward and backward similar to a scope that you adjust magnification on. That's how they view things closer or further. Because of this, they have no way to adjust how fast they adapt to sudden changes in light.
• Human's have more cones in their eyes, thus we're able to see more of the colour spectrum
• Sharks have more rods (significantly more) which allows them to see better in the dark (makes sense based on their usual environments), among other things. Now because of the point above about not changing eye shape, it takes much longer for a shark's eyes to adjust with sudden light change (like dozens of minutes compared to our 2-3 minutes).

Understanding the above, what happens at dusk and dawn? Light either rapidly comes up, or goes down from the sun. This makes those shallower waters where a shark might be hunting reflect light more aggressively. Sharks looking up from the depth of the water then are affected by this and have poorer visibility during those points of the day.

So while it's a case of mistaken identify, it's a physical limitation of their anatomy combined with the adaptation of the environment. They only see in greyscale (similar to dogs) and their vision gets poorer when light rapidly changes. Us being the idiots we can be, decide to swim at dusk and dawn (when light changes the fastest) and then splash like we're a seal to an animal that can only tell black and white. Bam, mistaken shark meal.

This is the reason why those weird stripe type of wetsuits (just like zebras / tigers) result in less attacks, because despite the poorer visibility, there is enough difference in the grey scale look of these patterns that sharks know its not food.

To give a metaphoric equivalent: it's like someone deciding to dart out on the street when a car is coming from east to west during dusk where the sun is just blinding your vision. Person driving the car doesn't want to hit you and has no interest in hitting you. But given the circumstances, can't tell you're there and then you get run over. Maybe a terrible metaphor but hopefully you get the idea.