Why was it challenging for election organizers to determine who had the right to vote on the kansas referendum about slavery?

Border jumpers and a lack of paperwork: People had no official identification in the 1850s which allowed for Missouri residents to come into Kansas to vote on the issue of slavery.

This issue is known as Bleeding Kansas. As slavery was up for a vote in the Kansas territory, people from neighboring MO came across the border to influence votes and to vote themselves. With paperwork easy to forge and nothing official from the government it was easy to vote and move around anywhere. 

Prince henry the navigator is the man you're looking for.