To spoil a ballot (it's called, rather than a vote), one marks it improperly, perhaps for example by voting for more than one candidate for the same office, or by signing your name on the ballot or doing anything other than the prescribed format of completing the ballot.

To invalidate a vote would, I suppose, mean the same thing, but it's not the phrase I hear used.

To abstain from voting but still appearing at the poll, (in Canada at least, where we still use paper ballots) a citizen takes the ballot from the officer, folds it, and returns it to the officer. The officer marks the roll that you have voted (you can't change your mind) and deposits your ballot in the Declined pile where it will be counted, later.