I've just completed a short film and it's ready for festival submission.

Should I have a local public showing of the film? (I'm part of a group so we could have a film night open to the public) - does this affect eligibility for certain film festivals?

The film is on Vimeo but private - if I make the film public does that affect eligibility?

You're asking a legal question. IANAL. There's what I think is a quite reasonable definition for the US at https://www.wlu.edu/general-counsel/answer-center/copyright-and-intellectual-property/copyright-guidelines-for-showing-movies-and-other-audiovisual-works which I think also applies to the Australia and the UK - there was a UK case which I can't remember the details of but any copyright lawyer would know which seemed to reflect the same conclusion, i.e.

The showing of a movie will be considered to be a "public performance" if either of the following is true:

You will be showing the movie to people other than members of your family or a small group of your friends.
You will be showing the movie in a place that is open to people other than members of your family or a small group of your friends (for example, a classroom or the Commons Theater), whether or not any such people attend.

You have to look at the festival conditions. Showing it in public prior to some festivals, e.g. Tropfest, would disqualify it, making it public on Vimeo would rule it out, even if no-one watched it.