A multimeter is the best method for measuring a voltage. And you can position the clamps of your multimeter directly on the battery terminals. Positive reads as positive, and negative reads as negative. And you listed voltage, and you should have a little over 12 volts with the car switched off. That’s a battery that’s in decent condition.
And you’ll find that the voltage increases as you get closer to ideal. This shows that the battery is being charged by the alternator. We’re generating 13.8,13.9 volts. So, at this stage, you want to start gradually increasing the engine’s rpm.