A battery creates a potential difference, or voltage, which is really just storing electrical potential energy just as a pump would by moving water up against gravity, thereby storing gravitational potential energy. To complete the analogy, as current flows through the circuit, this potential energy is released and can be used by the components in the circuit. This is the same as water falling back down and turning some wheel or turbine and converting its gravitational potential energy into mechanical energy.
