(The following applies only to Intel x86-compatible CPU's.)
"32 bits" refers to the word size, the largest directly-addressable width for the accumulator (EAX). The other general registers (EBX, ECX, and EDX) follow suit. In order to get this much data at once, the data bus should be 32 bits wide. However, this is distinct from memory addressing, which uses a segment:offset scheme to facilitate process separation.
There are at least three active segments: code (CS), data (DS), and stack (SS). In the "old days," the segment register was merely multiplied by 16 and added to the offset, meaning the address bus needed to be 20 bits wide (1 megabyte).
Beginning with the 80286, the new "protected mode" meant that the segment register pointed to a selector, which described a segment in terms of "where does it start" and "how far does it go." A segment could still be only 65,536 bytes long, but its origin could be anywhere from byte 0 to (16M - 1). In order to access this much memory, the address bus needed to be 24 bits wide.
In the 80386, this segment origin was extended to 32 bits, allowing a full 4G of directly-addressable memory, and the allowed segment length was also extended to 4G. Note that one could create a segment starting at 4G - 1 byte, and 4G long, so the segment + offset scheme would wrap around the 4G limit. It was up to the software to keep this from happening, or deal with it properly if it did happen.
In the Pentium Pro and later, the address bus was widened to 36 bits, giving 64G of addressable memory. Segments are still a maximum of 4G long, but their origins can go clear up to 64G-1. Conceivably, one could create 16 segments, all 4G long, without any of them overlapping.
None of this has to do with paging, which deals with turning virtual addresses into physical addresses. That's a separate discussion.