You're probably scanning things in at a high resolution and full color. You realistically don't need more than 150dpi and grayscale, unless perfect visual quality is a must.
Try using something like NAPS2 to have more control over your scans. You don't need an absurdly high resolution, and you can scan in grayscale or black & white to make things a little smaller. For stuff you already have scanned, there are a bunch of pdf compressors, including plenty that do the work in your browser, but this one's my favorite.