I believe that you’re taking a lot of extra time using your current methods. depending on which program you’re exporting (or if you aren’t including any interactive elements, printing) to .pdf to, you should have the option to adjust compression and exclude layers. If you intend your portfolio to be printed, then keep it at 150ppi and at the actual physical size of your intended document. If you intend it to be a 100% digital document, then you can lower the resolution down to 100-ish ppi (or keep it at 150 if you want people to be able to zoom in a bit) but adjust the physical size to better match actual screen sizes.
It helps to create portfolio specific images for your final document, so take that 24×36″ drawing and resample it down to 3×5″ or whatever size it will actually be in your portfolio, then save it out as a .jpg or if you’re concerned about color clipping, use a layerless .png or .tga.
If you have Acrobat Pro (and not just Acrobat Reader) you can adjust a .pdf’s file size by using the “reduce file size” or “create optimized” options under FILE – SAVE AS.
I have managed to get my 20 page 11×14″ portfolio down to 6.25MB with no noticeable loss in quality or clarity.