Includes only if required?

Hey, normally I prefer to do all my includes (classes + config files) at the beginning if my php files. But recently I thought about including some classes only if they are needed later in my code (like pear Mail and Mail_mime only if an e-mail is about to be sent) because of speed issues. So my question: Is this something I should consider? Or is it something I normally will not notice? -eddy