Avatars are hosted directly on the project server and are (usually) streamed using the same connection that the html page was downloaded with (ie. pretty fast).
Signatures are on different 3rd party servers. I guess it would be a bad idea for the projects to be responsible for the caching. Actually if the 3rd party sites make their signatures in the right way they will not slow down the page loading at all. Only BOINC.dk does this at the moment (correct me if I'm wrong here, I'm possibly pretty biased...).

The forum preferences have an option to show images as links. Speeds things up considerably! :)

<a href="http://www.boinc.dk/index.php?page=user_statistics&amp;project=cpdn&amp;userid=355"><img border="0" height="80" src="http://355.cpdn.sig.boinc.dk?188"></a>