The short version without going into details:
Case 1: After Effects output H264(.mp4) = Bad
Case 2: After Effects output .avi/.mov + feed into Adobe Media Encoder to output H264(.mp4) = Good
I've been encountering this problem for quite a while now, having rendering out various image formats such as PNGs or TIFFs, then going forward to composite in AE.
Occasionally, if not all the time, if I output my footages in AE as .mov or .avi the quality is amazing as expected, but with the big file sizes not fit for uploading. So I go with H264 as the common standard. The .mp4 that comes out looks terrible.
When I say terrible it meant that I see blurry or pixelated edges, as if it's still buffering. Back then I did imagine it was just somehow buffering as it plays, or the loss of resolution is inevitable from compression; because all other formats I tried didn't seem to give me better results either. Until one day someone looked at my footages and said my renders look bad (could look better) because of the blurry edges..etc
I wanted to scream for my innocence, it's AE's H264! Not me! But I withheld, because none of my peers seem to encounter this problem, H264 looks great they say. I debated if it was my image format, but TIFF is so common and popular there is no way it's cursed to have bad H264 outputs, can it??
Eventually after some digging I've came the conlusion that I need to encode it somewhere else other than AE. So that's what I did, I output with AE into .mov/.avi, because I knew those looked okay. Then I opened the file in Adobe Media Encoder, and made it ouput as H264. The .mp4 came out slightly bigger than the one I'd get from AE, but blurry free and looks pretty much the same as my .avi/.mov counterparts, mission accomplished!!
So yeah, even though this seems to be a rare issue, it bugged me for a long time without knowing this solution/workaround. So hopefully someone else will benefit from this too.
David Sushi