I'm not totally sure of the cause, but I have an inkling it's related to export region size being fractional frame.
I have a silent video I'm exporting as MJPEG. I noticed there's a blip sound at the end of the exported file. When I checked my REAPER session, I noticed that the time selection used for the export bounds wasn't snapped to the frame grid and that the noise started exactly when the last frame finished and stopped exactly where the time selection finished.
Coincidence? I think not.
Sorry I don't have more time to investigate right now.