Sounds like you downloaded the Static build rather than the Shared build. The Shared build makes the dll files available.
http://ffmpeg.zeranoe.com/builds
Select 3.2.4 / 64 or 32 bit to match your REAPER installation / Shared
That should get you the dll files.
Your approach to audio replacement works for me.
Once you have the shared libraries in place you should see the h.264 codec. As far as I know that's the highest quality render available for mp4/mov container.
I was able to choose 320 kbps aac for this render. I was also able to choose 24 bit PCM. Not sure if Final Cut will ingest a .mov with uncompressed audio, but it's worth a try.
Fran