Replaying a video on YouTube makes it buffer all over again instead of it playing instantly like Chrome does. Is there any fix for this?

Re-buffering happens if you replay a video after it buffers completely till the end and if you click on the timeline far enough back during buffering.

I've looked through many threads but didn't find any fix/workaround for this. I saw some users saying changing values for media.cache_readahead_limit and media.cache_resume_threshold to 99999 fixed it but that didn't help and actually caused Firefox to stop buffering midway through videos.