For some reason, there isn't a default way to embed a local video file in a jupyter notebook.
If you're using a python kernel, you can make use of this hack, which inserts the whole video, base64-encoded, into the generated HTML. But because this runs in a code block, not a markdown block, it's dependent on the kernel you're running. Notebooks only support one kernel, so if the rest of your code is R, you'll need an R version.
Nobody else seems to have written the equivalent code for R, so here it is: