I did a bit of testing yesterday: watching stuff in your local network over Jellyfin works just as well as Plex, but the UI is uglier.
Then I looked into enabling remote access for Jellyfin, but it's too much of a hassle for me right now, so I won't pursue that any further. I spend 99% of my time at home anyway, so it's pointless to invest so much time in remote access.
Just out of curiosity I checked out Plex Remote access (used the free trial for their remote watch pass) and it is much much easier to setup than Jellyfin's remote access, but it didn't work at all or only worked with heavy visual artifacts, don't really know what happened there. So I guess I'll keep on using Plex locally for the meantime.