If I'd get an USB stick of 16GB or 32GB, would it be possible to put a file that's larger than 4GB on it? What filesystem would it need for that, that would work on both Windows and Linux, and on other people's PC without them having to install something?

I believe the default flash drive file system is FAT32, which doesn't support 4GB files. Since Windows is the limiting factor when it comes to supported file systems, I would reformat it as NTFS.