Most Valheim mods are client-side only, meaning that even if you had a dedicated server, you wouldn't have to install them on the server.
Very few mods (Valheim+, Plant Everything, etc) require installation on both client and server side.
When on a website like nexusmods.com or thunderstore.io, if a mod does not specifically mention being installed on a server, then it is a client-side only mod.