This tutorial is for Forge servers.
You'll first need to find the mod you wish to install. Downloads can be found on a variety of sites. Many mods can be found on CurseForge. Make sure you trust the source of your downloads.
In your server's root directory, locate the
/mods folder, if it's not there then create a new directory named
/mods. Navigate into this folder, and upload your mods' file there! The vast majority of mods require you to restart your server to take effect.
Be sure to use the correct version of the mod. Unlike Spigot/Paper, mods are likely not compatible cross version.
Nearly all Forge mods need players to have them installed as well. Consult the documentation of your mod to see if this is required. Some mods may provide a seperate file for players and servers.