The use of the word “YubiKey” in that context is misleading. It is a reference to the Yubico OTP method, which uses a Yubikey (and a connection to the YubiCloud) to generate a 44-character one-time use code, which can be accepted as 2FA by some services (including Bitwarden). This is a very esoteric method, that is rarely used — but if you insist on using it as the 2FA for logging in to your Bitwarden vault, then yes, you will need a Premium subscription.
If you just want to use a YubiKey as a FIDO2/WebAuthn passkey (either for logging in to Bitwarden in one step, or as a two-step login 2FA factor to complement a different primary login method), then you do not need a Premium subscription.