The effect of KDF on unlock speed is independent of the number of items in your vault, since the KDF is only used to extract the account encryption key using your master password. The decryption of your vault contents using the account encryption key always happens after the KDF algorithm has been completed.
Very good note, albeit to clarify since some users get confused here: For most users, 90%+ of unlock time is the KDF - since more unlock time also necessitates slower brute-forcing. Some users with a lot of entries (500-1000+) will notice this time added on top (not scaled with KDF parameters as you point out).
And, as you point out, with argon2 you can use memory to slow down brute-forcing, while not affecting unlock time (if you in turn decrease iterations).