If a spell states to add "spellcasting ability modifier" do I add just the modifier (wis or int) or the full spellcasting ability (wis + proficiency)
Spiritual Weapon states damage is "1d8 + your spellcasting ability modifier". As a Cleric my "spellcasting ability modifier" is wisdom with +3.
Does this really mean I only get 1d8+3 since it specifies "modifier" instead of just "spellcasting ability" +5?
You're conflating two separate things.
Spell attack bonus is to hit the target, which gives you a bonus of your spellcasting ability modifier (Wisdom in your case) plus proficiency. That's the "melee spell attack" referred to in the spell's description.
The damage done by the spell if it hits is 1d8 plus your ability modifier (Wisdom again). So yes it's "only" 1d8+3 for you right now. But remember, spiritual weapon uses your bonus action to make an attack/move, meaning you can attack with it and your own attack, or attack with it and cast a spell, or use an item, and so on, and all without requiring concentration to maintain the spell. It's pretty powerful in that regard.