Counting occurrences of a pattern in a file, include zero for missing patterns

I am trying to count occurrences of a pattern (listed in pattern.txt) in a file (file.txt) using:

grep -o -w -f pattern.txt file.txt | sort | uniq -c > output.txt

This works great, but I would like the output to also include 0 for patterns that do not occur in the file.

How might I accomplish this?