How to properly use transient keyword when de-serializing?

I have the following class

public class Num implements Serializable, Observable {
    private int num; // number which should be serialized

    // listeners who are watching for changes in this Num
    // No need to serialize these -- pointless to do so.
    private transient Set<Observer> listeners = new HashSet<>();

    public void addListener(Observer o) { listeners.add(o); }

    public void removeListener(Observer o) { listeners.remove(o); }

    public void set(int newVal) {
        if (num != newVal) {
            num = newVal; // set the value.
            for (Observer o : listeners)
                o.notify(); // Notify listeners that the value changed.
        }
    }
}

When I serialize this class, it works well and num is saved. When I de-serialize the class, num is loaded but listeners is not and is set to null. My program then crashes on the line for (Observer o : listeners).

I came up with some solutions but they are all terrible solutions.

1) Have a setup method which 'reconstructs' the transient fields.

public void setup() {
    if (listeners != null) throw new Exception("Already setup!");
    listeners = new HashSet<>();
}

This way is annoying through because the de-serialization method needs to remember to setup the object. It's very unintuitive for other people working on the project.

2) Have the set method automatically check and repair itself

public void set(int newVal) {
    if (num != newVal) {
        if (listeners == null) listeners = new HashSet<>();
        ...
    }
}

This way is also bad because the check will keep happening over and over, even though it only needed to be done once.

3) Remove Observable from class Num, get rid of listeners, etc. Then make a non-serializable class which contains a Num instance.

public class Num implements Serializable {
    public int num;
}

public class ObservableNum impements Observable {
    private Num n;

    public ObservableNum() { n = new Num(); } // constructor
    private ObservableNum(Num n) { this.n = n; } // constructor

    ...

    public static ObservableNum loadNum(...) {

        ObjectInputStream ois = ...;
        return new ObservableNum((Num) ois.readObject());
    }
}

But this way also seems needlessly complicated. Surely there must be a better solution? How is transient properly used?