Should a builder accept primitives or value objects

Given an Address at a minimum must have a $firstLine and $postcode but can contain optional properties, I am looking to implement a builder to ease the construction of an Address.

An abridged Address might look like:

class Address
{
    /**
     * @var AddressLine
     */
    private $firstLine;

    /**
     * @var null|AddressLine
     */
    private $secondLine;

    /**
     * Other properties excluded for brevity
     */
    ...

    /**
     * @var Postcode
     */
    private $postcode;

    /**
     * @param AddressLine $firstLine
     * @param null|AddressLine $secondLine
     * ...
     * @param Postcode $postcode
     */
    public function __construct(AddressLine $firstLine, AddressLine $secondLine, ... , Postcode $postcode)
    {
        $this->firstLine = $firstLine;
        $this->secondLine = $secondLine;
        ...
        $this->postcode = $postcode;
    }

    public static function fromBuilder(AddressBuilder $builder)
    {
        return new self(
            $builder->firstLine(),
            $builder->secondLine(),
            ... ,
            $builder->postcode()
        );
    }
}

The above seems to make sense for me, a public constructor which protects its invariants through typehints and allowing traditional construction, additionally a factory method which accepts an AddressBuilder that might look something like the following:

class AddressBuilder
{
    public function __construct(...)
    {
    }

    public function withSecondLine(...)
    {
    }

    public function build()
    {
        return Address::fromBuilder($this);
    }
}

In regards to the AddressBuilder, should it accept primatives which are validated during the build() method, or should it expect the relevant Value Object?

With Primitives

public function withSecondLine(string $line)
{
    $this->secondLine = $line;
}

public function build()
{
    ...
    $secondLine = new AddressLine($this->secondLine);

    return new Address(... , $secondLine, ...);
}

With Value Objects

public function withSecondLine(AddressLine $secondLine)
{
    $this->secondLine = $secondLine;
}

public function build()
{
    return Address::fromBuilder($this);
}