Architecture Best Practices : Javascript and Java

I've been using a certain pattern to store my data objects like this :

JavaScript :

var person = {
    data : {}, // json data will be populated here
    getId : function(){
       return this.data.id;
     },
    getName : function(){
      return this.data.name
    }
  }

Java :

class Person {
    HashMap<String, Object> data = new HashMap<>(); // json converted to hashmap

    public String getId(){
      return this.data.get("id");
    }

    public String getName(){
      return this.data.get("name");
    }
}

Benefits of using this pattern -

1. Never have to worry about serializing and deserializing

2. Can save and load the 'data' object anywhere easily(localstorage or elsewhere)

3. Never lose reference to the 'person' object anywhere in the code even though internally the object changed

4. Can have custom logic for getters and setters

5. Don't have to worry about API changes on the server - accessing properties only happens inside getter functions so if ever a property value needs to be changed or that property has been mapped to another property in an object - we can just change the property in the getter and we're done. For eg - if we've been using person.name everywhere in the code and we now need to use person.fullName - if we have the getter we can just change it in the getter and we would be good to go.

Downsides -

1. The only downside I'm assuming is reading might be a bit slow

Wanted to know what other downsides there might be to following this pattern? Or any glaring red-flags as to why this should be avoided?