I've given my theory in the past about what I think happened. I think EA had an engine designed and built for a Sims-Multiplayer-type game (maybe an updated Sims Online). However, they decided after Sims Online fell apart that it was the wrong direction to go. I think then, the resources that they had developed to that point were given to a new group of developers and that group was told, "Take this code and make it work for Sims 4". Well, the engine was designed for every Sim in the world to be played by another player. So there was no story progression, no optimization for things happening anywhere else in the world than on the active lot...because each person was their own active lot and their computer would be handling their own things.
I think the team that is working on the Sims 4 is probably doing the best they can with what they were given and trying to recover as much as possible.
But, that's just my tinfoil-hat conspiracy theory.