CPUs are unbelievably complicated. They basically spend most of their power optimizing the code they're executing at runtime, and the code they run had its basic structure designed a long time ago. Most of the recent performance gains CPUs have made are from improving their "instruction-level parallelism", which is funny because the code that a given CPU core executes isn't parallel at all.

GPUs can continue to get better (for now) because they're not a fixed target (you run "shaders" on them, not machine code) and the kind of workloads they do are very conductive to being split across multiple processors.