Leaving "More Is Better" Behind... Theoretically