Maybe Why is there no item counts in all sections of the sidebar? - #4 by Andrei is an option.
I would then suggest:
- Only show count + time estimate in collapsed view: this way you have a clean interface and the option to see more info
- Only show total time for projects when you click into the project, so the project list remains clean
What do you think?