With the new toolbar, sometimes when I check the review queue, I'll see a red dot.
However, when I go into the queue, it says that its been cleared. The red dot will persist for a bit afterwards.
The red dot is showing up when there are a lot of items in the review queue - this indicator is site-wide and shared across all users.
What it doesn't do is take account of your specific review work.
So, what happens is that you have gone through all items in the review queue, however, not all of these items have been removed from the queue as other people need to also review them.
This means that though you, personally, don't have any more review items to work on (as you have gone through all of them), they are still in the system, waiting for other people to complete them.
I appreciate this isn't ideal (being notified about work required in a review queue only to find that there's nothing for you to do there).
The current design is this way due to performance considerations - fetching the review queue status + accounting for queue items already reviewed by the specific user is expensive :(
Hope this explains what you are seeing and why it is this way.