When you read all the threads inside a "sub forum" the button stays on instead of switching back to

Yep, that's how it works.

It's kind of a limitation... we don't know a good way of finding out if every thread in each forum is read or not...

There are ways, of course, but they would probably have a noticable negative effect on speed.