[Mod edit: This thread originally began after this post].As another lifelong restaurant person who has worked in such situations as a maitre'd, I can perhaps answer some of your questions.
A restaurant such as this usually plans out its evening by mapping out the tables according to the reservations ahead of time. Only a certain number of reservations are taken at any given time to ensure that the kitchen and the wait staff is not overwhelmed. Although a restaurant may have a certain number of seats available (let's say 100), most of the time it cannot handle having all of those 100 seats taken at the same time. The kitchen just isn't big enough. The maitre'd works with the chef to determine how many people the kitchen can take at one time, and plans the seating along these lines, taking into account customer seating requests and multiple other factors, including the need to have the table back for a later party or the table's ability to flip up to accommodate a larger party, etc. All of this leads certain restaurants to adhere to a policy of only seating complete parties, and only seating parties at their reservation time.
I have known a few people who have worked at North Pond over the years, and from our conversations I have learned about the goal of the chef and the restaurant in general, which is to cook seasonal cuisine, bought daily from local farms in quantities appropriate to the number of reservations. The kitchen is small and the chef chooses not to take large parties when it's busy or many tables at one time to ensure that the food receives the correct attention and preparation. Unfortunately, this kind of reservation system can lead to some misunderstandings, such as the misconception that they are out to gouge the guest by forcing them to have a drink at the bar or in any other way (knowing the sincerity of the people I knew there, I would be truly surprised if this were indeed the case).
I can't speak to specific instances, of course, but I would venture a guess as to why a walk in table of six would be seated immediately when a table of four was made to wait: perhaps there was a slot specifically for six open at that time on the computer, or in a section of the restaurant that was due to be seated (most seating goes on rotation so as not to slam any particular server). North Pond in particular does have to deal with regulations other restaurants do not as well, being in the park: the park closes at 11 pm, and that's why they can only take tables until a certain time as opposed to equivalent upscale restaurants (and if I had a nickel for every time someone has called me to tell me they'd "only be 15 minutes late" and arrived 45 minutes later, I wouldn't be in this business anymore).
Of course individual attitude from hostesses or anyone else is inexcusable, but I'm sure in the above general situations, a hostess probably feels badly enough that she can't seat you immediately. The idea that people in the service industry receive some perverse pleasure from denying customers something is a fallacy, at least at this kind of fine dining restaurant (which is similar to my experience). Making guests happy is why I and many others are in this business, and when that doesn't happen, I do whatever I can to remedy that. In these situations, what I can do is seat you as soon as I can, and be honest with you about it.
More than anything I wanted to speak to the bizarre idea that just because a restaurant has an open table, that that means you can or should be seated. The plan a maitre'd follows is as much for the guests' benefit as his own, to ensure that his food and service is as good as possible. If I were to give you any old table without regard to how busy the kitchen was or how slammed your server was at the time, not only would I not be doing my job properly (by not planning properly), you would most likely have a poor experience. I think even the most ADD among the complainers could understand why a restaurant would not want to have most of its tables seated at the same time. So basically what I would implore upon you all is to try and understand that although you may see an open table, it may not be
yours, for many different reasons.