Personally, I tend to support direct-trade products over fair trade. However, I'm quite pleased when I can purchase something that brings me closer to the grower. More than fair-trade, I also like the idea developing projects dedicated to sustainable agriculture in Palestine. I see that as absolutely positive. I also buy products produced in Kibutz's quite regularly. Someone on the other side of the Israel / Palestine debate might take issue with that as well.
In the end, I am a supporter of peace and can't find and contradiction in supporting the craft of olive farmers in a very troubled region. It's not so much about politics as it is about supporting non-commodity food products and productive economic development. It's certainly true that there are many other places to get fine olive oil. I'd just rather support what seems to be admirable economic development project in a place where it's incredibly difficult to maintain a thriving business considering the lack of a stable state, infrastructure, etc. Whose to blame for those conditions is beyond me, and perhaps the context of this conversation.
As for fair-trade in general, it's certainly not a sustainable answer to economic development in troubled countries. However, considering that many of our food products come from countries which have lax or non existent labor standards, I can't see the harm in the first world consumer joining in solidarity with the producers of their most valued commodities (especially when consider sugar, coffee, bananas and cacao).
I've been to several fair-trade coffee cooperatives in both Mexico and Guatemala, and can assure you that what's happening there is overwhelmingly positive. However, each project / cooperative is unique, and they are certainly not without their issues.
In general, I agree with what you are saying about purchasing products that stand on their own merit. In general though, I find that fair-trade / direct trade / alternative trade products to be of quite high quality.