My Dad lives in Bluffton; you'll pass his place on the way in to HHI on 278. We don't get out to dinner much since my stepmom took ill but I still do lunch in the area. I'll admit I almost never drive onto the island anymore since traffic got so bad, but if you are looking to get out and try something local and definitely non-touristy for lunch some good bbq and southern/soul cooking can be found at Sgt. White's Diner in Beaufort (1908 Boundary St., aka Highway 21 - 843-522-2029). Do call ahead to make sure they are open. It's not in the most picturesque area of Beaufort but the food's good. Once you are in the Beaufort area drive downtown and walk around - I lived in the area about 30-35 years ago and can't believe the transformation. It was a neat little water-town then but now it's really beautiful.
I used to really like hitting Steamers on Lady's Island but my dad tells me it's not the same since it was sold. All manner of seafood sold here but the main draw for us was the steamed oysters. A pitcher of beer, a roll of paper towels, a bucket of steamed oysters for the table with an empty bucket for shells in the hole in the middle of the table. Pass the shucking knife around (of course now we all have our own :p) to shuck the oyster and then, holding it between thumb and knife, dip it into the little cup of drawn butter and then devour. Repeat as necessary.

I'll visit them this Christmas season when I'm down there to see what their current status is, would be a real shame if they've lost it as it was a fave destination.
Objects in mirror appear to be losing.