Steve's Deli in River North is my go-to place, including for deli sandwiches like corned beef and pastrami. There are some things that Steve's does even better than Kaufman's, such as their beef-stuffed cabbage, chicken pot pie, and knishes. Steve's uses a Detroit-deli-style rye bread, which is different from Kaufman's New-York-style rye bread; both are excellent. The New York style features lots and lots of caraway seeds; the Detroit style doesn't, and benefits from a quick re-heating before serving.
If you're driving to Steve's, they validate for free parking in the lot behind the store (the lot for Petco). Also, they sometimes have coupons on their website, and they send out coupons if you sign up for their e-mails through their website.
There are a couple of items I get elsewhere. The bagels at Steve's are nothing special, and New York Bagel and Bialy is as good a place as any for bagels while Kaufman's is closed. Also, Steve's doesn't have the smoked fish selection that Kaufman's does; even before the fire, I've been getting kippured salmon at Fox & Obel (where it's just as good as Kaufman's, and cheaper too).