I am really lucky. Nearly every week my mother-in-law volunteers to come over for the morning and play with my two smaller boys so that I can go shopping alone. I get so much more done without having to load everyone in and out of the car and keep them happy at the store! Of course on those days that I do need to bring my boys along they are usually pretty good. Occasionally, I remember to do something beforehand to keep them busy while we're there.
I try to make a quick list of things from my list . . . things that I can draw (I am not an artist). The boys love having their own list and it keeps them busy looking for items instead of bugging their brothers.
If I do forget to make a list (which is usually the case) I have them look anyway. We play the "can-you-help-me-find-the-______" game. It keeps them distracted from how long the shopping is taking.
And if you have kiddos that know their letters you could always have them go on an alphabet hunt--either looking things that begin with a certain letter (ie: find something that starts with 'b'--banana, beans, bacon, etc) or you could have them play the alphabet game finding every letter of the alphabet on signs, labels, etc.
These games have kept some of our shopping trips a little more sane! What tricks do you have up your sleeve for shopping trips that get a little hairy?