If an opponent takes a card from the discard pile, you can remember which one, and use that information to discard more wisely during the rest of the hand.
I've developed a set of "Cheat Sheets" that I used to begin tracking what cards have gone where in a hand. These sheets are only for learning, you'll never use them in a live game. I use them when playing online, or against a computer just to get my brain accustomed to thinking "in a Gin mindset".
There is an image of the Cheat Sheet to the right. I use it to keep track of cards that have either been picked up and placed in my opponents hand, or tossed into the discard pile.
By keeping notes at first, I became accustomed to keeping track of the cards that an opponent used. And, I could often spot patterns where an opponent would take cards from the discard pile, only to return them later. Using this information I could see how he would try to develop his hands.
The sheets were just a guide to help me develop my "card memorizing" skills, and I can recommend them for that purpose only. I have two different versions that you can choose from. There is a printer-friendly web page with four sheets per page. And, if you'd prefer to alter the sheet at all, I have a link to the original spreadsheet version of the Gin Cheat Sheet.
After using the sheets for a while, it won't take long until pencil hand wears out and your memory starts to take over. But, if you're unaccustomed to tracking cards (say, you've been playing a lot of poker before taking up gin), filling in these sheets will help focus your mind on the gameplay at hand, and help reveal patterns that might otherwise go unnoticed.

