What are the first indications that Charlie's intelligence might be temporary?

Charlie, and the reader as well, both begin to have an inkling that his intelligence may not be permanent as he listens to Dr. Nemur's presentation in "Progress Report 13." Charlie even realizes that Nemur did not take into account his rapid rise in intelligence, and that now, Charlie may even regress into a lower IQ than before the experiment.