It's called ignoring the memories and only pulling the actual knowledge. Like all the spells they developed and designed. The important stuff.

[ Do you think he'd count people as important? Then again... ]

Only the strong minds of the magicians that chose to be a part of it, like the old man, had enough will to call back on, anyway. When you have millions, they blend back into a single source.