Determining an appropriate sample size for Factor Analysis is tricky. You can’t easily use a power calculation because Factor Analysis ordinarily does not have a traditional research hypothesis. The page cites various recommendations for sample size (along with references!).
Karen Grace Martin. How Big of a Sample Size do you need for Factor Analysis? The Analysis Factor. Available in html format