From what I've learned, the doves, lambs and other sacrificial animals had to be acceptable for the sacrifices. This could mean without a blemish in their coloring, or had to be first born, or whatever the requirement. If one bought the animal outside the temple, you had no guarantee it was acceptable for the sacrifice. The priests had the final say as to if it was.
But if you bought it in the Temple area, it was prequalified, so to speak. Yet another way the corrupt could "game the system".
I believe the clean and without blemish requirement was for the sin offering. There were many sacrifices for many purposes. An offering made because of increase, whether grain grown or new livestock was to be from the first fruits, meaning among other things before subtracting for other uses (in today's economic system this would be called pretax gross earnings), and of the best quality. An offering of general thanksgiving could to be made from the best the believer had to offer. But, one was never allowed to 'sacrifice' using one's poorest quality goods. God expects (and deserves) our best.
I do not believe that the offering made to enter the temple had to be perfect and without blemish. That would be a lot of perfect animals every week. There may have been periods when the system became so corrupt that the priests required temple goers to buy from them, but I do not think this was an on-going long term practice, but I do not really know.