Why the face drifts
Every generation is a fresh roll of the dice. Ask for “a 24-year-old woman with dark hair” twice and you get two different people, because nothing ties the second image to the first. Text alone cannot carry identity — a description narrow enough to fix a face would have to specify measurements no model reads reliably.
Rule 1 — build the anchor first
Generate 4–6 variants with no reference at all, then pick one or two where the face reads the same from different angles. Those become the anchor: the permanent identity of the character. Everything you make from then on references them. Spend real time here, because every later photo inherits whatever you settle for.
Rule 2 — never send more than two references
Three or more reference photos make the model average them, and the average of three faces is a fourth face. Two is the practical limit: one front, one at an angle. Past that the drift starts again, only slower and harder to notice.
Rule 3 — a fixed angle set, generated once
Front and over-the-shoulder, generated once from the anchor, cover most scenes. Reuse those files rather than regenerating the angle each time — a regenerated angle is a new roll of the dice, and it will not match the previous one exactly.
Rule 4 — keep the identity block identical
Write the identity part of the prompt once, word for word, and change only the scene around it: “Same woman as in the reference image. Keep her exact face: identical facial proportions, same eyes, nose, lips, jawline and eyebrows. Do not idealize, beautify or alter any of her features.” Rewriting that block between generations is the most common cause of a face that slowly becomes someone else.