r/LargeLanguageModels • u/Inside-Barracuda6658 • 4d ago
LLM drift detection
hey everyone so i have a final year project this year im working on,
quick overview: our topic is persona drift detection in LLM-based character agents.
so we are developing the persona of a movie character based on the script of the movie then we implement a baseline LLM agent that ingests the persona and starts acting like it, then we should build experiments and conversations with different scenarios to test the persona and see where it drift
now what i need some suggestions with is how will the drift detection work.
and please please dont tell me do your research, im doing that, im just asking people too for advice.
what are some of the things i can think of while building the drift detector.
ill paste the description of the detector specifically from the project proposal so you have a better idea:
(Design a Drift Detector that computes weighted deviations from the baseline persona in multiple dimensions (traits, emotions, goals, relationships) and classifies each turn into one of six categories: stable, contextually‑justified, legitimate development, temporary anomaly, persona drift, or severe failure. )
also if you know any github repos that could help please suggest them (even tho i think i looked through them all)
1
1
u/cosmiccedar871 2d ago
Try to start with clear persona contradictions, then check the surrounding turns to see if the change makes sense in context.
3
u/Senior-Assumption864 4d ago
are you planning to compare each turn against the baseline directly or track how traits shift across a longer window?