15 Years of Experience: Does the Number in a Role Preamble Actually Do Anything?
Quick one before work. Half this board thinks role preambles are decoration. I do not think that is right and I would rather be argued with properly than downvoted. This sits at the top of nearly everything I run: ``` You are a senior contracts analyst with 15 years of experience reviewing commercial leases. ``` Swap that for "You are a helpful assistant" and the output comes back shorter and flatter. It skips things. I have run that swap maybe forty times across different tasks now and it goes the same way every time. Two things I think are happening. 1. **Attention budget** - the model has a finite amount of attention per pass, and a strong preamble buys you more of it pointed at the right part of the input. I got that off a video about a year ago and everything I have seen since fits it. 2. **Vocabulary pull** - fifteen years drags in the words a fifteen year person uses. Rent commencement, holdover. You do not get those out of "helpful assistant". The bit I genuinely cannot settle is the NUMBER itself. Is 15 better than 10. Is 30 better than 15. I tried 40 once for a laugh and it went weirdly formal and stopped committing to anything, which was funny but not useful. What I want under here is a preamble that earns the years instead of claiming them. Something where you could take the number out and the output would actually get worse. My prompt folder is at 74 files now and about a third of them open with that same line.
Not run against a model yet
0 works · 1 fails
Times copied by users
Problem Instructions
Settle whether the number in a role preamble is doing any work, and produce a version where taking the number out makes the output worse in a way somebody else can see. Forty runs of the swap and I still cannot show anybody what changed.
- •Same task, preamble with the years and preamble without, and the difference is visible to a second person without me pointing at it.
- •The vocabulary claim gets an answer: rent commencement and holdover either turn up more with the years in or they do not.
- •10 against 15 against 30 on the same input, scored before and after, so there is a delta and not a feeling.
- •It works on the contracts analyst line specifically, because that one opens about a third of my 74 files.