Jack & Jill: Salary Benchmarks – Listing vs. Survey Data

A salary number is only as good as the data behind it, and the difference matters. Listing based benchmarks show what employers offer today. Survey based benchmarks show what people said they earned, sometimes years ago. Here is how to tell which one you are looking at.

Salary numbers fall apart under one question

Salary numbers look authoritative in a screenshot. Ask where the data came from. The two main methods, real job listings and self reported surveys, can produce noticeably different numbers for the same role.

Why the data source behind a salary number matters

A benchmark is only as good as the evidence behind it, and the two main methods measure different things.

A benchmark built from job listings reflects what companies are advertising for a role right now. It tends to move with the market, so a rise in pay for a role shows up in listings sooner than in a survey.

A benchmark built from self reported surveys reflects what a sample of people said they earn when they answered a form. Surveys are useful for long term trends across whole industries. They are slower. Respondents round, guess, or report total compensation in different ways, and the data is only as current as the last survey cycle.

Neither method is wrong. But they answer different questions, and if you are about to negotiate, you want the one that describes the offer you are about to receive.

Listings, surveys and what to remember

Survey based benchmarks describe what people said they earned, often with a lag of months or years.
Listing based benchmarks describe what employers are advertising right now, so they track the market faster.
Not every tool labels its method clearly, so check before you rely on a number.
You can verify the source in a few minutes by reading a tool's methodology page and testing a number you already know.
Whatever the source, treat any benchmark as a starting point for a conversation, and practice the conversation itself.

How to check what a salary tool is using

You do not need insider knowledge to work this out. You need about ten minutes and a little skepticism.

Find the methodology page. Look for a link called methodology, data sources, or how we calculate pay. If a tool has no methodology page, treat its numbers as a rough guide and confirm them elsewhere.
Read what the words actually say. "Crowdsourced" and "community reported" usually mean self reported. "Posted salaries", "job listings", or "employer reported offers" point to listing or employer data. Watch for vague phrasing like "market data" that names no source at all.
Check the sample size and the date. A benchmark built from 40 responses for your role in your city is a hint rather than a benchmark. Ask when the data was collected. A survey from two years ago may describe a market that has moved on.
Test it against a number you know. Pick a role where you know the real pay, either your own or a friend's. If the benchmark is far off, adjust how much you trust it for the roles you care about.
Look at how it handles your specific case. Broad averages hide the things that move pay most, which are your role, your level, your industry, and your location. The more a tool lets you narrow those, the more useful the number becomes.

What listing based data does well, and where it falls short

Listing based benchmarks have a clear strength. They are hard evidence of what an employer put in writing. A posted salary range is a commitment, made recently, by the exact kind of company you might work for.

The limits are worth knowing too. Not every employer posts a range. Some posted ranges are wide, or aimed at the lowest level of a role. And a listing tells you the opening offer, while survey data can capture bonuses and raises that came later. Using both gives a fuller picture, with listing data for what employers offer now and survey data for context.

What to watch for before you trust any benchmark

A few habits will keep you honest with any salary tool, whatever its method.

Prefer tools that name their data source, sample size, and collection date in plain language.
Be careful with numbers that cannot be narrowed to your role, level, and location.
Remember that total compensation includes more than base salary, and check that the benchmark is comparing like with like.
Treat a single number as the middle of a conversation. Ranges, and the reasoning behind them, are more useful than one figure.
If a tool will not tell you where its data comes from, use it for orientation only and confirm the number elsewhere before you negotiate.

To see the difference, compare a vague label with a clear one. A label that says "market data" tells you nothing about what went in. A line that says "salary surveys, public job postings and compensation databases, normalized across role, level and location" tells you the sources and how they were adjusted.

Jack & Jill combines data from salary surveys, public job postings and compensation databases, then uses AI to normalize across role titles, levels and locations, the kind of method line this checklist asks for. Jack offers salary benchmarking on that basis and salary negotiation practice so you can rehearse the conversation. He is free for candidates.

Questions about salary data sources

How can I tell if a salary site uses survey data or job listing data?

Read its methodology page and look for the exact words. "Community reported" or "crowdsourced" points to self reported survey data. "Posted salaries" or "job listings" points to listing data. If the site names no source, treat its numbers as a rough guide and confirm them somewhere that does.

Which kind of benchmark should I use in a negotiation?

Listing based data often works well as your anchor, because it shows what employers are offering right now for the role you are discussing. Survey data adds useful context on bonuses and long term trends. Bring the current number to the table and let the survey data support it.

Why do different salary tools give me such different numbers?

Different sources, different samples, different dates. One tool may average self reported pay from a global survey while another reads posted ranges in your city. When two numbers disagree, check which one describes your role, your level, and your location most closely, and weight it accordingly.

How recent does salary data need to be?

Ideally within the last year, and sooner in a fast moving market. Pay for the same role can move noticeably between hiring cycles, so a benchmark that cannot tell you its collection date is hard to trust for a negotiation happening now.

Find out the source before you negotiate

The tool matters less than the habit. Before any salary conversation, find out what the data behind your number is, how fresh it is, and how closely it matches your actual role. Then practice saying it out loud once before the conversation.