Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end? | Manifold

Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end?

25

Ṁ1kṀ4.7k

Dec 31

84%

chance

1H

6H

1D

1W

1M

ALL

PII - personal identification information

Stuff like people's names, numbers and codes that identify stuff (SSN, phone number, passport etc), places, locations, names of orgs, attributes that can be used to identify a person, etc.

GPT-4 outperforms Presidio, Microsoft's custom built tool for PII detection. GPT-4 matches ground truth ~77.4% of the times, while it misses a single PII element ~13% of the time.

Market context

Get

1,000

to start trading!

Sort by:

Assume this includes both false positives and false negatives? What's the denominator?

predictedYES

Just a complete side question, what are the legalities or what are the complicating factors in using a GPT against PII? So, it has to be trained on dummy PII, right? How much dummy PII is needed to train that 85% level you are referring to?

@PatrickDelaney I think microsoft tested against their in house system, which does detect PII on real data

People are also trading

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

Will there by a major breakthrough in LLM continual learning before 2027?

Will the highest-scoring LLM on Dec 31, 2026 show <10% improvement over 2025's best average benchmark performance?

Will LLMs become a ubiquitous part of everyday life by June 2026?

Will there be any major breakthrough in LLM continual learning before 2028?

Will any widely used LLM be pre-trained with abstract synthetic data before 2030?

Will there be any major breakthrough in LLM continual learning before 2029?

Will there be a state-of-the-art LLM that is NOT based on next raw token prediction before 2029?

Will the most interesting AI in 2027 be a LLM?

Will there be any major breakthrough in LLM continual learning before 2030?

Related questions

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

Will there by a major breakthrough in LLM continual learning before 2027?

Will the highest-scoring LLM on Dec 31, 2026 show <10% improvement over 2025's best average benchmark performance?

Will LLMs become a ubiquitous part of everyday life by June 2026?

Will there be any major breakthrough in LLM continual learning before 2028?

Will any widely used LLM be pre-trained with abstract synthetic data before 2030?

Will there be any major breakthrough in LLM continual learning before 2029?

Will there be a state-of-the-art LLM that is NOT based on next raw token prediction before 2029?

Will the most interesting AI in 2027 be a LLM?

Will there be any major breakthrough in LLM continual learning before 2030?

© Manifold Markets, Inc.•Terms•Privacy