Rendered at 03:31:24 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
sean_pedersen 3 hours ago [-]
An ill advised undertaking IMO: Any attempt to build AI generated content detection systems is flawed, since the outputs of such a system may be used to train an even better fake data generator. This leads to an equilibrium state of digital uncertainty: nothing in the digital realm can be deemed as real anymore - only as digital.
80fj0348i 15 minutes ago [-]
> the outputs of such a system may be used to train an even better fake data generator.
[citation needed] - my understanding of how entropy works in real photos vs. synthetic images (AI or not) is that this is extremely difficult without severe degradation of image quality.
zeroonetwothree 3 hours ago [-]
How is that a downside exactly? If we can’t tell then we are no worse off.
sean_pedersen 2 hours ago [-]
It will create false trust in their score - giving people a false sense of certainty.
karahime 2 hours ago [-]
Adding to this, the most motivated actors to get around it or find an edge case, e.g. in politics, will use it to say "Look, of course the scene depicted in this compromising image actually happened, the detector said so".
nullsanity 10 minutes ago [-]
[dead]
hex4def6 4 hours ago [-]
At least on gemini generated images vs real ones, it works surprisingly well.
Taking a photo of an AI generated image on my grubby laptop screen with my phone, moire and all, still got detected. So the signal it's using is pretty robust.
Trung0246 2 hours ago [-]
Interesting, it detected NovelAI stuff now. Looks like one can easily defeat it by having pixel-level modifications to make AI images to have characteristics of digital brush strokes and reducing "noise" in output. At least in my case for anime stuff.
alkonaut 4 hours ago [-]
Can’t these detectors be used in the training loop of models?
JacobAsmuth 4 hours ago [-]
People have tried it, it doesn't work.
bobajeff 3 hours ago [-]
Which people? I'm no AI researcher but I believe basically all GANs employ a discriminator which basically to me it sounds like this is.
dwohnitmok 2 hours ago [-]
I don't know about Pangram's viability as a discriminator but I can tell you as someone who has recreationally trained GANs that they're really really finicky to train and training fails in all sorts of inscrutable ways. It wouldn't surprise me if even though in theory you could do a GAN-style form of training with Pangram, in practice it might actually be very difficult.
XTXinverseXTY 3 hours ago [-]
In theory yes, but likely not economical for the slop arbitrageur
It is not a lucrative business and they are not clever people to begin with
my002 6 hours ago [-]
People often shit on AI detectors but Pangram's text AI detection has been very good, IME. Not perfect, but good enough to be useful in most circumstances.
jszymborski 3 hours ago [-]
My trouble with Pangram is that the calibration is totally off. It can tell you if there is a drop or an ocean of AI in the text you submit. The "X% of AI" message doesn't really mean much. You can test this yourself: Slowly add generated text to a human write paragraph and you'll see it hit "100% AI" quite quickly.
Matticus_Rex 3 hours ago [-]
It analyzes in segments, and the percentage is of the segments, not of the words.
jrhizor 3 hours ago [-]
Good?
Alpha3031 2 hours ago [-]
I don't really know why you would expect a "X% chance of containing AI" number to rise slowly or smoothly alongside the percentage of text that has a 100% chance of containing AI.
xnx 5 hours ago [-]
I'm very skeptical of AI detection and watermarking for text (especially short passages), but AI detection for images and videos seems much more plausible (for now at least).
I would love if Meta licensed this tech to use for everything on Facebook, Instagram, and Whatsapp. Including ads!
JacobAsmuth 4 hours ago [-]
You should look at the data. Look at the actual measurements on the accuracy of Pangram's V4 detection model. Try to fool it. It's not easy at all.
zeroonetwothree 3 hours ago [-]
It’s often inaccurate with short text. For example it has flagged text I wrote by hand with no AI input or editing.
But I agree it’s good for longer text of 1000+ words.
mediaman 4 hours ago [-]
Why are you so skeptical of text detection?
Do you believe it generally doesn't work or do you believe that if you work at it you can evade them and therefore there's no point to it?
74948494i4 4 hours ago [-]
What’s the use case for these things?
tantalor 4 hours ago [-]
A few likely use cases,
1. Social media moderation. You want to limit the proliferation of this stuff because it annoys users.
2. Trades where authenticity matters, like journalism, law, education. You have to weed out the crap and people will try to deceive you.
80fj0348i 14 minutes ago [-]
eCommerce - you may want to reduce fraudulent listings, e.g. used goods, housing, collectibles/auctions...
poszlem 4 hours ago [-]
What's the use case for knowing when you are looking at a fake AI image and not at a real photo? Is that seriously your question?
leothetechguy 3 hours ago [-]
Awesome research like this restores my faith in humanity.
[citation needed] - my understanding of how entropy works in real photos vs. synthetic images (AI or not) is that this is extremely difficult without severe degradation of image quality.
Taking a photo of an AI generated image on my grubby laptop screen with my phone, moire and all, still got detected. So the signal it's using is pretty robust.
It is not a lucrative business and they are not clever people to begin with
I would love if Meta licensed this tech to use for everything on Facebook, Instagram, and Whatsapp. Including ads!
But I agree it’s good for longer text of 1000+ words.
Do you believe it generally doesn't work or do you believe that if you work at it you can evade them and therefore there's no point to it?
1. Social media moderation. You want to limit the proliferation of this stuff because it annoys users.
2. Trades where authenticity matters, like journalism, law, education. You have to weed out the crap and people will try to deceive you.