I was thinking about this after a discussion at work about large language models (LLMs) - the initial scrape of the internet before Chat GPT become publicly usable was probably the last truly high quality scrape of human-made content any model will get. The second Chat GPT went public, the data pool became tainted with people publishing information from it. Future language models will have increasingly large percentages of their data tainted by AI-generated content, skewing the results away from how humans actually write. To get actual human content, they may need to turn to transcriptions of audio recordings or phone calls for training, and even that wouldn’t be quite correct because people write differently than they speak.
I sort of wonder if eventually people will start being influenced in how they choose to write based on seeing this AI content. If teachers use AI-generated texts in school lessons, especially at lower levels, will that effect how kids end up writing and formatting their work? It’s weird to think about the wider implications of how this AI stuff will ultimately impact society.
What’s your predictions? Is there a future where AI can get a clean, human-made scrape? Are we doomed to start writing like AIs?
What I predict is that they’ll try to implement data filters to avoid feedback loops, but there will be an enshittification process for AI too.
What might put the nail into the coffin much quicker isn’t the feedback loop, but trying to monetize the whole thing. I think it’s only a matter of time until OpenAI will try to get money off of it, like putting certain features behind a paywall, especially those that professionals might use.
But OpenAI already has a monetization system now, and last update they reduce the price
This is already happening though and it’s somewhat understandable.
Gpt4 is locked behind a pay wall, and there are a lot of companies that offer twicked versions for specific uses for a price.
Considering the server space, filtering, and training this thing takes, asking for some fee is almost a given.
On the other hand you have the HuggingFace models that try to create an open source space for AI and I really hope that goes well!
I don’t mind them charging a fee. The freemium model is nice and usually sustainable. It lets people pay for extra features but provides the free tier to those that don’t want to.
I kinda wish we collectively started becoming okay with paying for internet-based products and services.
Even something like beehaw. Why not pay a couple of bucks a month for something you get value and joy out of?
I say this as a guy who runs Linux fulltime and uses mostly free and open source software, but who also donates to System76 (my OS’s maintainer) and to a few of the projects I really like.