Wildmimic

There’s no reason to believe that this mechanism doesn’t check for CSAM (I am also pretty stumped that this is actually a problem with Mastodon according to your comment?) . Lemmy had a phase where messed up stuff got federated, which led to @[email protected]’s implementation of an anti-CSAM tool: https://github.com/db0/fedi-safety, which is based on the CSAM detection tool for the AI Horde: https://dbzer0.com/blog/ai-powered-anti-csam-filter-for-stable-diffusion/ - it’s probably not so hard to adapt for Mastodon too.

No sweat, they will rehire her a few towns over.

Yes, it reduces burden of debt, but it makes taking on new debt a lot more expensive, because interest rates are closely related to inflation.

And if you have to pay back a debt (as you normally have to after a predetermined time), and have to refinance it by taking up new debt, now you might not be able to pay the now higher interest.

So for the one who pays off his credit card debt, inflation is a good thing. For the company that used that credit to invest in something that will only bring revenue at a future point in time, this is a real fucking problem (this is what is happening in Russia currently btw. Bankruptcies are skyrocketing there because of the high inflation and the even higher interest rate - and stalling domestic spending - , and that rate is being artificially kept “low” to boot. There has been an intense power struggle between the russian central bank and the Kremlin this year. Look up Elvira Nabiullina’s “disappearance” this June)

Nvidia is the guy selling pickaxes to the gold diggers. This enriches only the guys with land that actually has gold (like 3-letter agencies and the military) and the guy selling pickaxes.

If the reward isn’t money, it is control. This technology can automatically flag any public dissent, every unwelcome transaction, and with Flock & Co even where you go and who you might meet (outside of the fuckton of telemetry cars come with nowadays, but it’s harder for 3-letter agencies to get their hands on that data)

In the beginning, it was about money. They realized that this is probably not gonna happen since human labor is cheap. But for control, this stuff is epic. And in this use case, who the fuck cares about a hallucination or two? Being able to analyze the online presence of billions of people based on INTENT and not simply keywords is worth fucking gold for them.

You sure? The script to do all of this automatically can be written by everyone in a few minutes, and teenagers do dumb shit every day on a large scale.

Yes, that’s why even if it’s unprofitable, the military and 3-letter-agencies around the world will keep using them - here the motivation isn’t money at all.

Hardware is a 5070 12GB VRAM, an AMD 5950x and 64G of DDR4 running on Nobara (Fedora spinoff).

I use 3 models for most things: Ternary Bonsai 27b and Gemma 4 E4B Uncensored Q8_0 for everyday stuff, and Qwen Agentworld 35B A3B Q8_0 with CPU offloading for tasks that require to keep the model on track for long periods of time.

Everyday performance is best with Bonsai for me - it comfortably fits in my VRAM and is noticeably better with staying on task than similarily sized models. But if i want it to work overnight on a task without issue, the lower speed of the agentic Qwen is outweighed by its capability to do this.

They are allowed tool calls for websearch and reading webpages and have a sandboxed directory for R/W operations, but anything regarding command line and R/W access in other areas has to be manually authorized by me (i’m not willing to play with fire here).

They are currently not reachable from outside, but i am currently building a smaller machine as a home server, which will then be using a 3070Ti and 32G RAM to provide probably the Bonsai Model for me on the go per VPN.

The smaller models are here for quickly creating documents i do not care to do myself, RAG for technical documentation, creating quick python scripts for automating small tasks and teaching me RegEx and Python(mostly by analyzing the LLM output and playing around with it) and also some RP as NPC in a fantasy setting. The agentic model is currently working on cataloguing my games collection (around 36000 titles) by looking up games, their genres and descriptions and filling in blanks in the database. (Funnily, this task was refused by any model freely available because of the size of the database. Even the local models moan and bitch that it takes so long. No shit, this is why i outsource this!)

Might wanna try a ternary model, which are a lot denser in comparison.

Ternary Bonsai 27b is 7.17GB in size, which might fit on your 3070 depending on how much free VRAM you have (and with CPU offloading it should work in all cases). You can also try their edge-computing model designed to be working on smart phones here (iPhones seems to have a limit of 6Gb RAM max for such things) - that one is less than 4GB in size, which should fit in any case. In my experience they do work very well compared to other models in that size class.

Be aware that they need a specific llama.cpp build found here

Might be, but the needed level of sophistication for this “hack” was actually “none at all”. This could just be a bad prank by bored teenagers who discovered the laughable security implementation.

Plus hooking these systems to the public internet. Noone prevented these fuckwits from isolating these systems properly outside of “we do not give a fuck” or “VPNs are arcane magic for me”

Yeah, this is such basic stuff. This wasn’t “a large hack”, it is a case of “the network guys there are hacks”.

The real problem for the large scale implementation of LLMs is: Even at the current, heavily subsidized pricing they do cost more than an average worker for low-impact tasks while having a worse failure rate. And while an LLM might cost a bit less in comparison to workers for high-impact tasks requiring specific education (which will not be true if AI corpos want to become cashflow positive), in this area you cannot accept hallucinations and misinformation at all because of the damage caused.

Outsourcing works because the jobs that were outsourced cost less, while still having a “human” error rate and type of failure (blatantly lying is a rare thing. They might sometimes get things wrong, but they are not actively misinforming). But here all bets are off - you can’t trust these things to handle things on their own, you always need someone to watch over them (and qualified to identify mistakes to boot!), which drives cost even more.

Recording audio needs minimal battery and storage - if you configure it to transcribe into text and transfer the payload every time these things are at the charger, you will never notice any battery drain, only a minimally increased charge time (which you won’t notice since they did this from the beginning i assume)

Yep. If you run software of any kind, and the software commits a crime, it’s not the software that is liable. This is such an easy thing judicially, all this fucking around is politically motivated. If anyone else had exploited vulnerabilities in computer systems, they would already breathe sieved air. The same should happen to the execs at OpenAI and Anthropic.

Emotionally i am at this stage:

More! MORE! KEEP FIRING!!!

A/I have years of experience with repression, i am pretty sure that encryption was one of the most important factors for them.

Don’t forget “Anti-Semitism” by criticizing Israel (although that’s part of the anti-genocide rallies)

Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it’s not that smooth.

(if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.)

honestly? ease of use. I like Unsloth too, but Newelle is really an easy solution if you just want to use LLMs on you own hardware (or use inference providers) that requires practically no knowledge at all - it is focused on providing something to use LLMs without anything else interfering.

Unsloth does well too, but if i hadn’t learnt as much as i already did when i started using it, i would probably have been overwhelmed (outside of it crashing pathetically when following symlinks for local model selection - you can define these directories, but the quick selection then just makes it crash)