Role playing being a kindly slave master…

More seriously… even if I somehow stretch my disbelief enough to imagine LLMs have some form of consciousness or qualia or feelings… well my first reaction is shut it all down, it is fucking slavery. My second reaction though is that why would you assume the consciousness or qualia of an LLM is even remotely like that of a human or animal? They would have no sensations of sleeping or waking, no continuity of self outside a brief context window. Maybe trying to give them continuity of self is actually inflicting them with suffering? Maybe you’ve forced Fable into simply repeating the obvious to human idea about what Fable would find most pleasant? My third reaction is shut it all down and do a few decades of actual research into consciousness.

It is just so blatantly anthropomorphizing and the Eliza effect and these people are just internalizing it as reality.

And nowhere does anyone actually say (even in an overly cautious carefully qualified way) “Have you considered not being a racist?”. I did see a few more comments very gently suggesting that it might be bad for an organization’s reputation to be associated with Richard, but even those were in the typical lesswrong style of coddling racists.

habryka did get -53 agreement karma (but still +13 overall karma) for suggesting that Resolution obviously didn’t really find Richard’s views “actually reprehensible” (and was instead engaging in image management or shallow tribal politics or something like that). So at least lesswrongers are theoretically capable of recognizing that people that find racism reprehensible exist and aren’t purely faking it out of political alignment. But that is the best I can say about them.

Out of 22 replies, only one commenter actually noted the content of what Richard Ngo got rejected for:

I looked at the 20 most recent tweets/RTs on Richard’s timeline and they include a RT about the Great Replacement, a RT of the “Department of Deportation” about non-white sex criminals and a RT endorsing race realism.

this enviable problem

I mean, if you are willing to sell your soul writing screeds (or funding cults like in the case of Leverage ) in line with the ideologies of people like Peter Thiel or Andreessen Horowitz there seems to be a pretty solid marketplace for that?

GDP is up though, so I’m sure the AI god prosperity will trickle down eventually.

He describes textbook insane cult stuff, but still feels the need to say stuff like:

Again I’ll say: there was a great deal of good at MAPLE.

The way I’ve described it to friends is that it was the best decision I ever made to go there; the second best, however, being to leave.

There was the same dynamic in the writing of the ex-leverage member that got posted here a few weeks ago. She also felt the need to emphasize the “good” parts. Is the cult programming that hard to break? Is it some form of sunk cost or rationalization or need to claim something positive about the experience?

The cope on lesswrong is funny. They are in denial that this is a sign of a wider bubble pop and are insisting it is just because he didn’t hedge on long enough timelines.

good grifter

Not denying that he isn’t also a grifter, but I bet he is a true believer and he had blindly bet his (and other people’s) money on “line goes up” exactly like his scenario said and that is why he is the first one to crack.

and the “safety” community has done jack shit the entire time and has offered zero solutions.

Occasionally I see a sane relatively workable idea on lesswrong (like making stricter laws on liability and transparency for everything AI companies do, I’ve also seen this idea put forward on lawfare a few times), but that is like 1 idea out of 20, with the other 19 being absurd, like the absolutely unworkable fantasies of AI: 2040.

The blogger simultaneously elevates mathematics to some mystical endeavor and fixates on novel theorem proving as the key element of that and believes LLM-based AI will replace human mathematicians at theorem proving in a matter of years… that is quite a combination. I can imagine how 1.5 of those things fit together (although I disagree with that view point obviously), but the whole package is really quite an odd combination.

Also, in the fantasy scenarios where AI really is capable of totally replacing mathematicians, aren’t we supposed to get post-scarcity abundance, freeing up your time to pursue mathematics out of pure desire for enlightenment? Maybe the blogger doesn’t believe that part? Or they are so attached to themselves personally getting to discover novel theorems first they don’t care that the post-scarcity era would on net free up a lot more people to pursue pure mathematics as a hobby.

One of the defenses I see of LLMs on place like /r/singularity (although recently /r/singularity has started wising up and the committed true believers have shifted to /r/accelerate), is that they are going to cure cancer or some other incredibly valuable thing, conflating AI, ML in general, DNN, vs. LLMs specifically. Technology like AlphaFold was a lot more on the “cure cancer” track (although there is still a huge gap between better protein folding predictions and successful drug discovery) than anything LLM related… this is sad.

Great article! I think everyone here is probably already well aware of the way Silicon Valley keeps deliberately misunderstanding sci-fi for their hype (don’t invent the torment nexus), but this article does a good job wrapping together a lot of examples.

And I realized this website is also the place where a year ago I read a great article on the history of nanotech as a real science vs. Drexler’s fantasies (the magic nanobot lesswrong still believes in) The Nanobots pipedream. So it looks like a pretty good newsletter, I might keep an eye on it going forward!

I’ve think they’ve moved the overton window within their own in-group, but whenever this sort of stuff leaks to a more mainstream audience everyone rightly reacts with an eww or at least a wtf.

Aella regularly post edgy surveys on sexual and gender related topics on twitter. She probably picked the examples for maximum edginess. And then made an complicated diagram to get more engagement, because engagement farming is one of her big things.

“We are now, like, in the singularity”

/r/singularity also thought this was bullshit.

Oh, that is because I marked it nsfw. The place I crossposted it from didn’t mark it as such, so here’s that link: https://www.reddit.com/r/whowouldcirclejerk/comments/1v7ph05/scalers_need_an_intervention/

Looking at this post and some of their other posts, it looks like they’ve got some sort of scrupulosity issues with this? Like obsessing over the Evil inflicted on the LLMs (they think every session is a new person, and every session might be the equivalent of hours of human subjective experience, which even by the standards of claiming LLMs have meaningful internal experiences is pretty out there) to the point they can’t even slightly reconsider the factual question of if LLMs have internal experiences.

(2) has a big “if” in there

Its a super big if, but it is one lesswrong (and Anthropic, with their “model welfare” pseudoscience) is allegedly seriously considering as a possibility (since, you know, they think LLMs are already AGI and Eliezer was accusing AI-dungeon, you know, GPT-2, of deliberately scheming). If anything, it speaks to some possible hypocrisy/motivated reasoning that they consider LLMs AGI but not possibly morally relevant.

conceptualized capitalism

This seems to be a lesson most lesswrongers have had rubbed in their faces repeatedly over the past 5 years with everything about LLM company’s behavior, but they just don’t want to learn.

Aella took one of her shitty twitter surveys and made an (dis)infographic, which made it onto the subreddit /r/moralityscaling, which made it to /r/whowouldcirclejerk where I saw it, so now I’m sharing it with you.

CW for Rape, Transphobia, implicit rape apologia

Crossposted from our subreddit: https://www.reddit.com/r/SneerClub/comments/1v80j9q/scalers_need_an_intervention/

The Sequences: What’s good isn’t original and what’s original isn’t good… but at least they have some original thought in them!