“But I thought chatbots were great at security! They hacked huggingface and also fable will fix all security bugs in firefox”.

I think even if development at an instant is some % faster to start with, it writes the same functionality in more lines of code which will rapidly lead to a cumulative slowdown.

Imagine you are traveling the oregon trail in a wagon and a devil offers you an extra horse but all of its manure has to go into the wagon. Initial speed up followed by a cumulative slowdown. If you keep accepting devil’s deals as your wagon keeps slowing down to a crawl, the progression is governed by a horseshit version of Tsiolkovsky’s equation.

Well, the big issue is that there’s only this much demand. How many web browsers is there genuine end user demand for, worldwide? Probably just 1. Several are made thanks to other demands by large corporations. How much value does a web browser provide, to billions of people who use it? What ever the number may be, it’s orders of magnitude more than it costs to develop.

Why is software observably hard for companies? When 90% of people are working on projects that are going nowhere, that creates a situation where competence is quite rare. Because in most projects it is objectively unnecessary (would make no difference). Where majority of software engineers never seen a successful project all the way from the start to the finish line.

Then there’s startups, these are (predominantly) mere vehicles for moving money from the coffers of large publicly traded corporations into pockets of VCs (who sell startups to said corporations). Whole world of pretend software engineering.

Then there’s large corporations, where CEOs want to get paid more and to be able to get paid more they need to add more layers to the hierarchy underneath them, which they need to fill somehow, and they can’t fill on merit because they just don’t have enough stuff happening that they could measure merit in.

edit: basically the management always wants to do the equivalent of having hired a dozen heart surgeons to work on one heart and a “heart surgery architect” as well, the more layers the better because the more layers the higher is the pay their layer can get.

As a software engineer, all of the free portion of the article is 100% correct about software engineering dysfunction.

There’s another reason why it is that way, though. A software engineer who is competent and productive and working under competent leadership making an useful product, would make a very large multiple of their salary in profits.

That permits all sorts of utter dysfunction while still breaking even on the average. You can have incompetent leadership in 90% of projects, you can saddle your developers with any kind of bad practices, you can impose all sorts of productivity damaging nonsense and yet still break even.

The dysfunction can be presented as relentless innovation and pursuit of new, uncertain opportunities, as a shrewd strategy.

If you fall below break-even, the consequences will not be visible for many years. And up until it becomes impossible to recover, on the technical level it is extremely cheap to recover: it’s easier to do nothing than to convert your OS’s start menu to React. Tech companies don’t fail because of tech.

This also relates to using AI to write code. You already can have a human write your code and pay them $1 for $10 you would make from it - or $0 you would make if you fucked up. You want it to become $0.5 with AI coding tools? Then you’re multiplying your probability of success by what’s left after the probability that you get some unmaintainable slop . It’s simply not worth it.

Into a black hole.

Who needs cures when you can instead simply replace doctors with chatbots?

Yeah I have to admit I don’t have the energy to gaze into that particular idiotic abyss. How do they even write that kind of fiction, in a world where AI companies routinely use critihype to obtain more investments? The least unhinged CEO: “we caught our AI trying to start a nuclear war, it’s almost skynet, please give us another trillion dollars”, the least unhinged investors: “shut up and take our money”.

I thought it was 2027. What, the timelines slip or something?

Yeah… I think the core point is, LLM’s shit is generated artifacts, which need to be kept separate from sources. That you can’t actually re-generate the same artifacts, is just a dingleberry on top of the shit pile.

Perfect for when your workplace will fire you if you dont use enough tokens, I’m sure.

I think there’s something to the compilers analogy, in the sense that if you used a C compiler through a chat interface with one liners to insert snippets into assembly without keeping the source code in source files as such, it would probably be slower than just writing in assembly yourself.

That LLMs have a number of extremely undesirable properties as “compilers” (from nondeterminism to plagiarism) does not make the dumb ass chat interface any less fundamentally shit. It just makes them shit in other ways too.

I just love how they are adding the absolutely fucking most stupid exploits ever to everything, and then trying to whitewash it with all that bs about Fable or whatever the fuck.

There could be but it would be difficult to implement - something a bit like putting together a shredded paper, heh.

It’s a perfect example of how “using LLMs for test coverage” can also be harmful. He expected the tests to to prevent introduction of said regressions, probably based on a combination of the quantity of tests and their style (they look like what decent human written tests look like). But the tests are AI slop, and so they give a lot less value per line of code than he expects, hence a significant regression.

It is literally useful to call these tests AI slop, and the problem is in part caused by not calling them AI slop, and having consequent inflated expectations.

A guy I know because he invented a mechanism and I built an alternative implementation of the same behavior out of Lego and put on youtube, cited my thing in his doctoral thesis and will be defending on Monday, and I can watch that online. Can’t wait.

And 100% of them are just trying to suck up to the rich the hardest to get some cash thrown their way for posting. Their whole community has been built 100% around that from day 1.

To be entirely honest I don’t even like the arguments against EDT.

Smoking lesion is hilarious. So theres a lesion that is making people smoke. It is also giving them cancer in some unrelated way which we don’t know, trust me bro. Please bro don’t leave this decision to the lesion, you gotta decide to smoke, it would be irrational to decide not to smoke if the lesion’s gonna make you smoke. Correlation is not causation, gotta smoke, bro.

Obviously in that dumb ass hypothetical, the conditional probability is conditional on the decision, not on the lesion, and the smoking in cancer cases is conditional on the lesion, not on the decision. If those two were indistinguishable then the right decision would be not to smoke. And more generally, adopting causal models without statistical data to back them up is called “being gullible”.

The tobacco companies actually did manufacture the data, too, thats where “type-A personality” comes from.

Tbh whenever I try to read anything on decision theory (even written by people other than rationalists), I end up wondering how do they think a redundant autopilot (with majority vote) would ever work. In an airplane, that is.

Considering just the physical consequences of a decision doesn’t work (unless theres a fault, consequences don’t make it through the voting electronics, so the alternative decisions made for the alternative that there is no fault, never make it through).

Each one simulating the two or more other autopilots is scifi-brained idiocy. Requiring that autopilots are exact copies is stupid (what if we had two different teams write different implementations, I think Airbus actually sort if did that).

Nothing is going to be simulating anything, and to make matters even worse for philosophers amateur and academic alike, the whole reason for redundancy is that sometimes there is a glitch that makes them not compute the same values, so any attempt to be clever with “ha, we just treat copies as one thing” doesn’t cut it either.

Even to the extent that they are “prompting it wrong” it’s still on the AI companies for calling this shit “AI”. LLMs fundamentally do not even attempt to do cognitive work (the way a chess engine does by iterating over possible moves).

Also, LLM tools do not exist. All you can get is a sales demo for the company stock (the actual product being sold), built to impress how close to AGI the company is. You have to creatively misuse these things to get any value out of them.

But just hear me out: if you delete your old emails, you won’t be roped into paying for extra space, and Microsoft or Google will have a little less money to buy water with!

Switch to Linux and avoid using any Microsoft products to conserve even more water.