www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [Not really OT] Crowdstrike Analysis: It was a NULL pointer from the

reply mw <mw gmail.com> writes:
https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19

Crowdstrike Analysis:

It was a NULL pointer from the memory unsafe C++ language.

Since I am a professional C++ programmer, let me decode this 
stack trace dump for you.

....
Jul 19
next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Jul 19, 2024 at 10:49:23PM +0000, mw via Digitalmars-d wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19
 
 Crowdstrike Analysis:
 
 It was a NULL pointer from the memory unsafe C++ language.
 
 Since I am a professional C++ programmer, let me decode this stack
 trace dump for you.
[...] It's 2024, and a NULL pointer brought down half the world's servers. Just gives you *so* much confidence in technology. :-D // In other news, this is yet another nail in the coffin of memory-unsafe languages. We're slowly, but surely inching towards the day when the likes of C and C++ will finally be relegated to the dustbin of history... T -- If it's green, it's biology, If it stinks, it's chemistry, If it has numbers it's math, If it doesn't work, it's technology.
Jul 19
next sibling parent Hoet <tit 4tat.com> writes:
On Friday, 19 July 2024 at 23:33:44 UTC, H. S. Teoh wrote:
 On Fri, Jul 19, 2024 at 10:49:23PM +0000, mw via Digitalmars-d 
 wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19
 
 Crowdstrike Analysis:
 
 It was a NULL pointer from the memory unsafe C++ language.
 
 Since I am a professional C++ programmer, let me decode this 
 stack trace dump for you.
[...] It's 2024, and a NULL pointer brought down half the world's servers. Just gives you *so* much confidence in technology. :-D // In other news, this is yet another nail in the coffin of memory-unsafe languages. We're slowly, but surely inching towards the day when the likes of C and C++ will finally be relegated to the dustbin of history... T
C is going to be around long after both you and I are gone. Languages can affect how thew ratio f new code is written and in which fields but by the very same token, the entire planet works quite so well on a day to day basis , powered by precisely C.. the fact Windows is used however, is a joke, and has been since the last millenium but ah well. THAT one I believe can and will change before you and I are gone, however.,
Jul 19
prev sibling next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 20/07/2024 11:33 AM, H. S. Teoh wrote:
 On Fri, Jul 19, 2024 at 10:49:23PM +0000, mw via Digitalmars-d wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19

 Crowdstrike Analysis:

 It was a NULL pointer from the memory unsafe C++ language.

 Since I am a professional C++ programmer, let me decode this stack
 trace dump for you.
[...] It's 2024, and a NULL pointer brought down half the world's servers. Just gives you *so* much confidence in technology. :-D // In other news, this is yet another nail in the coffin of memory-unsafe languages. We're slowly, but surely inching towards the day when the likes of C and C++ will finally be relegated to the dustbin of history... T
It is not if, but when will D be banned from being used in critical or long standing applications. I really need to get this across to everyone, that its going to happen a lot sooner than you may be thinking. We either get the DFA I'm building, something like it or we're toast in commercial usage.
Jul 19
next sibling parent reply claptrap <clap trap.com> writes:
On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) Andrew 
Cattermole wrote:

 It is not if, but when will D be banned from being used in 
 critical or long standing applications.
**Shrug** Banned from a party it's not even invited too in the first place.
Jul 20
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 20/07/2024 9:03 PM, claptrap wrote:
 On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:

 It is not if, but when will D be banned from being used in critical or 
 long standing applications.
**Shrug** Banned from a party it's not even invited too in the first place.
Oh that's funny. We're not only invited, one of our biggest users of D does exactly this. "The notable exception is the Weka Virtual File System (VFS) kernel driver, which provides the POSIX filesystem interface to applications. Using the kernel driver provides significantly higher performance than what can be achieved using a FUSE user-space driver, and it allows applications that require full POSIX compatibility to run on a shared storage system." https://www.weka.io/wp-content/uploads/files/2017/12/Architectural_WhitePaper-W02R6WP201812-1.pdf
Jul 20
parent claptrap <clap trap.com> writes:
On Saturday, 20 July 2024 at 09:06:35 UTC, Richard (Rikki) Andrew 
Cattermole wrote:
 On 20/07/2024 9:03 PM, claptrap wrote:
 On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:

 It is not if, but when will D be banned from being used in 
 critical or long standing applications.
**Shrug** Banned from a party it's not even invited too in the first place.
Oh that's funny. We're not only invited, one of our biggest users of D does exactly this.
https://6sense.com/tech/data-science-and-machine-learning/weka-market-share So Ds biggest example of being "at the party" has 0.03% market share. My mate Dave was an extra in Sharknado, he watches the oscars at home with a 6 pack of cheap bear. Thats the party D is invited to. ;)
Jul 20
prev sibling next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) Andrew 
Cattermole wrote:
 We either get the DFA I'm building, something like it or we're 
 toast in commercial usage.
What is DFA?
Jul 20
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 21/07/2024 12:57 AM, ryuukk_ wrote:
 On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 We either get the DFA I'm building, something like it or we're toast 
 in commercial usage.
What is DFA?
Dataflow analysis.
Jul 20
prev sibling parent reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Saturday, 20 July 2024 at 12:57:10 UTC, ryuukk_ wrote:
 On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:
 We either get the DFA I'm building, something like it or we're 
 toast in commercial usage.
What is DFA?
I always read “deterministic finite automaton.” (I hate acronyms on the forums.)
Jul 23
next sibling parent matheus <matheus gmail.com> writes:
On Tuesday, 23 July 2024 at 11:53:24 UTC, Quirin Schroll wrote:
 On Saturday, 20 July 2024 at 12:57:10 UTC, ryuukk_ wrote:
 On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:
 We either get the DFA I'm building, something like it or 
 we're toast in commercial usage.
What is DFA?
I always read “deterministic finite automaton.” (I hate acronyms on the forums.)
And I read that as Data Flow Analysis. As an ESL (English as a Second Language), I wish we had some sort reference for this in this Forum. Matheus.
Jul 29
prev sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Tuesday, 23 July 2024 at 11:53:24 UTC, Quirin Schroll wrote:
 On Saturday, 20 July 2024 at 12:57:10 UTC, ryuukk_ wrote:
 On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:
 We either get the DFA I'm building, something like it or 
 we're toast in commercial usage.
What is DFA?
I always read “deterministic finite automaton.” (I hate acronyms on the forums.)
Same
Jul 30
prev sibling next sibling parent monkyyy <crazymonkyyy gmail.com> writes:
On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) Andrew 
Cattermole wrote:
 It is not if, but when will D be banned from being used in 
 critical or long standing applications.
Dont fall for political memes; while nist/cia holds allot of sway over the tech ecosystem, its not total and irresponsible. They can put money in, and ask nicely for programmers to use rust; it wont go over well if they mandate it.
Jul 20
prev sibling parent reply aberba <karabutaworld gmail.com> writes:
On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) Andrew 
Cattermole wrote:
 On 20/07/2024 11:33 AM, H. S. Teoh wrote:
 On Fri, Jul 19, 2024 at 10:49:23PM +0000, mw via Digitalmars-d 
 wrote:
 [...]
[...] It's 2024, and a NULL pointer brought down half the world's servers. Just gives you *so* much confidence in technology. :-D // In other news, this is yet another nail in the coffin of memory-unsafe languages. We're slowly, but surely inching towards the day when the likes of C and C++ will finally be relegated to the dustbin of history... T
It is not if, but when will D be banned from being used in critical or long standing applications.
Can't critical applications be written with safe turned on? How does D fall short in that regard?
Jul 21
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/07/2024 5:31 AM, aberba wrote:
 On Saturday, 20 July 2024 at 05:58:19 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 On 20/07/2024 11:33 AM, H. S. Teoh wrote:
 On Fri, Jul 19, 2024 at 10:49:23PM +0000, mw via Digitalmars-d wrote:
 [...]
[...] It's 2024, and a NULL pointer brought down half the world's servers. Just gives you *so* much confidence in technology. :-D // In other news, this is yet another nail in the coffin of memory-unsafe languages. We're slowly, but surely inching towards the day when the likes of C and C++ will finally be relegated to the dustbin of history... T
It is not if, but when will D be banned from being used in critical or long standing applications.
Can't critical applications be written with safe turned on? How does D fall short in that regard?
That covers non-lifetime, non-segfault, and non-assert issues. Unfortunately you cannot rely on using things like signal handlers to throw an exception, as you may not own the thread let alone the process to have one. Asserts/boundchecks/null deref these things cannot bring down the process. This includes for web services too. Imagine trying to explain to somebody that they lost 100k in sales because the web server they were connected to segfaulted out because it was written in D and not in an application VM language like Java. It would go down very well!
Jul 21
next sibling parent reply aberba <karabutaworld gmail.com> writes:
On Sunday, 21 July 2024 at 17:39:23 UTC, Richard (Rikki) Andrew 
Cattermole wrote:
 On 22/07/2024 5:31 AM, aberba wrote:
 [...]
That covers non-lifetime, non-segfault, and non-assert issues. Unfortunately you cannot rely on using things like signal handlers to throw an exception, as you may not own the thread let alone the process to have one. Asserts/boundchecks/null deref these things cannot bring down the process. This includes for web services too. Imagine trying to explain to somebody that they lost 100k in sales because the web server they were connected to segfaulted out because it was written in D and not in an application VM language like Java. It would go down very well!
live?
Jul 21
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/07/2024 5:42 AM, aberba wrote:
 On Sunday, 21 July 2024 at 17:39:23 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 On 22/07/2024 5:31 AM, aberba wrote:
 [...]
That covers non-lifetime, non-segfault, and non-assert issues. Unfortunately you cannot rely on using things like signal handlers to throw an exception, as you may not own the thread let alone the process to have one. Asserts/boundchecks/null deref these things cannot bring down the process. This includes for web services too. Imagine trying to explain to somebody that they lost 100k in sales because the web server they were connected to segfaulted out because it was written in D and not in an application VM language like Java. It would go down very well!
live?
Within a function body that provides spatial memory lifetime guarantees. That is the easy part, its inter-function which is the hard part and it does not attempt to solve that. I.e. storing a pointer into some objects and knowing that you have the only access to it, and if you extract it you have the only value of it.
Jul 21
parent reply aberba <karabutaworld gmail.com> writes:
On Sunday, 21 July 2024 at 17:45:26 UTC, Richard (Rikki) Andrew 
Cattermole wrote:
 On 22/07/2024 5:42 AM, aberba wrote:
 On Sunday, 21 July 2024 at 17:39:23 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:
 [...]
live?
Within a function body that provides spatial memory lifetime guarantees. That is the easy part, its inter-function which is the hard part and it does not attempt to solve that. I.e. storing a pointer into some objects and knowing that you have the only access to it, and if you extract it you have the only value of it.
Why isn't that the goal instead of just what live is proposing?
Jul 22
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/07/2024 7:15 PM, aberba wrote:
 On Sunday, 21 July 2024 at 17:45:26 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 On 22/07/2024 5:42 AM, aberba wrote:
 On Sunday, 21 July 2024 at 17:39:23 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 [...]
live?
Within a function body that provides spatial memory lifetime guarantees. That is the easy part, its inter-function which is the hard part and it does not attempt to solve that. I.e. storing a pointer into some objects and knowing that you have the only access to it, and if you extract it you have the only value of it.
Why isn't that the goal instead of just what live is proposing?
We have tried to explain this to Walter, many times. He hasn't understood it for whatever reason.
Jul 22
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/21/2024 10:39 AM, Richard (Rikki) Andrew Cattermole wrote:
 Unfortunately you cannot rely on using things like signal handlers to throw an 
 exception, as you may not own the thread let alone the process to have one.
 
 Asserts/boundchecks/null deref these things cannot bring down the process.
 
 This includes for web services too.
 
 Imagine trying to explain to somebody that they lost 100k in sales because the 
 web server they were connected to segfaulted out because it was written in D
and 
 not in an application VM language like Java. It would go down very well!
D is a systems programming language, and you can write hardware exception signal handlers in it.
Jul 22
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 23/07/2024 4:42 AM, Walter Bright wrote:
 On 7/21/2024 10:39 AM, Richard (Rikki) Andrew Cattermole wrote:
 Unfortunately you cannot rely on using things like signal handlers to 
 throw an exception, as you may not own the thread let alone the 
 process to have one.

 Asserts/boundchecks/null deref these things cannot bring down the 
 process.

 This includes for web services too.

 Imagine trying to explain to somebody that they lost 100k in sales 
 because the web server they were connected to segfaulted out because 
 it was written in D and not in an application VM language like Java. 
 It would go down very well!
D is a systems programming language, and you can write hardware exception signal handlers in it.
Indeed you can! If you own the process, and own the thread. If you do not own both, you cannot do this, which happens to be the scenario I am talking about.
Jul 22
prev sibling next sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Friday, 19 July 2024 at 23:33:44 UTC, H. S. Teoh wrote:

 It's 2024, and a NULL pointer brought down half the world's 
 servers.
...and the world didn't end.
Jul 19
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 20/07/2024 6:53 PM, Max Samukha wrote:
 On Friday, 19 July 2024 at 23:33:44 UTC, H. S. Teoh wrote:
 
 It's 2024, and a NULL pointer brought down half the world's servers.
...and the world didn't end.
Your dismissal is far too early. It affected hospitals, emergency dispatchers (911), not just financial systems. Until a postmortem is done, there is no way to know that nobody was harmed.
Jul 20
parent Max Samukha <maxsamukha gmail.com> writes:
On Saturday, 20 July 2024 at 07:13:21 UTC, Richard (Rikki) Andrew 
Cattermole wrote:

 
 It's 2024, and a NULL pointer brought down half the world's 
 servers.
...and the world didn't end.
Your dismissal is far too early. It affected hospitals, emergency dispatchers (911), not just financial systems. Until a postmortem is done, there is no way to know that nobody was harmed.
It'd also be nice to have a comprehensive account of the harm done by the soultions to the null deref problem. It must have caused quite a few suicides among programmers.
Jul 20
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/19/2024 4:33 PM, H. S. Teoh wrote:
 In other news, this is yet another nail in the coffin of memory-unsafe
 languages. We're slowly, but surely inching towards the day when the
 likes of C and C++ will finally be relegated to the dustbin of
 history...
It wasn't a memory unsafe error. It was a failure to deal with a program that self-detected a fault and exited. There are many kinds of hardware exceptions. The default behavior of them is to exit the program. The default behavior can be overridden with a handler that can then do whatever the programmer needs to happen (like restart, or engage the backup, or shut down gracefully). It's not different than failing to catch a thrown C++ exception - the default is exit the program.
Jul 21
prev sibling parent reply GrimMaple <grimmaple95 gmail.com> writes:
On Friday, 19 July 2024 at 23:33:44 UTC, H. S. Teoh wrote:
 It's 2024, and a NULL pointer brought down half the world's 
 servers.

 Just gives you *so* much confidence in technology. :-D
It's not a NULL pointer that did this, but rather a combination of * A NULL pointer * Poor testing that allowed the code to be pushed upstream * Bad Microsoft policies that lead to poorly tested code to be auto-installed via automatic updates In cases like this it's never just one small thing that causes Really Bad Things™, it's usually a combination of poor decisions. IMO, the only actually wrong thing in this whole situation is how shitty updates are forced onto everyone without proper testing periods
Jul 24
next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Wed, Jul 24, 2024 at 04:27:15PM +0000, GrimMaple via Digitalmars-d wrote:
 On Friday, 19 July 2024 at 23:33:44 UTC, H. S. Teoh wrote:
 It's 2024, and a NULL pointer brought down half the world's servers.
 
 Just gives you *so* much confidence in technology. :-D
It's not a NULL pointer that did this, but rather a combination of * A NULL pointer * Poor testing that allowed the code to be pushed upstream * Bad Microsoft policies that lead to poorly tested code to be auto-installed via automatic updates In cases like this it's never just one small thing that causes Really Bad Things™, it's usually a combination of poor decisions.
Which means that there must be a *lot* of poor decisions going around, enough for the wrong ones to line up coincidentally to cause a disastrous failure. Which just makes me all warm and fuzzy with confidence about the state of technology. :-P
 IMO, the only actually wrong thing in this whole situation is how
 shitty updates are forced onto everyone without proper testing periods
Or the fact that updates are pushed at all. Never been a fan of push updates. Or push anything, really. Binary blob pushes are the worst of them all. All it takes is for *somebody* to compromise the binary somewhere between the source and the user, and you're looking at half the world being compromised overnight. We're lucky that this buggy update was (very!) noticeable. It could have been a lot worse. Like a malicious backdoor that went unnoticed. In fact, it may have already happened, and we just haven't noticed it yet. Remember, you heard it here first. :-P T -- "I suspect the best way to deal with procrastination is to put off the procrastination itself until later. I've been meaning to try this, but haven't gotten around to it yet. " -- swr
Jul 24
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/24/2024 9:27 AM, GrimMaple wrote:
 In cases like this it's never just one small thing that causes Really Bad 
 Things™, it's usually a combination of poor decisions. IMO, the only
actually 
 wrong thing in this whole situation is how shitty updates are forced onto 
 everyone without proper testing periods
I also heard that Crowdstrike normally rate limits pushing updates, so if it is bad the damage was limited. But that limit was turned off for this update.
Jul 24
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Friday, 19 July 2024 at 22:49:23 UTC, mw wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19

 Crowdstrike Analysis:

 It was a NULL pointer from the memory unsafe C++ language.

 Since I am a professional C++ programmer, let me decode this 
 stack trace dump for you.

 ....
There's gonna be a lot of discussion about "use optional types", "nullable types are the evil", etc. In my opinion the problem is more a lack of code instrumentation. In styx you can instrument the code so that 'every fucking damn GEP' ("member access" for the profans) is checked against null. Same for member functions calls. It's very costly but that helps much. Anyway here is the obligatory legendary video of the slap: https://www.youtube.com/watch?v=bLHL75H_VEM take care B.
Jul 20
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/20/2024 4:37 AM, Basile B. wrote:
 There's gonna be a lot of discussion about "use optional types", "nullable
types 
 are the evil", etc. In my opinion the problem is more a lack of code 
 instrumentation. In styx you can instrument the code so that 'every fucking
damn 
 GEP' ("member access" for the profans) is checked against null. Same for
member 
 functions calls. It's very costly but that helps much.
What happens, then, with optional types and you've got an unexpected null that the program was not designed for? Probably exit the program. The same thing that an uncaught null exception does. You can write an exception handler to intercept them, and do whatever you want.
Jul 21
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/19/2024 3:49 PM, mw wrote:
 It was a NULL pointer from the memory unsafe C++ language.
https://stackoverflow.com/questions/2663456/how-to-write-a-signal-handler-to-catch-sigsegv
Jul 21
prev sibling next sibling parent reply Sergey <kornburn yandex.ru> writes:
On Friday, 19 July 2024 at 22:49:23 UTC, mw wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19

 Crowdstrike Analysis:

 It was a NULL pointer from the memory unsafe C++ language.

 Since I am a professional C++ programmer, let me decode this 
 stack trace dump for you.

 ....
another analysis with some critique of initial twit: https://x.com/taviso/status/1814762302337654829
Jul 21
parent Andrea Fontana <nospam example.com> writes:
On Sunday, 21 July 2024 at 18:00:12 UTC, Sergey wrote:
 another analysis with some critique of initial twit:
 https://x.com/taviso/status/1814762302337654829
And like I wrote yesterday in a reply to this tweet, it's strange he is a c++ professional... https://x.com/twittatore/status/1814781397435097352 Andrea
Jul 21
prev sibling next sibling parent Ogi <ogion.art gmail.com> writes:
On Friday, 19 July 2024 at 22:49:23 UTC, mw wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19

 Crowdstrike Analysis:

 It was a NULL pointer from the memory unsafe C++ language.
Would be match better if it was an unchecked Rust’s `Result`. It’s not crashing, it’s falling in style!
Jul 22
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 19 July 2024 at 22:49:23 UTC, mw wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19

 Crowdstrike Analysis:

 It was a NULL pointer from the memory unsafe C++ language.
I heard the cause was corrupted configuration files. What a safe program should do in this case? On error resume next?
Jul 23
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 23 July 2024 at 08:36:56 UTC, Kagamin wrote:
 On Friday, 19 July 2024 at 22:49:23 UTC, mw wrote:
 https://twitter.com/Perpetualmaniac/status/1814376668095754753?t=fAIwQeQpf0HJ9uh-gxqE9Q&s=19

 Crowdstrike Analysis:

 It was a NULL pointer from the memory unsafe C++ language.
I heard the cause was corrupted configuration files. What a safe program should do in this case? On error resume next?
What about sanitise the input while ingesting it? What about reading Perrow's "Normal Accident"? The failure is not technical, actually it's worst, it's a company cultural failure. And, btw, for who dislike macOS (I've ready hilarious comments in these days around) https://eclecticlight.co/2024/07/22/could-our-macs-be-crowdstruck/ /P
Jul 23
parent reply Kagamin <spam here.lot> writes:
On Tuesday, 23 July 2024 at 09:28:07 UTC, Paolo Invernizzi wrote:
 What about sanitise the input while ingesting it?
What it should do after sanitization? Resume? In misconfigured state?
Jul 24
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 24 July 2024 at 07:31:21 UTC, Kagamin wrote:
 On Tuesday, 23 July 2024 at 09:28:07 UTC, Paolo Invernizzi 
 wrote:
 What about sanitise the input while ingesting it?
What it should do after sanitization? Resume? In misconfigured state?
As it seems that the kernel module that's ingesting the configuration files is marked as required for boot, that module should just raise an hand signalling "Huston, we have a problem" to the management console, and do nothing at all. /P
Jul 24
parent reply Kagamin <spam here.lot> writes:
I don't think computers can do nothing. When they are turned off 
they do nothing indeed, but something must turn off the computer 
for it to become turned off, and that's an involved procedure 
that doesn't count as "do nothing". So it's not clear what you 
mean, what CPU instruction should execute to "do nothing"?
Jul 24
next sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 24 July 2024 at 14:03:31 UTC, Kagamin wrote:
 I don't think computers can do nothing. When they are turned 
 off they do nothing indeed, but something must turn off the 
 computer for it to become turned off, and that's an involved 
 procedure that doesn't count as "do nothing". So it's not clear 
 what you mean, what CPU instruction should execute to "do 
 nothing"?
Are you kidding me?
Jul 24
prev sibling parent reply Abdulhaq <alynch4048 gmail.com> writes:
On Wednesday, 24 July 2024 at 14:03:31 UTC, Kagamin wrote:
 I don't think computers can do nothing. When they are turned 
 off they do nothing indeed, but something must turn off the 
 computer for it to become turned off, and that's an involved 
 procedure that doesn't count as "do nothing". So it's not clear 
 what you mean, what CPU instruction should execute to "do 
 nothing"?
https://en.wikipedia.org/wiki/NOP_(code)
Jul 24
parent Kagamin <spam here.lot> writes:
On Wednesday, 24 July 2024 at 16:47:26 UTC, Abdulhaq wrote:
 On Wednesday, 24 July 2024 at 14:03:31 UTC, Kagamin wrote:
 I don't think computers can do nothing. When they are turned 
 off they do nothing indeed, but something must turn off the 
 computer for it to become turned off, and that's an involved 
 procedure that doesn't count as "do nothing". So it's not 
 clear what you mean, what CPU instruction should execute to 
 "do nothing"?
https://en.wikipedia.org/wiki/NOP_(code)
So a safe program must execute nop instruction in an infinite loop? How is it better than crashing?
Jul 25
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I talked with a person who has more in depth knowledge about it.

The null pointer came from reading a file and treating it as a data structure. 
The file was unexpectedly full of zeros.

Any language that allows casting a buffer read from the disk to a pointer would 
fail. This includes any language with unsafe blocks, or uses a FFI to get
around 
the language protections.
Jul 24
next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Wed, Jul 24, 2024 at 05:12:28PM -0700, Walter Bright via Digitalmars-d wrote:
 I talked with a person who has more in depth knowledge about it.
 
 The null pointer came from reading a file and treating it as a data
 structure. The file was unexpectedly full of zeros.
Reading a file and treating it as a data structure without verification or at the very least sanity checking is a rather unwise thing to do. Unfortunately this is a rather common practice in enterprise code. Sad to say, I've been guilty of this myself. :-P Not catching this during testing, though, is rather disappointing. Though also unsurprising... having worked for a few decades in the industry dealing with very large codebases, I've seen firsthand how testing is often skimped on, especially when there's a looming deadline or some similar pressure to get it done quickly and move on to the next item in a large pile of things that need to get down by last week. And sometimes the tester also has no idea what he's testing and what corner cases to watch out for; he also has a large pile of other work to get to and so often just randomly does a couple of ordinary runs, observes nothing unusual, and calls it done. This is especially bad with software that requires lots of repetitive testing of features previously verified. The temptation to skimp out a few cases to speed things up is very strong. All sorts of things may slip through unnoticed. This is why D's built-in unittests, in spite of their warts, is a major tool for improving software quality. You *want* tests to be as automated as possible, because the rate of human error is especially high when it comes to repetitive (but very necessary) testing of previously working features. Having the machine automatically verify previous working features is a big step in eliminating human error (aka laziness) in this area.
 Any language that allows casting a buffer read from the disk to a
 pointer would fail. This includes any language with unsafe blocks, or
 uses a FFI to get around the language protections.
Casting a buffer read from disk to a pointer is, in general, a risky and unwise thing to do. But the temptation to do it anyway is very strong, especially in low level code or where performance is a concern. At least, one should be thankful the file was filled with zeroes and not something else, like malicious code or code that coincidentally did something destructive like format the disk or overwriting the boot sector. T -- The past, present, and future walk into a bar. It was tense. Then the physicist walks in, and it was tensor. Finally, the mathematician walks in, and it was ten sets.
Jul 24
prev sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 25/07/2024 12:12 PM, Walter Bright wrote:
 I talked with a person who has more in depth knowledge about it.
 
 The null pointer came from reading a file and treating it as a data 
 structure. The file was unexpectedly full of zeros.
 
 Any language that allows casting a buffer read from the disk to a 
 pointer would fail. This includes any language with unsafe blocks, or 
 uses a FFI to get around the language protections.
You have almost got it. It is not any language with that capability, it is any language with that capability that does not force you to check for null before the dereference.
Jul 24
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 25/07/2024 2:24 PM, Richard (Rikki) Andrew Cattermole wrote:
 On 25/07/2024 12:12 PM, Walter Bright wrote:
 I talked with a person who has more in depth knowledge about it.

 The null pointer came from reading a file and treating it as a data 
 structure. The file was unexpectedly full of zeros.

 Any language that allows casting a buffer read from the disk to a 
 pointer would fail. This includes any language with unsafe blocks, or 
 uses a FFI to get around the language protections.
You have almost got it. It is not any language with that capability, it is any language with that capability that does not force you to check for null before the dereference.
Note: you do not need to leave safety to have this exact situation to occur. ```d void func(int* ptr) safe { int v = *ptr; // BOOM! } ``` The problem isn't going into unsafe code, its that you made an assumption that either is the reality, or is never correct and is guaranteed to error out.
Jul 24
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/24/2024 7:32 PM, Richard (Rikki) Andrew Cattermole wrote:
 The problem isn't going into unsafe code, its that you made an assumption that 
 either is the reality, or is never correct and is guaranteed to error out.
A null pointer seg fault is not an unsafe thing. Memory unsafety is about memory corruption. A seg fault is not memory corruption. Consider: ``` int* p; *p = 4; // seg fault, program terminates ... int* q; assert(9); // program terminates ``` Both of these cases are memory safe. Both default to summarily terminating the program. Both can have handlers installed to "recover" and do something nice. Before there was hardware memory protection, with DOS writing through a null pointer meant scrambling the operating system, leading to all kinds of horrible things like scrambling your hard disk as well. Imagine trying to find what went wrong. It's like your house burned to ashes and now you have to figure out the source of the blaze. When protected mode became available, it was a miracle. Having the hardware check *every* pointer *every* time for validity was a huge advancement. And the code still runs at full speed! Even better, you could get a stack trace pointing at the bug in your code. I used to spend *weeks* trying to find memory corruption bugs. Today it's a few seconds. Seg faults are a great gift!
Jul 24
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I occasionally peruse lists of software bugs that are sources of security 

Null pointer seg faults don't even make the list - because they are not
security 
vulnerabilities.

D nails the array overflow prevention.
Jul 25
parent kdevel <kdevel vogtner.de> writes:
On Thursday, 25 July 2024 at 07:47:11 UTC, Walter Bright wrote:

 D nails the array overflow prevention.
Sure. But D still allows corruption going unnoticed: mems.d ``` import std.stdio; auto foo () { int [4] f = [1, 2, 3, 4]; int [] q = f; return q; } void main () { auto s = foo; writeln (s); } ``` $ dmd mems.d $ ./mems [4, 0, -10016, 32767] It requires safe annotation PLUS -preview=dip1000 in order for dmd to catch this (Error: scope variable `q` may not be returned).
Jul 25
prev sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 25/07/2024 6:48 PM, Walter Bright wrote:
 I used to spend /weeks/ trying to find memory corruption bugs. Today 
 it's a few seconds. Seg faults are a great gift!
Right up until they bring down 8.5 million computers world wide, and impact almost everyone on the planet. We got lucky this time, that there is an "easy" fix to get these machines working again. It does not matter that there probably won't be a CVE from this outage. Fact is, some data was sourced, that was not validated before access that could have been caught before a world wide outage that took out _hospitals_.
Jul 25
next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Jul 26, 2024 at 06:42:55AM +1200, Richard (Rikki) Andrew Cattermole via
Digitalmars-d wrote:
 On 25/07/2024 6:48 PM, Walter Bright wrote:
 I used to spend /weeks/ trying to find memory corruption bugs. Today
 it's a few seconds. Seg faults are a great gift!
Right up until they bring down 8.5 million computers world wide, and impact almost everyone on the planet. We got lucky this time, that there is an "easy" fix to get these machines working again. It does not matter that there probably won't be a CVE from this outage. Fact is, some data was sourced, that was not validated before access that could have been caught before a world wide outage that took out _hospitals_.
Fact is, reading in a file and casting the contents into a pointer without prior verification is a very unwise thing to do. No amount of language features will save you from the consequences. Somebody has to write the code to verify the data before acting on it. If nobody wrote the verification code, whether program segfaults, continues silently and corrupts data, formats the harddrive, or launches nuclear missiles, is really just secondary. Also, the fact that one tiny flaw like this can bring down half the computers across the whole world is another major lesson that people don't seem to be learning from. Basically, the OS is a single point of failure; when it fails, you're up the creek without a paddle. Maybe it's time somebody pulled a Walter to design fault-resistant redundant OS instances, airplane-style. :-P At the very least, OS upgrades should be handled much more conservatively than they are right now. For example, the patched OS should be something separate from the running OS; it should be brought up separately before the old OS retires itself and hands over control. Easier said than done, of course, but given what has happened, people really need to be thinking about this seriously. Another factor is, push updates are evil. What really ought to have happened is that an update notification should have been sent, and the admins should have approved it before it was actually installed. (After testing the patch in a controlled environment, before pushing it out to live systems.) But I'm probably barking up the wrong tree here... people these days are all gung-ho about fully unattended upgrades and fully automated everything, who needs anybody to check the sanity of an upgrade. Well, we're staring at the consequences of this attitude right now. T -- Век живи - век учись. А дураком помрёшь.
Jul 25
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 26/07/2024 7:00 AM, H. S. Teoh wrote:
 Another factor is, push updates are evil. What really ought to have
 happened is that an update notification should have been sent, and the
 admins should have approved it before it was actually installed. (After
 testing the patch in a controlled environment, before pushing it out to
 live systems.) But I'm probably barking up the wrong tree here... people
 these days are all gung-ho about fully unattended upgrades and fully
 automated everything, who needs anybody to check the sanity of an
 upgrade.  Well, we're staring at the consequences of this attitude right
 now.
You are indeed barking up the wrong tree. I am already convinced that there was multiple failures went on. I was able to determine them just from the failures I was seeing on Twitter a few hours in. But, I can't solve those. I can however solve forcing a D user to check for nullability, and if that is the best that we can do, then that's all we can do.
Jul 25
next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Jul 26, 2024 at 07:07:44AM +1200, Richard (Rikki) Andrew Cattermole via
Digitalmars-d wrote:
[...]
 I am already convinced that there was multiple failures went on.
 
 I was able to determine them just from the failures I was seeing on
 Twitter a few hours in.
 
 But, I can't solve those.
 
 I can however solve forcing a D user to check for nullability, and if
 that is the best that we can do, then that's all we can do.
Null checking doesn't solve this particular problem, though. We just got lucky that the file contained zeroes. If the file had contained non-zeroes that weren't what the code was expecting, treating that as a pointer may have lead to even more disastrous consequences. A null check won't save you then. (I've actually seen this before, in case you think this is theoretical.) T -- I've been around long enough to have seen an endless parade of magic new techniques du jour, most of which purport to remove the necessity of thought about your programming problem. In the end they wind up contributing one or two pieces to the collective wisdom, and fade away in the rearview mirror. -- Walter Bright
Jul 25
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 26/07/2024 7:30 AM, H. S. Teoh wrote:
 On Fri, Jul 26, 2024 at 07:07:44AM +1200, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:
 [...]
 I am already convinced that there was multiple failures went on.

 I was able to determine them just from the failures I was seeing on
 Twitter a few hours in.

 But, I can't solve those.

 I can however solve forcing a D user to check for nullability, and if
 that is the best that we can do, then that's all we can do.
Null checking doesn't solve this particular problem, though. We just got lucky that the file contained zeroes. If the file had contained non-zeroes that weren't what the code was expecting, treating that as a pointer may have lead to even more disastrous consequences. A null check won't save you then. (I've actually seen this before, in case you think this is theoretical.) T
I don't think that this is a theoretical problem at all. I considered this ages ago. For D we've solved all the non-null pointer safety issues with `` safe`` that we can solve. The only thing left is nullability which would have protected against the symptom and therefore prevented the outage.
Jul 25
next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Jul 26, 2024 at 07:42:43AM +1200, Richard (Rikki) Andrew Cattermole via
Digitalmars-d wrote:
 On 26/07/2024 7:30 AM, H. S. Teoh wrote:
 On Fri, Jul 26, 2024 at 07:07:44AM +1200, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:
[...]
 I can however solve forcing a D user to check for nullability, and
 if that is the best that we can do, then that's all we can do.
Null checking doesn't solve this particular problem, though. We just got lucky that the file contained zeroes. If the file had contained non-zeroes that weren't what the code was expecting, treating that as a pointer may have lead to even more disastrous consequences. A null check won't save you then. (I've actually seen this before, in case you think this is theoretical.)
[...]
 I don't think that this is a theoretical problem at all. I considered
 this ages ago.
How does null prevention solve the case of garbage data read from file being cast into a pointer? A null check won't save you from that.
 For D we've solved all the non-null pointer safety issues with
 `` safe`` that we can solve.
 
 The only thing left is nullability which would have protected against
 the symptom and therefore prevented the outage.
Only in this particular instance. It wouldn't have prevented anything had the file contained unexpected data other than zeroes. T -- A linguistics professor was lecturing to his class one day. "In English," he said, "A double negative forms a positive. In some languages, though, such as Russian, a double negative is still a negative. However, there is no language wherein a double positive can form a negative." A voice from the back of the room piped up, "Yeah, yeah."
Jul 25
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 26/07/2024 8:59 AM, H. S. Teoh wrote:
 On Fri, Jul 26, 2024 at 07:42:43AM +1200, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:
 On 26/07/2024 7:30 AM, H. S. Teoh wrote:
 On Fri, Jul 26, 2024 at 07:07:44AM +1200, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:
[...]
 I can however solve forcing a D user to check for nullability, and
 if that is the best that we can do, then that's all we can do.
Null checking doesn't solve this particular problem, though. We just got lucky that the file contained zeroes. If the file had contained non-zeroes that weren't what the code was expecting, treating that as a pointer may have lead to even more disastrous consequences. A null check won't save you then. (I've actually seen this before, in case you think this is theoretical.)
[...]
 I don't think that this is a theoretical problem at all. I considered
 this ages ago.
How does null prevention solve the case of garbage data read from file being cast into a pointer? A null check won't save you from that.
 For D we've solved all the non-null pointer safety issues with
 `` safe`` that we can solve.

 The only thing left is nullability which would have protected against
 the symptom and therefore prevented the outage.
Only in this particular instance. It wouldn't have prevented anything had the file contained unexpected data other than zeroes. T
It doesn't solve the underlying issue, I acknowledge that, but it would force the question on validation which obviously wasn't being asked.
Jul 25
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/25/2024 2:04 PM, Richard (Rikki) Andrew Cattermole wrote:
 It doesn't solve the underlying issue, I acknowledge that, but it would force 
 the question on validation which obviously wasn't being asked.
The hardware will not only check for null pointer accesses, but check for *any* pointer that points to an address that is not mapped to the process.
Jul 26
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/25/2024 12:42 PM, Richard (Rikki) Andrew Cattermole wrote:
 The only thing left is nullability which would have protected against the 
 symptom and therefore prevented the outage.
If your null check is: assert(p != null); when it trips, your program exits, causing the outage. Assert's are for detecting programming bugs. There is no recovery from programming bugs, because the program has entered an unknown state and cannot be relied on for anything.
Jul 26
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/25/2024 12:07 PM, Richard (Rikki) Andrew Cattermole wrote:
 I can however solve forcing a D user to check for nullability, and if that is 
 the best that we can do, then that's all we can do.
The hardware already does that.
Jul 26
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
It did crash the computers. But AFAIK, nobody said any data was corrupted or
lost.

Without hardware memory protection, this could have been much much worse.
Jul 25
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 26/07/2024 6:57 PM, Walter Bright wrote:
 It did crash the computers. But AFAIK, nobody said any data was 
 corrupted or lost.
 
 Without hardware memory protection, this could have been much much worse.
Yes as a fallback it did potentially prevent corrupting the kernel silently and that is a good thing! No arguments against the MMU acting in this way. However where we are differing is not that the MMU did good, but that it is a fallback. If the MMU is throwing an exception something has gone terribly wrong elsewhere and having a programming language force you to check for null is a reasonable precaution against this. There is a reason why application VM languages have over the last 15 years been adopting such analysis. They may control the process & threads, but even then it still makes sense to prevent such errors.
Jul 26
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/26/2024 3:21 AM, Richard (Rikki) Andrew Cattermole wrote:
 If the MMU is throwing an exception something has gone terribly wrong
elsewhere 
 and having a programming language force you to check for null is a reasonable 
 precaution against this.
``` assert(p != null); // throws an unrecoverable exception ``` Array bounds errors also throw an unrecoverable exception
Jul 26
parent reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Friday, 26 July 2024 at 16:11:28 UTC, Walter Bright wrote:
 On 7/26/2024 3:21 AM, Richard (Rikki) Andrew Cattermole wrote:
 If the MMU is throwing an exception something has gone 
 terribly wrong elsewhere and having a programming language 
 force you to check for null is a reasonable precaution against 
 this.
``` assert(p != null); // throws an unrecoverable exception ``` Array bounds errors also throw an unrecoverable exception
From what I've seen, the best argument against adding something to the type system to assist the programmer with handling null correctly where it may occur is that new, expensive analysis and possibly new control-flow statements are needed so that in cases where something nullable has been definitely ruled out of being null, the compiler won't complain anymore about the programmer failing to handle null. The difference between an array index and a possibly null pointer is that almost all the time almost all the possible values for the array index are invalid, so it's quite obvious that out of bounds access must be accounted for, but also static analysis can't really make sure an index is in bounds. For null, however, it's a single value that's problematic. That a variable has a value different from one known iffy one is rather easy to express in a type system. What helps is that some core language constructs (`new`) return non-null values, but as soon as the value is created, the type system forgets its non-nullness. It's not about having defined semantics or undefined behavior, the semantics and behavior are defined. A type system that distinguishes nullable and non-nullable reference types helps programmers keeping track of when they have to handle null and when they don't, and whether a function expects null arguments or not. When all reference types can be null everywhere, but won't be in practice, it teaches programmers not to think of them. But the worst offender are `ref` parameters. They can be null, too, (or `*null` if you're pedantic), but practically no function taking a reference expects and handles the case where the reference is null — or even documents that assumption. Even seeing `&var is null` probably looks to most like an obvious case of a vacuously `false` expression. The reason why null references are allowed is because a reference can easily become null by accident in ` safe` code, which means they need to have defined behavior. But that's only because pointers aren't distinguished if they can be null or not. Most references are initialized by obvious non-null expressions, only a pointer dereference is the an exception.
Jul 26
parent reply Walter Bright <newshound2 digitalmars.com> writes:
You're right that ref's can be null, too. C++ says they can't be null, but it's 
trivial to make one in C++, so a fat lot of good that does.

Let's say we have a linked list. It will have a `next` pointer:

```
struct List {
     int payload;
     List* next;
}
```

A null value is perfectly valid for `next`, as that is how the end is found. At 
least in my coding, I use null values all the time to signify there is nothing 
there.

It it wasn't null, some other value would have to be there to signify "not a 
valid item". If there must be something there, then the value would have to be 
checked against the "not a valid item" value. What should I then do if it 
unexpectedly is the "not a valid item"? The only sensible thing is to abort the 
program, as it's a program bug.

But with null, I don't have to check, because the hardware does it for free.

The only time a null pointer dereference is an actual problem is when running
on 
a machine that does not have memory protection, which are decades in
obsolescence.

The real issue with null pointer seg faults is the screen dump of mysterious 
numbers and letters.

The real biggest mistake in C is the eager decay of arrays to pointers, and the 
tragedy of C is nobody has any interest in fixing it.
Jul 26
next sibling parent reply Elias (0xEAB) <desisma heidel.beer> writes:
On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
 But with null, I don't have to check, because the hardware does 
 it for free.

 The only time a null pointer dereference is an actual problem 
 is when running on a machine that does not have memory 
 protection, which are decades in obsolescence.
Seems not to be exactly true… <https://neugierig.org/software/blog/2022/06/wasm-c++.html>: “In Wasm a null pointer just refers to memory[0] and it is a legal address to read and write from.”
Jul 26
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/26/2024 8:23 PM, Elias (0xEAB) wrote:
 <https://neugierig.org/software/blog/2022/06/wasm-c++.html>: “In Wasm a null 
 pointer just refers to memory[0] and it is a legal address to read and write
from.”
Wow. A big mistake by Wasm.
Jul 27
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 28/07/2024 6:17 AM, Walter Bright wrote:
 On 7/26/2024 8:23 PM, Elias (0xEAB) wrote:
 <https://neugierig.org/software/blog/2022/06/wasm-c++.html>: “In Wasm 
 a null pointer just refers to memory[0] and it is a legal address to 
 read and write from.”
Wow. A big mistake by Wasm.
Turns out that the proposal to rectify this is WIP: https://github.com/WebAssembly/memory-control/blob/main/proposals/memory-control/Overview.md Although nothing in there regarding the trapping itself, but does mention one use case with protecting null.
Jul 27
prev sibling next sibling parent reply Nick Treleaven <nick geany.org> writes:
On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
 The only time a null pointer dereference is an actual problem 
 is when running on a machine that does not have memory 
 protection, which are decades in obsolescence.
I fail to see how this reply relates to Quirin's post. It's just a fact that programmers accidentally forget to handle the null case *when there is a valid way to handle null*. That is obviously a problem because users don't want their programs to abort unnecessarily. That valid way can just skipping the code that needs a dereference, or printing an error message, instead of trying to dereference null, depending on the situation. You know this. There is no single solution to always handle null, because it is not a valid address for data (for most programs). Aborting the program is fine, if the programmer was aware that a null dereference might occur. The problem is they often are not aware, even good programmers, because the type system always conflates nullability with pointers and programs are complex. It would be acceptable if you just said that making D null-safe was too disruptive. It is not acceptable to deny the problem exists.
Jul 27
parent reply Walter Bright <newshound2 digitalmars.com> writes:
It's true that many algorithms depend on a null pointer being a "sentinel", and 
people sometimes forget to check for it. That means:

1. if they forgot to check for the null special case, then the seg fault tells 
them where the error is

2. if null was supposed not ever happen, then the seg fault tells where the
error is

Dereferencing a null pointer is always a bug.
Jul 27
next sibling parent kdevel <kdevel vogtner.de> writes:
On Saturday, 27 July 2024 at 18:23:28 UTC, Walter Bright wrote:
 It's true that many algorithms depend on a null pointer being a 
 "sentinel", and people sometimes forget to check for it. That 
 means:

 1. if they forgot to check for the null special case, then the 
 seg fault tells them where the error is

 2. if null was supposed not ever happen, then the seg fault 
 tells where the error is

 Dereferencing a null pointer is always a bug.
The derefencing is not the bug, it is at best one of its symptoms. In D (in constrast to C++) the null ptr dereference is implementation defined.[1] I.e. if you need a test program that dereferences a null ptr you can write one in D (you cannot in C++). If that program does what it is expected ("[The] program will be aborted" [1]) one can hardly say that such a program contains a bug. [1] https://dlang.org/spec/type.html#pointers
Jul 27
prev sibling parent Nick Treleaven <nick geany.org> writes:
On Saturday, 27 July 2024 at 18:23:28 UTC, Walter Bright wrote:
 It's true that many algorithms depend on a null pointer being a 
 "sentinel", and people sometimes forget to check for it.
Or they aren't supposed to have a sentinel but they accidentally got passed a null value because the type system allows it.
 That means:

 1. if they forgot to check for the null special case, then the 
 seg fault tells them where the error is

 2. if null was supposed not ever happen, then the seg fault 
 tells where the error is
You don't get a segfault if your tests weren't run or don't (*or can't*) cover every case in development. Then *your users* get the segfault. Do you accept that the developer detecting those bugs at compile-time is advantageous to the user having their program abort? The user might not even know how to file a bug, and it could cost them money, time or worse.
Jul 29
prev sibling next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 27/07/2024 1:12 PM, Walter Bright wrote:
 The real issue with null pointer seg faults is the screen dump of 
 mysterious numbers and letters.
No. This isn't a problem at all. What is a problem is the loss of money. When you crash you take out many users interactions with your system. If somebody is mid-way through buying something that gives them a chance to decide not to. It is effectively a Denial of Service (DoS) attack that can occur with the press of the F5 key. It can take out your entire multi-server money source without any defense. In a lot of cases you do not own the thread, or the process. You can't just set the signal handler and do it at runtime. Audio plugins, game engines, interop with an application VM, kernel drivers, web services, COM (Office suite plugins, Windows services, explorer extensions). All of these are where D makes its money. It is where the foundation gets its donations from. If you are using an application VM language, an exception is thrown and you can handle it. It's solved. For a native language IFF you own the process AND thread you can use signal handlers safely to throw an exception (MOSTLY true, signals are very hairy and should be avoided if at all possible). There are two solutions for a native language that isn't just ignoring it like we do today. Check it at CT and warn/error if you do not handle null, or inject read barriers like we do for bounds checks to cause the runtime exception. Note: this is solved in C/C++ world. They have such analysis in their compilers: analyzer-null-dereference https://gcc.gnu.org/onlinedocs/gcc-10.1.0/gcc/Static-Analyzer-Options.html It only takes one major outage where a business loses money before they consider dumping a D companies solution. No client wants to hear: "We did this in a known unsafe language for this particular error, when a more main stream language has solutions to it and D doesn't."
Jul 27
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/27/2024 2:30 AM, Richard (Rikki) Andrew Cattermole wrote:
 There are two solutions for a native language that isn't just ignoring it like 
 we do today. Check it at CT and warn/error if you do not handle null, or
inject 
 read barriers like we do for bounds checks to cause the runtime exception.
How is `assert(p != null)` better?
 Note: this is solved in C/C++ world. They have such analysis in their
compilers: 
 analyzer-null-dereference
 
 https://gcc.gnu.org/onlinedocs/gcc-10.1.0/gcc/Static-Analyzer-Options.html
-Wanalyzer-possible-null-argument That's not solving it. Many simple cases can indeed be found by DFA. But not all. There's also a lot of security problem with C and C++. D does insert checks for array bounds overflow, because the hardware is of no help.
 It only takes one major outage where a business loses money before they
consider 
 dumping a D companies solution. No client wants to hear: "We did this in a
known 
 unsafe language for this particular error, when a more main stream language
has 
 solutions to it and D doesn't."
How is this different from `assert(p != null)` ?
Jul 27
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 28/07/2024 6:33 AM, Walter Bright wrote:
 On 7/27/2024 2:30 AM, Richard (Rikki) Andrew Cattermole wrote:
 There are two solutions for a native language that isn't just ignoring 
 it like we do today. Check it at CT and warn/error if you do not 
 handle null, or inject read barriers like we do for bounds checks to 
 cause the runtime exception.
How is `assert(p != null)` better?
Two reasons. 1. You made the decision. You had to consider what would happen in this situation. The question was asked, what happens should this be null? If you want to assert that's fine. What is not fine is making an assumption and never state it, never have it proven to be true. 2. It throws an exception (in D), which can be caught safely. It is only when exception chaining occurs that the state may not have been cleaned up correctly. An even better solution to using an assert, is to use an if statement instead. ```d if (int* ptr = var.field) { // success } else { // fail, you can gracefully degrade/log here } ``` With the help of ``?.`` operator, we can also make this very nice to use for loads and stores, AND be temporally safe! ```d if (S1* ptr1 = var.field1) { if (S2* ptr2 = ptr1.field2) { // success goto AfterFailure; } } // failure AfterFailure: ``` From: ```d if (int* ptr = var?.field1?.field2) { // success } else { // failure } ```
 Note: this is solved in C/C++ world. They have such analysis in their 
 compilers: analyzer-null-dereference

 https://gcc.gnu.org/onlinedocs/gcc-10.1.0/gcc/Static-Analyzer-Options.html
-Wanalyzer-possible-null-argument That's not solving it.
Yes and no. It cannot solve it completely in C or C++ because there is no language integration. For temporal safety, you have to make sure you do a load into a variable, and only then do the test. Which means this: ```d void func(int** ptr) { assert(*ptr !is null); int i = **ptr; // ERROR: `**ptr` is in an unknown type state, it could be null } ```
 Many simple cases can indeed be found by DFA. But not all. There's also 
 a lot of DFA trying to eliminate array bounds errors, but such errors 

 
 D does insert checks for array bounds overflow, because the hardware is 
 of no help.
I agree about bounds checks. However as read barriers, these are things that we do control, and throw an exception that we can catch. For this reason, bounds checks do not need CT analysis. They do not have to bring down an application if caught. On the other hand, null dereference requires signal handling to catch. If you don't own thread and process you cannot use that to throw an exception. For this reason it's not comparable and requires CT analysis to prevent bringing down the process. On the other hand, if you were to propose read barriers that throw exceptions for null dereference, I would agree to the statement that they are comparable.
 It only takes one major outage where a business loses money before 
 they consider dumping a D companies solution. No client wants to hear: 
 "We did this in a known unsafe language for this particular error, 
 when a more main stream language has solutions to it and D doesn't."
How is this different from `assert(p != null)` ?
See at top of my reply. A joke I've been making at BeerConf atm (more or less): User: Press F5 User: Press F5 User: Press F5, WHY ISN'T THE ABOUT PAGE SHOWING?????? User: Press F5 * 1000 Developer: Why are all the web server instances down? Developer: Oh a segfault due to null dereference User: Press F5 * 100000000000 Owner: Why did you use D! If you just used <insert mainstream application VM language> we wouldn't have lost millions of dollars!!!
Jul 27
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/27/2024 12:00 PM, Richard (Rikki) Andrew Cattermole wrote:
 1. You made the decision. You had to consider what would happen in this 
 situation. The question was asked, what happens should this be null? If you
want 
 to assert that's fine. What is not fine is making an assumption and never
state 
 it, never have it proven to be true.
?? Dereferencing a null pointer is always a bug, whether you decided to check for it or not.
 2. It throws an exception (in D), which can be caught safely. It is only when 
 exception chaining occurs that the state may not have been cleaned up
correctly.
Exceptions in D are the same as the ones used for seg faults (except for on Win64, where I couldn't figure out how the system exceptions worked).
 An even better solution to using an assert, is to use an if statement instead.
 
 ```d
 if (int* ptr = var.field) {
      // success
 } else {
      // fail, you can gracefully degrade/log here
 }
 ```
The reason exceptions were invented was because such code for every pointer dereference made reasonable code look quite ugly. Of course, you can still write such code if you like.
 Which means this:
 
 ```d
 void func(int** ptr) {
      assert(*ptr !is null);
 
      int i = **ptr; // ERROR: `**ptr` is in an unknown type state, it
could be null
 }
 ```
Which means the language will require you to manually insert assert()s everywhere.
 For this reason, bounds checks do not need CT analysis.
Whether it is needed or not, the compiler can't do it.
 They do not have to bring down an application if caught.
Array overflows are fatal programming bugs. A huge discussion about this raged in the n.g. many years ago. There was a camp that maintained that assert failures should be recoverable to the point that the program could continue. The other camp maintained that when a program enters a state unanticipated by the programmer, then the program is not recoverable, because one no longer has any idea what will happen. The only path forward is to stop the program, gracefully or not. Obviously, I'm in the latter camp. Obviously, one can write programs in the first camp (D lets you do whatever you want) but I cannot endorse it.
 On the other hand, null dereference requires signal handling to catch. If you 
 don't own thread and process you cannot use that to throw an exception. For
this 
 reason it's not comparable and requires CT analysis to prevent bringing down
the 
 process.
Whoever catches the signal will then stop the buggy, broken program from doing more damage. It's kinda the point of a computer with memory protection.
 On the other hand, if you were to propose read barriers that throw exceptions 
 for null dereference, I would agree to the statement that they are comparable.
That'll turn D into a very poorly performing language. Besides, throwing an exception is just what sig faults are. It's the same mechanism.
 Owner: Why did you use D! If you just used <insert mainstream application VM 
 language> we wouldn't have lost millions of dollars!!!
D is often unfairly accused of flaws.
Jul 27
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 28/07/2024 9:19 AM, Walter Bright wrote:
 On 7/27/2024 12:00 PM, Richard (Rikki) Andrew Cattermole wrote:
 1. You made the decision. You had to consider what would happen in 
 this situation. The question was asked, what happens should this be 
 null? If you want to assert that's fine. What is not fine is making an 
 assumption and never state it, never have it proven to be true.
?? Dereferencing a null pointer is always a bug, whether you decided to check for it or not.
The point is to make it not possible for you to dereference a null pointer to begin with. The compiler won't let you do it without dropping to inline assembly or doing some unsafe casts. If the compiler forces a check to occur, and it becomes null after, there is something very wrong going on. Likely stack corruption. At which point a segfault is absolutely the right tool for the job! ```d void func(int* ptr) { if (ptr !is null) { writeln(*ptr); // ok pointer is known to be good writeln(*ptr); // IF ptr is null this needs to segfault!!! STACK CORRUPTION??? } writeln(*ptr); // Error: ptr could be null!!! This should not compile } ```
 2. It throws an exception (in D), which can be caught safely. It is 
 only when exception chaining occurs that the state may not have been 
 cleaned up correctly.
Exceptions in D are the same as the ones used for seg faults (except for on Win64, where I couldn't figure out how the system exceptions worked).
Unless I can do: ```d try { ... } catch (NullPointerError) { ... } ``` It is not the same system from a "I have to write code to handle it" stand point.
 An even better solution to using an assert, is to use an if statement 
 instead.

 ```d
 if (int* ptr = var.field) {
      // success
 } else {
      // fail, you can gracefully degrade/log here
 }
 ```
The reason exceptions were invented was because such code for every pointer dereference made reasonable code look quite ugly. Of course, you can still write such code if you like.
It can be improved greatly by using ``?.`` operator for long chains. Yes, exceptions are how application VM languages do this and that's a good solution for them. But we can't. We can't setup the signal handler to throw the exception. We won't use a read barrier. So we have to force the check upon the user at CT.
 Which means this:

 ```d
 void func(int** ptr) {
      assert(*ptr !is null);

      int i = **ptr; // ERROR: `**ptr` is in an unknown type state, it 
 could be null
 }
 ```
Which means the language will require you to manually insert assert()s everywhere.
Have another look at the example. The assert is useless and that is the whole point of that example. You should be using if statements more often than not, AND you should only need to do this for variables that are loaded from an external source. ```d int* global; void func() { if (int val = *global) { // ok a check has occured } writeln(*global); // Error: no check has occured assert(global !is null); // Useless, it could change before next statement (not temporally safe). writeln(*global); // Error: no check has occured } ``` Asserts are not as much use as one may think. They can only check variables in a function body. You have to perform a load into a variable before you can do the test and a if statement is far better at that.
 For this reason, bounds checks do not need CT analysis.
Whether it is needed or not, the compiler can't do it.
For what we'd want agreed. However there is DFA to do this verification, so I thought it was worth mentioning.
 They do not have to bring down an application if caught.
Array overflows are fatal programming bugs. A huge discussion about this raged in the n.g. many years ago. There was a camp that maintained that assert failures should be recoverable to the point that the program could continue. The other camp maintained that when a program enters a state unanticipated by the programmer, then the program is not recoverable, because one no longer has any idea what will happen. The only path forward is to stop the program, gracefully or not. Obviously, I'm in the latter camp. Obviously, one can write programs in the first camp (D lets you do whatever you want) but I cannot endorse it.
There is a third camp. This camp is the one that I and a few others are in. It is used by application VM languages. It is also the only camp that is practical for long lived applications. You need to distinguish between "cannot cleanup" and "cannot continue request". If you cannot cleanup, then we align, shutdown process. There is no way to know what state the process is in, or what it could become. It could infect other processes and with that the user. A good example of this is the stack corruption above. If you load a pointer into a variable, and then check to see if it is null and it is not. But then when you go to dereference it, you find that it is null. Absolutely segfault out. You have stack corruption. Same situation with unmapped memory. That should not be possible, crash time! On the other hand, a bounds check failing just means your business logic is probably faulty. You cannot fulfill the request. BUT the data itself could still be cleaned up, it hasn't been corrupted. This is where exception chaining comes in, if you attempt cleanup and it turns out to throw another exception that isn't caught that means cleanup failed. The program is once again in an unknown state and should crash. ```d struct Foo { ~this() { throw new Exception; } } void callee() { try { called(); } catch(Exception) { // is chained exception, oh noes cleanup failed!!!! } } void called() { Foo foo; throw new Exception; } ``` It is important to note that requests have fixed known entry points (i.e. they continue execution of a coroutine). This isn't any random place in a code base. If you threw in the eventloop you'd still crash out.
 On the other hand, null dereference requires signal handling to catch. 
 If you don't own thread and process you cannot use that to throw an 
 exception. For this reason it's not comparable and requires CT 
 analysis to prevent bringing down the process.
Whoever catches the signal will then stop the buggy, broken program from doing more damage. It's kinda the point of a computer with memory protection.
Agreed, that is the behavior you want. HOWEVER if you need that behavior, it better be because the compiler could not have helped you in anyway to not trigger it. This should not compile: ```d func(null); void func(?nonnull int* ptr) { int val = *ptr; } ``` This sort of thing is the norm in application VM languages now. It isn't a hypothetical improvement. https://kotlinlang.org/docs/generics.html#definitely-non-nullable-types https://kotlinlang.org/docs/null-safety.html https://developer.apple.com/documentation/swift/designating-nullability-in-objective-c-apis#Annotate-Nullability-of-Individual-Declarations https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/attributes/nullable-analysis#postconditions-maybenull-and-notnull
 On the other hand, if you were to propose read barriers that throw 
 exceptions for null dereference, I would agree to the statement that 
 they are comparable.
That'll turn D into a very poorly performing language. Besides, throwing an exception is just what sig faults are. It's the same mechanism.
Agreed, I would never propose it, hence the hypothetical you. I used the idea for a comparison to bounds checks.
 Owner: Why did you use D! If you just used <insert mainstream 
 application VM language> we wouldn't have lost millions of dollars!!!
D is often unfairly accused of flaws.
As an issue, this one has had CT analysis in application VM languages for around 15 years so this one is kinda fair. Especially with D's Java heritage.
Jul 27
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
BTW,

```
void func()
{
     int* p = null;
     *p = 3;
}
```

```
dmd -c null.d -vasm
_D4null4funcFZv:
0000:   31 C0                    xor       EAX,EAX
0002:   C7 00 03 00 00 00        mov       dword ptr [RAX],3
0008:   C3                       ret
```

```
./cc -c null.d -O
Error: null dereference in function _D4null4funcFZv
```

What is happening here? D's optimizer, as a result of doing a Data Flow
Analysis 
called "constant propagation", discovers an attempt to dereference a null
pointer.

It does intra-procedural, DFA, not inter-procedural, unless the called
functions 
get inlined. (The optimizer runs after inlining.)

----------------

Let's look at gcc:

```
void func()
{
     int* p = (int*)0;
     *p = 3;
}
```

```
 gcc -c null.c
 gcc -c null.c -O
```
Jul 27
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
It would help if you turned on the analysis ;)

```c
int main(int argc, char** argv)
{
      int* p = (int*)0;
      *p = 3;
      return 0;
}
```

GCC args: ``-fanalyzer``

GCC Output:

```
<source>: In function 'main':
<source>:4:9: warning: dereference of NULL 'p' [CWE-476] 
[-Wanalyzer-null-dereference]
     4 |      *p = 3;
       |      ~~~^~~
   'main': events 1-2
     |
     |    3 |      int* p = (int*)0;
     |      |           ^
     |      |           |
     |      |           (1) 'p' is NULL
     |    4 |      *p = 3;
     |      |      ~~~~~~
     |      |         |
     |      |         (2) dereference of NULL 'p'
     |
ASM generation compiler returned: 0
<source>: In function 'main':
<source>:4:9: warning: dereference of NULL 'p' [CWE-476] 
[-Wanalyzer-null-dereference]
     4 |      *p = 3;
       |      ~~~^~~
   'main': events 1-2
     |
     |    3 |      int* p = (int*)0;
     |      |           ^
     |      |           |
     |      |           (1) 'p' is NULL
     |    4 |      *p = 3;
     |      |      ~~~~~~
     |      |         |
     |      |         (2) dereference of NULL 'p'
     |
Execution build compiler returned: 0
Program returned: 139
Program terminated with signal: SIGSEGV
```

CLANG args: ``--analyze``

CLANG Output:

```
clang: warning: -Wl,-rpath,./lib: 'linker' input unused 
[-Wunused-command-line-argument]
clang: warning: -Wl,-rpath,/opt/compiler-explorer/gcc-13.2.0/lib64: 
'linker' input unused [-Wunused-command-line-argument]
clang: warning: -Wl,-rpath,/opt/compiler-explorer/gcc-13.2.0/lib32: 
'linker' input unused [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L./lib' 
[-Wunused-command-line-argument]
<source>:4:9: warning: Dereference of null pointer (loaded from variable 
'p') [core.NullDereference]
     4 |      *p = 3;
       |       ~ ^
1 warning generated.
ASM generation compiler returned: 0
clang: warning: -Wl,-rpath,./lib: 'linker' input unused 
[-Wunused-command-line-argument]
clang: warning: -Wl,-rpath,/opt/compiler-explorer/gcc-13.2.0/lib64: 
'linker' input unused [-Wunused-command-line-argument]
clang: warning: -Wl,-rpath,/opt/compiler-explorer/gcc-13.2.0/lib32: 
'linker' input unused [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L./lib' 
[-Wunused-command-line-argument]
<source>:4:9: warning: Dereference of null pointer (loaded from variable 
'p') [core.NullDereference]
     4 |      *p = 3;
       |       ~ ^
1 warning generated.
Execution build compiler returned: 0
Program returned: 255
[F][2024-07-27T19:10:38+0000][1] runChild():487 Launching child process 
failed
```
Jul 27
prev sibling parent kdevel <kdevel vogtner.de> writes:
On Saturday, 27 July 2024 at 19:02:19 UTC, Walter Bright wrote:
 [...]
 Let's look at gcc:

 ```
 void func()
 {
     int* p = (int*)0;
     *p = 3;
 }
 ```

 ```
 gcc -c null.c
 gcc -c null.c -O
```
$ gcc -c -O -Wnull-dereference null.c null.c: In function 'func': null.c:4:8: warning: null pointer dereference [-Wnull-dereference] 4 | *p = 3; | ~~~^~~ It seems that at least optimization O1 is required to trigger the detection and -Wnull-dereference seems not to be included in -Wall -pedantic. Works since GCC 6.5.0. | -Wnull-dereference warns if the compiler detects paths that | trigger erroneous or undefined behavior due to dereferencing a null | pointer. This option is only active when -fdelete-null-pointer-checks | is active, which is enabled by optimizations in most targets. The | precision of the warnings depends on the optimization options used. [1] https://gcc.gnu.org/gcc-6/changes.html
Jul 27
prev sibling next sibling parent kdevel <kdevel vogtner.de> writes:
On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
 You're right that ref's can be null, too. C++ says they can't 
 be null, but it's trivial to make one in C++,
You can hardly produce one accidentally although it is trivial to make one. But such code is still not C++ as it wasn't in January 2023 [1]. [1] Re: Fixing C's Biggest Mistake https://forum.dlang.org/post/lzqfkfsuhoivsbbeuxmy forum.dlang.org
Jul 27
prev sibling parent Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
 You're right that ref's can be null, too. C++ says they can't 
 be null, but it's trivial to make one in C++, so a fat lot of 
 good that does.

 Let's say we have a linked list. It will have a `next` pointer:

 ```d
 struct List {
     int payload;
     List* next;
 }
 ```

 A null value is perfectly valid for `next`, as that is how the 
 end is found. At least in my coding, I use null values all the 
 time to signify there is nothing there.
Yes, in your example, by virtue of `List` being a linked list, that `null` is a valid value for `next` can be readily guessed. My suggestion would be that such a pointer would be defined as `List*?` to indicate *to the type system* that null is a valid value for it. This *forces* any function operating on lists to at least consider the possibility of a `null` pointer for `next`.
 It it wasn't null, some other value would have to be there to 
 signify "not a valid item". If there must be something there, 
 then the value would have to be checked against the "not a 
 valid item" value. What should I then do if it unexpectedly is 
 the "not a valid item"? The only sensible thing is to abort the 
 program, as it's a program bug.

 But with null, I don't have to check, because the hardware does 
 it for free.
I understand the hardware check. I really do. Most of us do. TL;DR: Let me rephrase it in terms of implicit conversions: If you have a reference-type object (e.g. a pointer) that may be null according to the type system (which, currently, is every reference type except slices), using it in a way that requires it not to be null is an implicit conversion to the non-null version of its type. (The fact that it has zero run-time cost is irrelevant.) We, the many on this thread, want this implicit conversion to be an error. That’s because it’s a wrong implicit conversion the same way calling a mutable member function on a const object would be a wrong implicit conversion from const to mutable. This does not bar anyone from using an explicit cast to assert the programmer’s wit over the rules of the type system. The other direction, non-null to nullable, is of course valid and should not require an explicit cast, same as mutable to const does not require an explicit cast. (End of TL;DR.) What we want is the type system reminding us to mind the null value where it could be a bug to ignore it. To take an analogy from the London Underground, you want to mind the gap, right? But what if you had to mind the gap on every possible step? Would you really do it? Of course you wouldn’t; it’s exhausting and wasted attention on almost every step. But sometimes, you’d trip because you actually should have minded the gap. What I’m saying is, if programmers can specify which pointers are expected to be null and which are expected not to be null, and the type system keeps track that a non-nullable pointer isn’t assigned a possibly null value without some explicit cast (which may even have zero-runtime cost with the right compiler switches, giving you core dumps or – on WebAssembly – UB), we can mind the null where it is to be minded, and rest assured that there won’t be nulls where we don’t expect them.
 The only time a null pointer dereference is an actual problem 
 is when running on a machine that does not have memory 
 protection, which are decades in obsolescence.
I don’t know myself, but people consistently point out that it’s not true. As far as I’m told, we’re here: - D can target WebAssembly and adds nullable/non-nullable annotations. - D’s null is ` safe`. Choose one.
 The real issue with null pointer seg faults is the screen dump 
 of mysterious numbers and letters.
What you’re saying is that null dereferences, which are bugs, can rather easily debugged. First, that depends on the experience of the programmer; I wouldn’t bet my life on being able to understand a core dump, let a lone that of a link-time optimized program. Second, I’d rather have a compile-error that tells me I’m risking a null dereference bug here than having to test and hopefully discover the bug before code goes to production. What I do with the error depends on circumstance. I have the following options, probably more: - Mark the left-hand side nullable. - Mark the origin of the right-hand side as non-nullable. - Handle the null case. - Insert a `cast(!null)`, risking a bug if I’m wrong. Only in the “handle the null case” does it incur a run-time cost, which I decided was actually needed and had merely forgotten to do. My suggestion is, adding two type suffixes: `T?` for indicating that `null` is a valid value and `T!` for indicating that `null` is not a valid value. Together with module defaults, that makes for a rather seamless transition. In the current state, the language default is `?`, i.e. every reference type is as if suffixed by `?`. A module default of `!null` changes that to `!`, so explicit `?` are needed. A module default only affects what is lexically in the module, not e.g. imported stuff. ```d // D tomorrow: module m; int* f(); // as if: `int*? f();` int*? g(); // same type as `f` int*! h(); // explicitly non-nullable result ``` ```d // D tomorrow: default(!null) module m; int* f(); // as if: `int*! f();`: explicitly non-nullable result int*? g(); // explicitly nullable result int*! h(); // same type as `f` ``` ```d // Possible future D edition where non-null as the language default: module m; int* f(); // as if: `int*! f();` int*? g(); // explicitly nullable result int*! h(); // same type as `f` ``` ```d // Possible future D edition where non-null as the language default: default(null) module m; int* f(); // as if: `int*? f();` explicitly non-nullable result int*? g(); // same type as `f()` int*! h(); // explicitly non-nullable result ``` This is how D could introduce non-nullable reference types (pointers, delegates, class handles, …) to the language. Only for `ref`, I’d immediately go with `ref?` is allowed to be `null`, but ordinary `ref` isn’t because practically, almost all `ref` function parameters are expected to be non-null and are bound to arguments that are obviously not `*null`. As pointed out, if D targets WebAssembly, the `cast(!null)` can’t be ` safe`. This isn’t even controversial. D can only target the WebAssembly that exists with all its flaws.
 The real biggest mistake in C is the eager decay of arrays to 
 pointers, and the tragedy of C is nobody has any interest in 
 fixing it.
I don’t disagree, but it’s unrelated.
Jul 29
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/24/2024 7:24 PM, Richard (Rikki) Andrew Cattermole wrote:
 You have almost got it.
 
 It is not any language with that capability, it is any language with that 
 capability that does not force you to check for null before the dereference.
Checking for null won't save you, as there are 4 billion addresses for a 32 bit pointer, some will seg fault, and some will overwrite your program's data with garbage.
Jul 24