digitalmars.D.learn - Returning a reference to be manipulated
- Chris Katko (25/25) Apr 13 2023 I'm trying to figure out how to return a reference to something
- IchorDev (55/80) Apr 13 2023 I'm pretty sure you can't return `ref` to a value-type without it
- Paul Backus (30/36) Apr 13 2023 You can use a wrapper struct with `alias this` to make the
- Jacob Shtokolov (29/31) Apr 13 2023 ```d
- kdevel (35/38) Apr 13 2023 Regarding the return ref I found this code of 2019 on my harddisk
- Bastiaan Veelo (5/37) Apr 14 2023 I think this is intended. Adding `@safe:` on top makes the
- kdevel (25/46) Apr 14 2023 What irritates me is that only with `return ref int`
- Dennis (44/48) Apr 14 2023 When using `return ref`, `return scope`, `scope` etc., you should
- kdevel (15/20) Apr 15 2023 We are writing at cross puroposes. I am not asking how I could
- Richard (Rikki) Andrew Cattermole (2/2) Apr 15 2023 It was bad analysis by the compiler, which has since been corrected.
- kdevel (16/19) Apr 15 2023 But why is the @unsafe programmer now left unprotected in cases
- Richard (Rikki) Andrew Cattermole (5/26) Apr 15 2023 @system (which is currently the default, there is a strong desire to
- kdevel (15/42) Apr 15 2023 When I insert `return` before `ref int` and compile the code dmd
- Dennis (17/20) Apr 15 2023 It could be done in the easy example you posted, but generalizing
I'm trying to figure out how to return a reference to something
that may not be a reference type.
```D
struct stats
{
float[string] data=0;
float ref opIndex(string key)
{
return data[key]; // want a ref to a specific element
}
}
void test()
{
stats foo;
auto x = foo.bar(); // returns some sort of reference
// data["tacos"] = 0
x++; // data["tacos"] = 1
}
```
Right now, I'm using pointers which resolves to:
```D
// float* opIndex(string key){...} using pointer
(*s["tacos"])++; // works with pointer, but is strange looking
s["tacos"]++; // preferred syntax or something similar
```
Apr 13 2023
IchorDev <zxinsworld gmail.com>
On Thursday, 13 April 2023 at 07:05:10 UTC, Chris Katko wrote:
I'm trying to figure out how to return a reference to something
that may not be a reference type.
```D
struct stats
{
float[string] data=0;
float ref opIndex(string key)
{
return data[key]; // want a ref to a specific element
}
}
void test()
{
stats foo;
auto x = foo.bar(); // returns some sort of reference
// data["tacos"] = 0
x++; // data["tacos"] = 1
}
```
Right now, I'm using pointers which resolves to:
```D
// float* opIndex(string key){...} using pointer
(*s["tacos"])++; // works with pointer, but is strange looking
s["tacos"]++; // preferred syntax or something similar
```
I'm pretty sure you can't return `ref` to a value-type without it
being a pointer, or a slice.
Unlike C++, D does not have `ref` variables (`&`). You can have
value-type function parameters that are `ref` though.
But I feel that the easier solution for the example you gave
would be to simply save the string used to index the float,
rather than a reference to the float itself:
```d
import std.stdio: writeln;
struct Stats{
float[string] data;
float opIndex(string key){ return data[key]; }
float opIndexAssign(float val, string key){ data[key] = val;
return val; }
}
void main(){
Stats foo;
foo.data = ["tacos": 0];
string ind = "tacos";
float x = foo[ind];
writeln(foo.data[ind]); //0
x++;
foo[ind] = x;
writeln(foo.data[ind]); //1
}
```
Passing pointers around might be faster, but it's error-prone,
and cumbersome for simple operators as you mentioned. However,
you could always wrap your pointer in a struct with some
necessary operator overloads:
```d
import std.stdio: writeln;
struct Float{
float* data;
float opUnary(string op: "++")(){ (*data)++; return *data; }
}
struct Stats{
float[string] data;
Float opIndex(string key){ return Float(key in data); }
}
void main(){
Stats foo;
foo.data = ["tacos": 0];
Float x = foo["tacos"];
writeln(foo.data["tacos"]); //0
x++;
writeln(foo.data["tacos"]); //1
}
```
Apr 13 2023
On Thursday, 13 April 2023 at 07:05:10 UTC, Chris Katko wrote:
Right now, I'm using pointers which resolves to:
```D
// float* opIndex(string key){...} using pointer
(*s["tacos"])++; // works with pointer, but is strange looking
s["tacos"]++; // preferred syntax or something similar
```
You can use a wrapper struct with `alias this` to make the
pointer a little nicer to use:
```d
struct Ref(T)
{
T* ptr;
ref inout(T) deref() inout { return *ptr; }
alias deref this;
}
Ref!T byRef(T)(ref T obj)
{
return Ref!T(&obj);
}
struct stats
{
float[string] data;
Ref!float opIndex(string key)
{
return data.require(key, 0).byRef;
}
}
void main()
{
stats foo;
auto x = foo["tacos"];
x++;
assert(foo["tacos"] == 1);
}
```
Apr 13 2023
On Thursday, 13 April 2023 at 07:05:10 UTC, Chris Katko wrote:I'm trying to figure out how to return a reference to something that may not be a reference type.```d safe: struct Stats { float[string] data; ref opIndex(string key) return { // The `require()` takes care of non-existing values, initializing them return data.require(key, 0); } } void main() { import std.stdio; Stats foo; // 1. You can't "capture" a reference, only use it directly foo["tacos"] += 2; // 2. You can use an alternative function-calling form (sometimes replaced by the `with` operator) ((ref float val) => val += 2)(foo["burritos"]); // 3. As a last resort, use pointers or make struct wrappers around them auto x = &foo["quesadillas"]; *x += 2; writeln(foo); } ```
Apr 13 2023
On Thursday, 13 April 2023 at 22:09:48 UTC, Jacob Shtokolov wrote:
[...]
ref opIndex(string key) return
[...]
Regarding the return ref I found this code of 2019 on my harddisk
which is from or was motivated by a dconf talk:
```
ref int foo (ref int i)
{
return i;
}
ref int bar ()
{
int i;
return foo (i);
}
void main ()
{
import std.stdio;
auto i = bar;
i.writeln;
}
```
Up to dmd v2.100.2 I am warned/get an error during compilation:
```
$ dmd returnref2.d
returnref2.d(3): Deprecation: returning `i` escapes a reference
to parameter `i`
returnref2.d(1): perhaps annotate the parameter with
`return`
$ dmd -dip1000 returnref2.d
returnref2.d(3): Error: returning `i` escapes a reference to
parameter `i`
returnref2.d(1): perhaps annotate the parameter with
`return`
```
With later dmd versions (up to including v2.102.2) the code
compiles without complaints. Is this intended?
Apr 13 2023
On Friday, 14 April 2023 at 00:50:31 UTC, kdevel wrote:
```
ref int foo (ref int i)
{
return i;
}
ref int bar ()
{
int i;
return foo (i);
}
void main ()
{
import std.stdio;
auto i = bar;
i.writeln;
}
```
Up to dmd v2.100.2 I am warned/get an error during compilation:
```
$ dmd returnref2.d
returnref2.d(3): Deprecation: returning `i` escapes a reference
to parameter `i`
returnref2.d(1): perhaps annotate the parameter with
`return`
$ dmd -dip1000 returnref2.d
returnref2.d(3): Error: returning `i` escapes a reference to
parameter `i`
returnref2.d(1): perhaps annotate the parameter with
`return`
```
With later dmd versions (up to including v2.102.2) the code
compiles without complaints. Is this intended?
I think this is intended. Adding ` safe:` on top makes the
complaint come back (in dmd 2.102 it is deprecated, in 2.103 it
is an error).
-- Bastiaan.
Apr 14 2023
On Friday, 14 April 2023 at 09:42:14 UTC, Bastiaan Veelo wrote:What irritates me is that only with `return ref int` ``` ref int foo (return ref int i) { return i; } [...] ``` dmd complains: ``` returnref2.d(9): Error: returning `foo(i)` escapes a reference to local variable `i` ``` The documentation [1] says that a `ref` parameter may not be returned unless it is `return ref`. But in fact it is returned unless it is `return ref`. Probably i get/got the meaning of the English modal verb "may" wrong. The documentation also says that `return ref` parameters are used to ensure that the returned reference will not outlive the matching argument's lifetime. I don't get it! Is there any legitimate use of returning a ref such that it outlives the matching argument's lifetime? If not: Isn't this `return ref` completely redundant? [1] https://dlang.org/spec/function.html[...] Up to dmd v2.100.2 I am warned/get an error during compilation: ``` $ dmd returnref2.d returnref2.d(3): Deprecation: returning `i` escapes a reference to parameter `i` returnref2.d(1): perhaps annotate the parameter with `return` $ dmd -dip1000 returnref2.d returnref2.d(3): Error: returning `i` escapes a reference to parameter `i` returnref2.d(1): perhaps annotate the parameter with `return` ``` With later dmd versions (up to including v2.102.2) the code compiles without complaints. Is this intended?I think this is intended. Adding ` safe:` on top makes the complaint come back (in dmd 2.102 it is deprecated, in 2.103 it is an error).
Apr 14 2023
On Friday, 14 April 2023 at 10:31:58 UTC, kdevel wrote:But in fact it is returned unless it is `return ref`.When using `return ref`, `return scope`, `scope` etc., you should be using the latest compiler and annotate functions you want checked with ` safe`. In previous versions, the compiler would often conflate `return ref` and `return scope`, and it was also inconsistent in whether it would do checks in ` safe`, ` system`, and even 'default/unannotated' functions. Now, it is more consistent, performing checks in ` safe` code only.I don't get it! Is there any legitimate use of returning a ref such that it outlives the matching argument's lifetime? If not: Isn't this `return ref` completely redundant?The annotation is needed because the compiler can't always figure out what you're doing with a `ref` parameter: ```D ref int mysteryFunc(ref int x) safe; // external implementation ref int escape() safe { int local; // allocated on stack frame, should not escape this function return mysteryFunc(local); // is this safe? } ``` Is this indeed ` safe`? It is, provided that `mysteryFunc` doesn't return its parameter `x`. It can be implemented like this for example: ```D ref int mysteryFunc(ref int x) safe { x++; return *(new int); } ``` But it wouldn't be safe if `x` were returned, so the compiler must know about that when it happens, hence `return ref`: ```D ref int mysteryFunc(return ref int x) safe { return x; } ``` Now the compiler can catch that `return mysteryFunc(local)` is unsafe. Note that if `mysteryFunc` is a template function, nested function, or returns `auto`, then the compiler infers attributes automatically, including `return ref`. Then you can still write it as `mysteryFunc(ref int x)` and it will automatically be treated as `return ref`.
Apr 14 2023
On Friday, 14 April 2023 at 11:18:21 UTC, Dennis wrote:On Friday, 14 April 2023 at 10:31:58 UTC, kdevel wrote:We are writing at cross puroposes. I am not asking how I could help the compiler. My point is: I have code from 2019 which produced a deprecation warning ``` returnref2.d(3): Deprecation: returning i escapes a reference to parameter i, perhaps annotate with return ``` when compiled with with the compiler of that time (v2.093.1). When I use a recent compiler (v2.102.2) the code compiles without any complaints. I am asking myself what has deprecacted? I mean: Deprecation notes are built into the compiler in order to enable a "soft landing" for a scheduled breaking change. But I cannot see any breaking change at all regarding the code example in https://forum.dlang.org/post/iysfuhjwyalnnmalbdrh forum.dlang.orgBut in fact it is returned unless it is `return ref`.When using `return ref`, `return scope`, `scope` etc., you should be using the latest compiler and annotate functions you want checked with ` safe`.
Apr 15 2023
It was bad analysis by the compiler, which has since been corrected. It should have been applied only to safe code.
Apr 15 2023
On Saturday, 15 April 2023 at 12:45:31 UTC, Richard (Rikki) Andrew Cattermole wrote:It was bad analysis by the compiler, which has since been corrected. It should have been applied only to safe code.But why is the unsafe programmer now left unprotected in cases like this ``` ref int foo (ref int i) // `------- automatically insert `return` here? { return i; } ``` where the compiler recognizes that the function returns a ref parameter? Under which circumstances is it a mistake to insert the `return` at the indicated position? If there are none why can't it be done implicitly (automatically)?
Apr 15 2023
On 16/04/2023 1:20 AM, kdevel wrote:On Saturday, 15 April 2023 at 12:45:31 UTC, Richard (Rikki) Andrew Cattermole wrote:system (which is currently the default, there is a strong desire to change this), does no safety checks. You are on your own to do whatever unsafe thing you want. If you want safety checks, use safe.It was bad analysis by the compiler, which has since been corrected. It should have been applied only to safe code.But why is the unsafe programmer now left unprotected in cases like this ``` ref int foo (ref int i) // `------- automatically insert `return` here? { return i; } ``` where the compiler recognizes that the function returns a ref parameter? Under which circumstances is it a mistake to insert the `return` at the indicated position? If there are none why can't it be done implicitly (automatically)?
Apr 15 2023
On Saturday, 15 April 2023 at 13:24:52 UTC, Richard (Rikki) Andrew Cattermole wrote:On 16/04/2023 1:20 AM, kdevel wrote:When I insert `return` before `ref int` and compile the code dmd says ``` returnref2.d(9): Error: returning `foo(i)` escapes a reference to local variable `i` ``` Isn't this a safety check? (The code does not contain any ` `.) My question was if it is problematic if dmd inserted a `return` before any `ref` parameter a function returns. Did the removed code which checked for returning ref parameters have false positives? If not again my question: Under what circumstances would it be a mistake to insert `return` before the respective `ref` parameter?On Saturday, 15 April 2023 at 12:45:31 UTC, Richard (Rikki) Andrew Cattermole wrote:system (which is currently the default, there is a strong desire to change this), does no safety checks.It was bad analysis by the compiler, which has since been corrected. It should have been applied only to safe code.But why is the unsafe programmer now left unprotected in cases like this ``` ref int foo (ref int i) // `------- automatically insert `return` here? { return i; } ``` where the compiler recognizes that the function returns a ref parameter? Under which circumstances is it a mistake to insert the `return` at the indicated position? If there are none why can't it be done implicitly (automatically)?
Apr 15 2023
On Saturday, 15 April 2023 at 13:20:09 UTC, kdevel wrote:Under which circumstances is it a mistake to insert the `return` at the indicated position? If there are none why can't it be done implicitly (automatically)?It could be done in the easy example you posted, but generalizing it is harder. When importing a module, the compiler currently doesn't need to analyze function bodies to get the signature of regular (non-auto/template) functions, which would have to change. Programmers can also currently rely on the fact that the signature they see is the signature they get, but not any longer. The identity function is really simple, but as soon as control flow (if-statements) come into play, the annotations and their inference become a conservative approximation, which might give false positives in ` system` code. There would need to be a second system, one which assumes the best instead of assuming the worst. This adds complexity, just to add some 'intermediate' safety between ` system` and ` safe` in a few cases. It's better to keep the rules simple and consistent.
Apr 15 2023
On Saturday, 15 April 2023 at 14:10:57 UTC, Dennis wrote:This adds complexity, just to add some 'intermediate' safety between ` system` and ` safe` in a few cases. It's better to keep the rules simple and consistent.To quote my past self:There used to be different rules for lifetime errors in all of these: - explicit ` system` functions - ` system` by default functions (yes, [they were special](https://issues.dlang.org/show_bug.cgi?id=19873)) - inferred functions - ` safe` functions - ` trusted` functions It was really complex and confusing, and I've worked on simplifying it such that all lifetime errors are safety violations like any other. The only exception is directly returning a dangling pointer to a stack variable, which is just an error even in system code ([issue 19873](https://issues.dlang.org/show_bug.cgi?id=19873)). I don't want to go back to more special cases, especially with the dip1000 by default transition which is complex enough as is.https://forum.dlang.org/post/nzotevvzvbpqscfxsuod forum.dlang.org
Apr 15 2023
On Saturday, 15 April 2023 at 14:10:57 UTC, Dennis wrote:[...] This adds complexity, just to add some 'intermediate' safety between ` system` and ` safe` in a few cases. It's better to keep the rules simple and consistent.Thanks for the answer! While playing around with return ref I came across this: ``` string foo (string s, return ref int i) { return s; } auto bar () { int i; string s; return foo (s, i); } ``` ``` $ dmd rr.d rr.d(10): Error: returning `foo(s, i)` escapes a reference to local variable `i` ``` Does that make sense?
Apr 15 2023
On Saturday, 15 April 2023 at 14:33:52 UTC, kdevel wrote:Does that make sense?Whether it makes sense is subjective, but it is by design. Escape analysis considers every pointer the same, it doesn't care about the type / mutability of the pointer. In ` system` / ` trusted` code, you could coerce `i` to become a string and return it: ```D string foo(string s, return ref int i) { return (cast(immutable char*) &i)[0..4]; } ```
Apr 15 2023
On Saturday, 15 April 2023 at 15:50:18 UTC, Dennis wrote:[...] care about the type / mutability of the pointer.Returning `i`'s address in a long does not trigger the escape detector: ``` long foo (long s, return ref int i) { s = cast (long) &i; return s; } auto bar () { int i; long s; return foo (s, i); } ``` dmd compiles this without complaints.
Apr 15 2023
On Saturday, 15 April 2023 at 21:00:01 UTC, kdevel wrote:On Saturday, 15 April 2023 at 15:50:18 UTC, Dennis wrote:It doesn't care about the type of pointer, but it does care about whether the type is/has a pointer in the first place. `T*`, `T[]`, `K[V]` (Associative arrays), `class`, `function`, `delegate` are pointers. Static arrays and structs depend on what they contain. Basic types such as `long` are not pointers, so lifetime checking doesn't apply.[...] care about the type / mutability of the pointer.Returning `i`'s address in a long does not trigger the escape detector:
Apr 16 2023